Llama-3.1-Swallow-8B-Instruct-v0.2
| Property | Value |
|---|---|
| Parameter Count | 8.03B |
| Model Type | Llama Architecture |
| License | META LLAMA 3.1 COMMUNITY LICENSE & Gemma Terms of Use |
| Languages | Japanese, English |
| Training Framework | Megatron-LM |
What is Llama-3.1-Swallow-8B-Instruct-v0.2?
Llama-3.1-Swallow-8B-Instruct-v0.2 is an advanced language model that enhances the Japanese language capabilities of Meta's Llama 3.1 while maintaining strong English performance. Built through continual pre-training on approximately 200 billion tokens from Japanese web corpus, Wikipedia articles, and specialized content, this model represents a significant step forward in bilingual AI capabilities.
Implementation Details
The model leverages the Llama 3.1 architecture and has undergone extensive instruction tuning using carefully curated datasets. It's implemented using the Megatron-LM framework and supports both Japanese and English text generation tasks.
- Continual pre-training from meta-llama/Llama-3.1-8B-Instruct
- Specialized instruction tuning with Japanese-focused datasets
- Supports BF16 tensor operations
- Compatible with vLLM for efficient inference
Core Capabilities
- Strong performance in Japanese tasks (achieving state-of-the-art results in multiple benchmarks)
- Maintained English language capabilities comparable to base Llama 3.1
- Excels in Japanese-English translation tasks
- Enhanced performance in mathematical reasoning and academic tasks
- Robust code generation capabilities in both languages
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines strong Japanese language capabilities with maintained English performance, making it especially valuable for bilingual applications. It shows particularly strong results in Japanese benchmarks while retaining competitive performance in English tasks.
Q: What are the recommended use cases?
The model is well-suited for bilingual applications including translation, content generation, question-answering, and code generation. It's particularly effective for Japanese language tasks while maintaining strong capabilities in English, making it ideal for applications requiring robust performance in both languages.
