Llama-3-SauerkrautLM-8b-Instruct
| Property | Value |
|---|---|
| Parameter Count | 8.03B |
| Model Type | Instruction-tuned LLM |
| Languages | German, English |
| License | Meta Llama 3 |
| Training Method | Two-stage DPO Fine-tuning |
What is Llama-3-SauerkrautLM-8b-Instruct?
Llama-3-SauerkrautLM-8b-Instruct is a collaborative effort between VAGO Solutions and Hyperspace.ai, built upon Meta's Llama-3-8B-Instruct base model. This specialized variant has been optimized for both German and English language processing through a sophisticated two-stage DPO (Direct Preference Optimization) fine-tuning process.
Implementation Details
The model underwent a carefully designed training procedure involving two epochs: the first with 70k data points and the second with 20k data points. This approach has resulted in notable improvements in German language capabilities while maintaining strong performance in English.
- Achieves 74.57 average score on standard benchmarks
- MT-Bench scores of 7.90 for English and 7.66 for German
- Exceptional 91% accuracy on German RAG evaluation tasks
Core Capabilities
- Bilingual instruction following in German and English
- Strong performance in context-based reasoning tasks
- Enhanced German language understanding and generation
- Robust performance across multiple evaluation benchmarks
Frequently Asked Questions
Q: What makes this model unique?
Its specialized two-stage DPO fine-tuning process and optimized performance for both German and English language tasks, while maintaining the core capabilities of Llama-3 architecture.
Q: What are the recommended use cases?
The model excels in bilingual applications requiring German and English language processing, including content generation, question-answering, and context-based reasoning tasks. It's particularly suitable for applications requiring strong German language capabilities.
