AlphaMonarch-7B
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Model Type | Text Generation |
| License | CC-BY-NC-4.0 |
| Context Length | 8K tokens |
What is AlphaMonarch-7B?
AlphaMonarch-7B is a sophisticated language model that represents a significant advancement in combining reasoning capabilities with conversational abilities. It's a DPO-tuned version of NeuralMonarch-7B, created through a careful merge of multiple high-performing models using LazyMergekit technology.
Implementation Details
The model is built on the Mistral architecture and has been fine-tuned using OpenHermes2.5-dpo-binarized-alpha preference dataset. It employs FP16 precision and supports an 8K token context window, making it suitable for extended conversations and complex reasoning tasks.
- Achieves state-of-the-art performance on multiple benchmarks including AGIEval, GPT4All, and TruthfulQA
- Implements the Mistral Instruct chat template for optimal interaction
- Available in multiple quantized versions (GGUF, GPTQ, AWQ, EXL2)
Core Capabilities
- Advanced reasoning and instruction following
- Sophisticated conversational abilities
- Strong performance in mathematical reasoning (66.72% accuracy on GSM8k)
- High truthfulness scores (77.91% on TruthfulQA)
- Excellent common sense reasoning (89.18% on HellaSwag)
Frequently Asked Questions
Q: What makes this model unique?
AlphaMonarch-7B stands out for achieving exceptional performance across both reasoning and conversational tasks, surpassing many larger models including some 70B and 120B parameter variants on certain benchmarks. It maintains a formal and sophisticated communication style while being highly adaptable through prompt engineering.
Q: What are the recommended use cases?
The model excels in instruction following, reasoning tasks, conversations, role-playing, and storytelling. It's particularly well-suited for applications requiring both analytical thinking and natural dialogue capabilities. The recommended inference parameters include temperature 0.8, top_k 40, top_p 0.95, min_p 0.05, and repeat_penalty 1.1.
