OpenHermes 2.5 Mistral 7B AWQ
| Property | Value |
|---|---|
| Model Size | 7B parameters (4-bit quantized) |
| Architecture | Mistral |
| License | Apache 2.0 |
| Format | AWQ (4-bit precision) |
| Authors | Base by Teknium, Quantized by TheBloke |
What is OpenHermes-2.5-Mistral-7B-AWQ?
OpenHermes 2.5 Mistral 7B AWQ is a quantized version of an advanced language model that combines the powerful Mistral architecture with extensive training on high-quality instruction data. This AWQ variant reduces the model size while maintaining performance, making it more accessible for deployment.
Implementation Details
The model uses the ChatML format for interactions and has been quantized using AWQ (Activation-aware Weight Quantization) to 4-bit precision, resulting in a significantly smaller file size while preserving model quality. It was trained on 1,000,000 entries of primarily GPT-4 generated data and other high-quality datasets.
- Achieves 50.7% pass rate on HumanEval coding benchmarks
- Implements ChatML format for structured dialogue
- Supports system prompts for consistent behavior
- Compatible with text-generation-webui, vLLM, and Hugging Face's TGI
Core Capabilities
- Enhanced code understanding and generation
- Strong performance on reasoning benchmarks (73.12% on GPT4All)
- Improved truthfulness (53.04% on TruthfulQA)
- Efficient 4-bit quantized format for reduced memory usage
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines Mistral's architecture with extensive instruction tuning and efficient quantization, offering a balance between performance and resource requirements. The inclusion of code-specific training has notably improved both coding and general reasoning capabilities.
Q: What are the recommended use cases?
The model excels at coding tasks, general dialogue, and reasoning problems. It's particularly suitable for deployment in resource-constrained environments thanks to its AWQ quantization, while maintaining strong performance across various tasks.
