Dolphin 2.5 Mixtral 8x7B GPTQ
| Property | Value |
|---|---|
| Model Size | 6.09B parameters |
| License | Apache 2.0 |
| Author | TheBloke (Quantized) / Eric Hartford (Original) |
| Tensor Type | I32, BF16, FP16 |
What is dolphin-2.5-mixtral-8x7b-GPTQ?
This is a GPTQ-quantized version of the Dolphin 2.5 Mixtral model, specifically optimized for efficient deployment while maintaining high performance. The model stands out for its exceptional coding capabilities and general-purpose use, trained on multiple high-quality datasets including OpenHermes, Magicoder, and PureDove.
Implementation Details
The model offers multiple quantization options, from 3-bit to 8-bit versions, with various group sizes to accommodate different hardware configurations. It uses the ChatML prompt format and requires Transformers 4.36.0 or later, along with specific AutoGPTQ requirements.
- Multiple GPTQ variants available (3-bit, 4-bit, 8-bit)
- Context length of 8192 tokens
- Optimized for both coding and general tasks
- Supports various group sizes for different VRAM requirements
Core Capabilities
- Advanced coding assistance and generation
- General conversation and task completion
- Structured output handling
- High compliance and instruction-following
- Multi-turn dialogue support
Frequently Asked Questions
Q: What makes this model unique?
This model combines the power of Mixtral's architecture with specific optimizations for coding tasks, while maintaining strong general-purpose capabilities. Its various quantization options make it highly accessible across different hardware setups.
Q: What are the recommended use cases?
The model excels in coding tasks, general conversation, and structured output generation. It's particularly well-suited for developers needing coding assistance, technical writing, and complex problem-solving tasks.
