Dolphin 2.0 Mistral 7B GPTQ
| Property | Value |
|---|---|
| Base Model | Mistral 7B |
| License | Apache 2.0 |
| Quantization | GPTQ (Multiple Options) |
| Training Time | 48 hours on 4x A100s |
What is dolphin-2.0-mistral-7B-GPTQ?
Dolphin 2.0 Mistral 7B GPTQ is a quantized version of Eric Hartford's Dolphin model, based on Microsoft's Orca approach. It's an uncensored, highly capable model that combines the Mistral architecture with specialized training for enhanced compliance and versatility. The model uses the ChatML format and offers multiple quantization options for different hardware configurations.
Implementation Details
The model implements GPTQ quantization with various configurations, ranging from 4-bit to 8-bit precision, with different group sizes (32g, 64g, 128g) and Act Order options. It's trained on a combination of the Dolphin dataset and Jon Durbin's Airoboros dataset for improved creativity and performance.
- Multiple GPTQ quantization options for different hardware requirements
- ChatML prompt format implementation
- Compatible with AutoGPTQ, Transformers, and ExLlama
- Provides 4-bit and 8-bit versions with varying group sizes
Core Capabilities
- Enhanced compliance and response generation
- Optimized for both commercial and non-commercial use
- Efficient memory usage through quantization
- Support for long context windows up to 4096 tokens
Frequently Asked Questions
Q: What makes this model unique?
This model combines the powerful Mistral architecture with specialized training for enhanced compliance, while offering multiple quantization options for different hardware setups. It's particularly notable for its uncensored nature and flexible deployment options.
Q: What are the recommended use cases?
The model is suitable for various applications requiring helpful AI assistance, from general text generation to specialized tasks. However, users should implement their own alignment layer before deploying it as a service due to its uncensored nature.
