Dolphin-2.1-Mistral-7B
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| License | Apache 2.0 |
| Training Time | 48 hours on 4x A100s |
| Format | ChatML |
| Tensor Type | BF16 |
What is dolphin-2.1-mistral-7b?
Dolphin-2.1-mistral-7b is an advanced language model built on MistralAI's architecture, specifically designed to be highly compliant while maintaining creative capabilities. Sponsored by a16z, this model represents an implementation of Microsoft's Orca approach, trained on a carefully curated dataset combining Dolphin and Airoboros data.
Implementation Details
The model underwent extensive training for 4 epochs using 4 A100 GPUs, implementing the ChatML prompt format for consistency and optimal performance. It's built using PyTorch and features Safetensors implementation for efficient handling of model weights.
- Uncensored architecture with filtered dataset removing alignment and bias
- Combined training on Dolphin and Airoboros 2.2.1 datasets
- Implements ChatML prompt format for standardized interactions
- Optimized for both commercial and non-commercial applications
Core Capabilities
- Advanced text generation and conversation handling
- High compliance to user requests and instructions
- Strong performance in various benchmark tests (53.47 average score)
- Excellent performance in HellaSwag (84.92) and Winogrande (77.74)
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its uncensored nature and high compliance, combined with the robust Mistral architecture and carefully curated training data. It's particularly notable for its commercial-friendly license and balanced performance across various tasks.
Q: What are the recommended use cases?
The model is suitable for a wide range of applications including conversational AI, text generation, and complex reasoning tasks. However, users should implement their own alignment layer before deploying it as a service due to its uncensored nature.
