Synthia-7B-v1.3-GPTQ
| Property | Value |
|---|---|
| Base Model | Mistral-7B-v0.1 |
| License | Apache 2.0 |
| Paper | Orca Paper |
| Quantization | GPTQ 4-bit |
What is Synthia-7B-v1.3-GPTQ?
Synthia-7B-v1.3-GPTQ is a quantized version of the Synthia model, based on Mistral-7B architecture and trained on Orca-style datasets. This GPTQ variant maintains the model's capabilities while reducing its size and memory requirements through 4-bit quantization, making it more accessible for consumer hardware.
Implementation Details
The model utilizes GPTQ quantization with multiple parameter options, including 4-bit and 8-bit versions with different group sizes (32g, 64g, 128g). The implementation achieves impressive benchmark scores, including 0.6237 on ARC Challenge, 0.8349 on HellaSwag, and 0.6232 on MMLU.
- Multiple quantization options for different hardware requirements
- Optimized for both instruction following and conversation
- Implements Tree of Thought reasoning capabilities
- Compatible with various inference frameworks including AutoGPTQ and ExLlama
Core Capabilities
- Uncensored, detailed response generation
- Long-form conversation handling
- Tree of Thought reasoning
- Factual accuracy with benchmark-proven performance
- Flexible deployment options across different hardware configurations
Frequently Asked Questions
Q: What makes this model unique?
Synthia-7B-v1.3-GPTQ combines the powerful Mistral architecture with Orca-style training and efficient quantization, offering a balanced mix of performance and accessibility. Its uncensored nature and Tree of Thought reasoning capabilities make it particularly suitable for detailed, analytical responses.
Q: What are the recommended use cases?
The model excels in instruction following, detailed explanations, and long-form conversations. It's particularly well-suited for applications requiring analytical reasoning, technical discussions, and comprehensive response generation while maintaining reasonable hardware requirements due to its quantization.
