Wizard Mega 13B GPTQ
| Property | Value |
|---|---|
| Parameter Count | 13B |
| Model Type | LLaMA-based |
| Quantization | 4-bit GPTQ |
| License | Other |
| Size | 7.45 GB |
What is wizard-mega-13B-GPTQ?
Wizard Mega 13B GPTQ is a quantized version of the original Wizard Mega model, optimized for efficient deployment while maintaining high performance. It's built on the LLaMA 13B architecture and fine-tuned on a carefully curated combination of ShareGPT, WizardLM, and Wizard-Vicuna datasets, with responses filtered to remove typical AI disclaimers and refusals.
Implementation Details
The model utilizes 4-bit quantization with a group size of 128 and no act-order, making it compatible with ExLlama and various GPTQ implementations. It's optimized for a 2048 token context window and uses the Vicuna prompt template for consistent interaction.
- GPTQ quantization parameters: 4-bit, GS=128
- Model size reduced to 7.45GB from original
- Compatible with AutoGPTQ and Transformers libraries
- Trained using Axolotl on 8xA100 80GB GPUs
Core Capabilities
- Comprehensive instruction following and chat interactions
- Code generation and technical explanations
- Detailed and polite responses to complex queries
- Efficient deployment on consumer hardware due to quantization
Frequently Asked Questions
Q: What makes this model unique?
This model combines the strengths of multiple high-quality datasets with efficient quantization, making it particularly suitable for deployment on consumer hardware while maintaining strong performance across a wide range of tasks.
Q: What are the recommended use cases?
The model excels at general dialogue, technical writing, code generation, and detailed explanations. It's particularly well-suited for applications requiring balanced performance and efficiency.
