Wizard-Vicuna-13B-Uncensored-GPTQ
| Property | Value |
|---|---|
| Parameter Count | 13B |
| Model Type | LLaMA Architecture |
| Quantization | 4-bit GPTQ |
| License | Other |
| Language | English |
What is Wizard-Vicuna-13B-Uncensored-GPTQ?
This is a quantized version of Eric Hartford's Wizard-Vicuna model, specifically optimized for efficient deployment while maintaining high performance. The model represents a 4-bit quantized version of the original 13B parameter model, compressed using GPTQ technology to achieve significant size reduction while preserving model capabilities.
Implementation Details
The model implements multiple quantization parameters across different branches, offering users flexibility in choosing the optimal version for their hardware requirements. The base implementation includes group size variations (128) and Act Order options, resulting in a compressed size of 8.11GB.
- Multiple branch options with varying quantization parameters
- 4-bit precision with configurable group sizes
- ExLlama compatibility for supported configurations
- Optimized for 2048 sequence length
Core Capabilities
- Unrestricted text generation and conversation
- Detailed and comprehensive responses
- Efficient deployment with reduced memory footprint
- Multiple inference options including text-generation-webui support
Frequently Asked Questions
Q: What makes this model unique?
This model offers an uncensored version of the Wizard-Vicuna architecture, specifically designed without built-in alignment constraints, allowing users to implement custom alignment approaches. The GPTQ quantization makes it particularly suitable for deployment on consumer hardware.
Q: What are the recommended use cases?
The model is optimized for conversational AI applications and text generation tasks where unrestricted outputs are desired. It's particularly suitable for scenarios requiring detailed, comprehensive responses while operating under memory constraints.
