Starling-LM-7B-alpha-GGUF
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Model Type | Mistral-based LLM |
| License | CC-BY-NC-4.0 |
| Base Model | OpenChat 3.5 |
| Research Paper | Link |
What is Starling-LM-7B-alpha-GGUF?
Starling-LM-7B-alpha is a cutting-edge language model developed by Berkeley-Nest, fine-tuned using Reinforcement Learning from AI Feedback (RLAIF). This GGUF version, quantized by TheBloke, makes the model more accessible for various deployment scenarios while maintaining impressive performance. The model achieves an remarkable 8.09 score on MT-Bench, surpassing most existing models except GPT-4 and GPT-4 Turbo.
Implementation Details
The model is available in multiple quantization formats ranging from 2-bit to 8-bit, offering different trade-offs between model size and performance. The recommended Q4_K_M variant provides a balanced option at 4.37GB file size.
- Multiple quantization options (Q2_K through Q8_0)
- Compatible with llama.cpp and various UI implementations
- Supports context length up to 8192 tokens
- Uses OpenChat prompt template format
Core Capabilities
- Strong performance on MT-Bench (8.09 score)
- 91.99% on AlpacaEval benchmark
- 63.9% on MMLU testing
- Efficient GPU layer offloading support
- Optimized for both CPU and GPU inference
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its use of RLAIF (Reinforcement Learning from AI Feedback) and its impressive benchmark scores that rival much larger models. It's built on the Nectar dataset and uses advanced reward training and policy tuning pipelines.
Q: What are the recommended use cases?
The model is well-suited for general language tasks, chat applications, and complex reasoning. It's particularly effective when deployed with GPU acceleration using the Q4_K_M quantization, offering a good balance of performance and resource usage.
