DataGemma RAG 27B GGUF
| Property | Value |
|---|---|
| Parameter Count | 27.2B |
| License | Gemma |
| Base Model | google/datagemma-rag-27b-it |
| Quantization | Multiple GGUF variants |
What is datagemma-rag-27b-it-GGUF?
DataGemma RAG 27B GGUF is a comprehensive collection of quantized versions of Google's DataGemma model, optimized for different hardware configurations and use cases. The model offers various quantization levels from full F16 precision down to highly compressed formats, enabling users to balance between model quality and resource requirements.
Implementation Details
The model uses llama.cpp for quantization and offers multiple specialized formats including K-quants and I-quants. It features unique optimization for ARM inference and special handling of embedding/output weights in certain variants.
- Supports multiple quantization levels from F16 (54.46GB) to IQ2_XXS (7.63GB)
- Specialized formats for ARM chips with sve/i8mm support
- Enhanced versions with Q8_0 quantization for embed and output weights
Core Capabilities
- Text generation with specialized prompt format
- Optimized inference on various hardware configurations
- Flexible deployment options based on available RAM/VRAM
- Support for both CPU and GPU acceleration
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its comprehensive range of quantization options, allowing users to choose the perfect balance between model size and performance for their specific hardware setup. It includes cutting-edge I-quant formats for optimal performance on modern GPUs.
Q: What are the recommended use cases?
For optimal performance, Q6_K_L and Q5_K_M are recommended for high-quality results. Users with limited resources can opt for I-quants like IQ4_XS, which offers good performance at smaller sizes. For ARM devices, specialized Q4_0_X_X variants provide substantial speedups.
