Tiny-Vicuna-1B
| Property | Value |
|---|---|
| Parameter Count | 1.1B |
| License | Apache 2.0 |
| Tensor Type | F32 |
| Downloads | 37,461 |
What is Tiny-Vicuna-1B?
Tiny-Vicuna-1B is a compact language model that represents a fine-tuned version of TinyLlama, specifically optimized using the WizardVicuna Dataset. This model maintains full compatibility with the Vicuna-v1.5 series while offering a more efficient architecture with just 1.1B parameters.
Implementation Details
The model utilizes PyTorch framework and implements the transformer architecture, with support for text generation tasks. It's distributed in Safetensors format and includes text-generation-inference capabilities.
- Achieves 34.76% average performance across standard benchmarks
- Implements few-shot learning capabilities (5-25 shots depending on task)
- Optimized for English language processing
Core Capabilities
- HellaSwag (10-Shot): 55.92% normalized accuracy
- Winogrande (5-shot): 58.41% accuracy
- MMLU (5-Shot): 25.45% accuracy
- TruthfulQA (0-shot): 33.82% MC2 score
- AI2 Reasoning Challenge: 33.45% normalized accuracy
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient architecture while maintaining compatibility with Vicuna-v1.5. It's particularly suitable for early experiments and iterations due to its smaller size and balanced performance across various tasks.
Q: What are the recommended use cases?
The model is well-suited for text generation tasks, particularly in scenarios requiring few-shot learning. It's ideal for development environments where computational resources are limited but reasonable performance is still required.
