LaMini-T5-738M
| Property | Value |
|---|---|
| Parameter Count | 738M |
| Base Model | T5-large |
| License | CC BY NC 4.0 |
| Paper | arXiv:2304.14402 |
| Training Data | 2.58M instruction samples |
What is LaMini-T5-738M?
LaMini-T5-738M is part of the LaMini-LM model series, developed by MBZUAI. It's a fine-tuned version of T5-large, specifically optimized for instruction-following tasks through training on a diverse dataset of 2.58M instruction samples. The model represents a balanced compromise between computational efficiency and performance capability.
Implementation Details
The model was trained using a carefully curated set of hyperparameters, including a learning rate of 0.0005, batch size of 512, and linear learning rate scheduling over 5 epochs. It uses the Adam optimizer with betas=(0.9,0.999) and epsilon=1e-08.
- Built on T5 architecture with 738M parameters
- Trained using gradient accumulation steps of 4
- Optimized for text-to-text generation tasks
- Implements instruction fine-tuning methodology
Core Capabilities
- Natural language instruction following
- Text generation and completion
- Question answering
- Task-specific text transformation
Frequently Asked Questions
Q: What makes this model unique?
LaMini-T5-738M stands out for its efficient balance between model size and performance, being part of a comprehensive series of models designed for different scaling needs. It's specifically optimized for instruction-following tasks through extensive fine-tuning on a diverse instruction dataset.
Q: What are the recommended use cases?
The model is best suited for text generation tasks that require following natural language instructions. It can be effectively used for content generation, question answering, and text transformation tasks where a medium-sized model with good performance is needed.
