RedPajama-INCITE-7B-Instruct
| Property | Value |
|---|---|
| Model Size | 6.9B parameters |
| License | Apache 2.0 |
| Language | English |
| Training Hardware | 8 A100 GPUs |
What is RedPajama-INCITE-7B-Instruct?
RedPajama-INCITE-7B-Instruct is an advanced language model developed through collaboration between Together Computer and prominent institutions including ETH DS3Lab, MILA, and Stanford CRFM. This instruction-tuned model is designed for versatile natural language processing tasks, built upon the RedPajama-INCITE-7B-Base architecture.
Implementation Details
The model supports multiple inference options, including GPU (16GB required), GPU with Int8 quantization (12GB required), and CPU deployment. It utilizes the transformers library (version 4.25.1 or higher) and can be fine-tuned with specific parameters like temperature (0.7), top_p (0.7), and top_k (50).
- Trained on 1B tokens using Adam optimizer
- Supports both float16 and bfloat16 precision
- Implements efficient inference strategies for various hardware configurations
Core Capabilities
- Sentiment Analysis and Classification
- Question Answering
- Topic Classification
- Text Paraphrasing
- Text Summarization
- Word Sense Disambiguation
- Natural Language Inference
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its versatile instruction-tuning capabilities and efficient deployment options across different hardware configurations. It's particularly notable for its comprehensive training on the RedPajama dataset and support for multiple NLP tasks.
Q: What are the recommended use cases?
The model is ideal for various natural language processing tasks including text classification, summarization, and question answering. However, it should not be used for safety-critical applications or decisions that significantly impact individuals or society.
