InternLM-Chat-20B
| Property | Value |
|---|---|
| Model Size | 20B parameters |
| Training Data | 2.3T Tokens |
| Context Length | 16k tokens |
| License | Apache-2.0 |
| Architecture | 60-layer Transformer |
What is internlm-chat-20b?
InternLM-chat-20b is a state-of-the-art large language model developed through collaboration between Shanghai AI Lab, SenseTime, CUHK, and Fudan University. It represents a significant advancement in LLM technology, combining deep architecture (60 layers) with comprehensive training on high-quality multilingual data including English, Chinese, and code.
Implementation Details
The model utilizes a deeper architecture than conventional 7B and 13B models, with 60 layers providing enhanced capabilities. It's implemented with bfloat16 precision support and includes sophisticated features for both standard and streaming chat interactions.
- Trained on over 2.3T tokens of curated data
- Supports 16k context length through inference extrapolation
- Implements both standard chat and stream chat capabilities
- Optimized for both CPU and GPU deployment
Core Capabilities
- Outstanding performance across language tasks (55% score)
- Superior knowledge comprehension (60.1% score)
- Enhanced reasoning capabilities (54.9% score)
- Strong programming abilities (25.61% on HumanEval)
- Exceptional examination performance (62.5% score)
Frequently Asked Questions
Q: What makes this model unique?
InternLM-20B stands out for its deeper architecture and comprehensive training approach, achieving performance levels that rival or exceed those of larger models like Llama-65B in many tasks. It's particularly notable for its balanced performance across different capability dimensions.
Q: What are the recommended use cases?
The model excels in various applications including academic research, knowledge-intensive tasks, programming assistance, and general language understanding. It's particularly well-suited for applications requiring strong reasoning and comprehensive knowledge processing.
