PhoGPT-4B-Chat
| Property | Value |
|---|---|
| Parameter Count | 3.7B |
| License | BSD-3-Clause |
| Language | Vietnamese |
| Research Paper | arXiv:2311.02945 |
What is PhoGPT-4B-Chat?
PhoGPT-4B-Chat is a state-of-the-art Vietnamese language model developed by VinAI Research. It's a fine-tuned variant of the base PhoGPT-4B model, specifically optimized for conversational applications. The model represents a significant advancement in Vietnamese natural language processing, with its architecture based on a powerful 3.7B parameter foundation.
Implementation Details
The model is built upon a comprehensive training approach, beginning with pre-training on 102B tokens of Vietnamese text. The architecture supports an impressive 8192-token context length and employs a carefully curated vocabulary of 20,480 token types. The chat variant underwent fine-tuning on 70K instructional prompts and responses, supplemented by 290K conversations.
- Pre-trained on 102B Vietnamese tokens
- 8192 context length capability
- 20,480 token vocabulary size
- Fine-tuned on 360K combined conversational samples
Core Capabilities
- Advanced Vietnamese language understanding and generation
- Optimized for conversational AI applications
- Long context handling with 8192 token support
- State-of-the-art performance in Vietnamese language tasks
Frequently Asked Questions
Q: What makes this model unique?
PhoGPT-4B-Chat stands out as one of the largest and most capable Vietnamese language models available, specifically designed and optimized for the Vietnamese language rather than being a multilingual adaptation. Its extensive pre-training on Vietnamese text and specialized fine-tuning for conversational tasks make it particularly effective for Vietnamese language applications.
Q: What are the recommended use cases?
The model is ideally suited for Vietnamese language applications requiring conversational AI capabilities, including chatbots, virtual assistants, and interactive language systems. It's particularly effective for tasks requiring deep understanding and generation of Vietnamese text within a conversational context.
