GPT2 Chinese Poem Generator
| Property | Value |
|---|---|
| Framework | PyTorch, Transformers |
| Training Data | 800,000 Chinese poems |
| Research Paper | UER Paper |
| Downloads | 5,295 |
What is gpt2-chinese-poem?
gpt2-chinese-poem is a specialized language model designed for generating classical Chinese poetry. Built on the GPT2 architecture and pre-trained using the UER-py framework, this model has been trained on an extensive corpus of 800,000 Chinese ancient poems collected from various sources including the chinese-poetry project.
Implementation Details
The model employs a sophisticated pre-training approach using UER-py and TencentPretrain frameworks. It was trained for 200,000 steps with a sequence length of 128, utilizing an extended vocabulary specifically designed to handle Chinese characters that appear frequently in classical poetry. The training process was conducted on Tencent Cloud infrastructure using 8 GPUs with a learning rate of 5e-4 and batch size of 64.
- Custom vocabulary handling for classical Chinese characters
- Specialized text generation pipeline for poetry
- Support for both masked and unmasked special tokens
Core Capabilities
- Generation of classical Chinese poetry in traditional styles
- Flexible text generation with customizable parameters
- Support for both complete poems and continuation from prompts
- Integration with HuggingFace Transformers pipeline
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines modern GPT2 architecture with classical Chinese poetry generation, featuring a specially curated vocabulary that includes characters occurring ≥100 times in the poetry corpus. Its integration with both UER-py and TencentPretrain frameworks enables efficient training and deployment.
Q: What are the recommended use cases?
The model is ideal for generating classical Chinese poetry, continuing partial poems, and creating poetry in response to prompts. It's particularly suitable for applications in digital humanities, educational tools, and creative writing assistance focused on classical Chinese literature.
