transformer-turkish-summarization
| Property | Value |
|---|---|
| Parameter Count | 125M |
| License | MIT |
| Paper | Mukayese: Turkish NLP Strikes Back |
| ROUGE-1 Score | 43.2049 |
What is transformer-turkish-summarization?
This is a specialized transformer model designed for Turkish text summarization, developed as part of the Mukayese project. The model was trained from scratch without pre-training, specifically on the mlsum/tu dataset. It represents a significant advancement in Turkish language processing, utilizing modern transformer architecture to generate high-quality summaries of Turkish text.
Implementation Details
The model employs a sophisticated training approach with mixed-precision training and distributed computing across 8 GPUs. It was trained using the Adam optimizer with carefully tuned hyperparameters, including a learning rate of 0.0001 and a batch size of 64.
- Training duration: 15 epochs
- Label smoothing factor: 0.1
- Gradient accumulation steps: 2
- Native AMP implementation for mixed-precision training
Core Capabilities
- Achieves 43.2049 ROUGE-1 score on evaluation
- 30.7082 ROUGE-2 score
- 38.1981 ROUGE-L score
- Optimized for Turkish language summarization
- Supports text-to-text generation tasks
Frequently Asked Questions
Q: What makes this model unique?
This model is specifically designed for Turkish language summarization, trained from scratch without relying on pre-training. It achieves competitive ROUGE scores while being optimized for Turkish text processing.
Q: What are the recommended use cases?
The model is ideal for automatic summarization of Turkish news articles, documents, and other long-form content where concise summaries are needed. It's particularly well-suited for media organizations and content platforms dealing with Turkish language content.
