DistilBERT Base Turkish Cased
| Property | Value |
|---|---|
| Parameter Count | 68.1M |
| License | MIT |
| Paper | View Paper |
| Author | dbmdz |
| Framework Support | PyTorch, TensorFlow |
What is distilbert-base-turkish-cased?
DistilBERT Base Turkish Cased is a lightweight, distilled version of the original BERTurk model, specifically designed for Turkish language processing. Developed by the MDZ Digital Library team at the Bavarian State Library, this model maintains case sensitivity while reducing the computational footprint of the original BERT architecture.
Implementation Details
The model was trained on 7GB of Turkish text data using knowledge distillation techniques, with the cased version of BERTurk serving as the teacher model. The training process took 5 days using 4 RTX 2080 TI GPUs, implementing the official Hugging Face distillation approach.
- Maintains case sensitivity for better handling of Turkish text
- Compatible with PyTorch-Transformers framework
- Achieves performance within 1.18% of the original BERTurk model
Core Capabilities
- Part-of-Speech (PoS) tagging with performance exceeding 24-layer XLM-RoBERTa
- Named Entity Recognition (NER)
- General Turkish language understanding tasks
- Efficient inference with reduced model size
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines the efficiency of distillation techniques with specific optimization for Turkish language processing, achieving near-original performance while being significantly smaller and faster.
Q: What are the recommended use cases?
The model is particularly well-suited for Turkish language processing tasks including PoS tagging and NER, especially in environments where computational resources are limited but high performance is required.
