BERT Base German Cased
| Property | Value |
|---|---|
| Parameter Count | 110M |
| License | MIT |
| Training Data | 12GB (Wiki, Legal, News) |
| Architecture | BERT Base Cased |
What is bert-base-german-cased?
bert-base-german-cased is a German language model based on the BERT architecture, specifically trained on a diverse corpus of German text. Developed by deepset.ai, this model has been trained on approximately 12GB of data including German Wikipedia dumps, OpenLegalData, and news articles. The model maintains case sensitivity, making it particularly useful for tasks where capitalization matters.
Implementation Details
The model was trained using Google's TensorFlow framework on a TPU v2 for approximately 9 days. The training process involved 810,000 steps with a batch size of 1024 for sequence length 128, followed by 30,000 steps with sequence length 512. The training utilized a learning rate of 1e-4 with linear warmup and 10,000 warmup steps.
- Training Infrastructure: Single Cloud TPU v2
- Vocabulary: Custom word piece vocabulary using sentencepiece
- Data Processing: Cleaned with tailored scripts and sentence segmentation using spacy v2.1
Core Capabilities
- Named Entity Recognition (NER) with strong performance on GermEval14 and CONLL03
- Sentiment Classification (GermEval18)
- Document Classification (GNAD)
- Fill-Mask Task Support
- Compatible with PyTorch, TensorFlow, and JAX
Frequently Asked Questions
Q: What makes this model unique?
This model is specifically optimized for German language tasks and achieves strong performance across various downstream tasks with minimal fine-tuning. It's particularly notable for its rapid convergence to optimal performance during training.
Q: What are the recommended use cases?
The model excels in German NLP tasks including named entity recognition, sentiment analysis, and document classification. It's particularly suitable for applications requiring understanding of German text with preserved case sensitivity.
