Bielik-7B-Instruct-v0.1
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Model Type | Causal decoder-only |
| License | CC BY NC 4.0 |
| Language | Polish |
| Base Model | Bielik-7B-v0.1 |
What is Bielik-7B-Instruct-v0.1?
Bielik-7B-Instruct-v0.1 is a state-of-the-art Polish language model developed through collaboration between SpeakLeash and ACK Cyfronet AGH. This instruction-tuned model represents a significant advancement in Polish natural language processing, featuring exceptional performance in RAG tasks and demonstrating strong capabilities in understanding and generating Polish text.
Implementation Details
The model was trained using the ALLaMo framework with sophisticated techniques including weighted tokens level loss, adaptive learning rates, and masked user instructions. Training utilized a context length of 4096 tokens and employed mixed precision BFloat16 training.
- Training utilized both Polish and English instruction datasets
- Implements cosine learning rate scheduling with adaptive adjustments
- Supports both chat template and manual prompt formatting
- Available in multiple quantized versions for different hardware configurations
Core Capabilities
- Achieves 39.28 average score on Polish LLM benchmarks
- Excels in RAG Reader tasks with 86.00% accuracy
- Supports extensive context window of 4096 tokens
- Optimized for instruction-following in Polish language
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its specialized Polish language capabilities and strong performance in RAG tasks, achieved through careful instruction tuning and sophisticated training techniques. It's particularly notable for achieving state-of-the-art results in Polish language understanding while maintaining efficient resource usage.
Q: What are the recommended use cases?
Bielik-7B-Instruct-v0.1 is ideal for Polish language tasks including question-answering, text generation, and RAG applications. It's particularly well-suited for academic and research purposes, though commercial use requires specific licensing.
