HallOumi-8B-classifier
| Property | Value |
|---|---|
| Parameter Count | 8 Billion |
| Model Type | Small Language Model |
| License | CC-BY-NC-4.0 |
| Base Model | Llama-3.1-8B-Instruct |
| Language | English |
What is HallOumi-8B-classifier?
HallOumi-8B-classifier is a groundbreaking hallucination detection model that achieves state-of-the-art performance with just 8 billion parameters. Developed by Oumi AI, it outperforms much larger models including DeepSeek R1 (671B), achieving an impressive 77.2% macro F1 score in hallucination detection tasks. This model represents a significant advancement in AI content verification, particularly crucial in an era where AI hallucinations can lead to serious consequences in legal, business, and personal contexts.
Implementation Details
The model is built on top of Llama-3.1-8B-Instruct and has been specifically optimized for binary classification of hallucinated content. It operates on a per-claim basis, requiring separate evaluation for each statement being verified.
- Training utilized multiple synthetic and curated datasets including oumi-synthetic-document-claims and oumi-anli-subset
- Environmental impact was minimized, using only 1.5 hours of A100-80GB GPU time
- Achieves superior performance while being truly open source
Core Capabilities
- Fast and accurate hallucination probability assessment
- Per-claim support for granular content verification
- Compatible with both AI and human-generated content
- Significantly smaller model size while maintaining SOTA performance
Frequently Asked Questions
Q: What makes this model unique?
HallOumi-8B-classifier stands out for achieving superior performance (77.2% F1 score) with just 8B parameters, surpassing much larger closed-source models while remaining truly open source. This efficiency makes it both accessible and practical for real-world applications.
Q: What are the recommended use cases?
The model is specifically designed for verifying claims and detecting hallucinations in scenarios where a known source of truth is available. It's particularly valuable for content verification in professional contexts where accuracy is crucial, such as legal documents, business communications, and educational materials.
