una-xaberius-34b-v1beta
| Property | Value |
|---|---|
| Parameter Count | 34.4B |
| Base Architecture | LLaMa Yi-34B |
| License | CC-BY-NC-ND-4.0 |
| Framework | Transformers 4.35.2-UNA |
What is una-xaberius-34b-v1beta?
una-xaberius-34b-v1beta is an advanced language model developed by juanako.ai that implements a novel Uniform Neural Alignment (UNA) technique. The model achieved remarkable performance, scoring 74.18 on average across major benchmarks and setting new records for open-source models in specific tasks like MMLU with a score of 78.15.
Implementation Details
Built on the LLaMa Yi-34B architecture, this model incorporates specialized training using SFT, DPO, and the proprietary UNA technique. It operates in FP16 precision and was trained on multiple high-quality datasets including Tree of Knowledge, Open-Platypus, UltraFeedback, and OpenOrca.
- Implements novel UNA technique (distinct from layer merging or SLERP)
- Optimized for both ChatML and Alpaca system prompts
- Trained on Cybertron's H100 infrastructure
- Compatible with text-generation-inference systems
Core Capabilities
- Exceptional performance on MMLU (78.15)
- Strong reasoning capabilities (ARC Challenge: 70.39)
- Advanced common sense understanding (HellaSwag: 86.77)
- Robust mathematical reasoning (GSM8K: 63.38)
- High truthfulness score (TruthfulQA: 61.45)
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its UNA (Uniform Neural Alignment) implementation, which sets it apart from traditional merging techniques while achieving state-of-the-art performance for its parameter size.
Q: What are the recommended use cases?
The model excels in various tasks including reasoning, mathematics, and general knowledge applications. It's particularly well-suited for applications requiring high accuracy in multiple-choice scenarios and complex reasoning tasks.
