UNA-Cybertron-7B-v2-BF16
| Property | Value |
|---|---|
| Parameter Count | 7.24B |
| Base Architecture | Mistral-7B |
| License | Apache-2.0 |
| Tensor Type | BF16 |
| HF Leaderboard Score | 69.67 |
What is una-cybertron-7b-v2-bf16?
UNA-Cybertron-7B-v2 is a state-of-the-art language model based on the MistralAI architecture, trained using the innovative UNA (Uniform Neural Alignment) technique. It achieves remarkable performance across various benchmarks, ranking #1 in its category with a 69.67+ score on the Hugging Face Leaderboard.
Implementation Details
The model utilizes a specialized training approach combining SFT, DPO, and UNA methodologies. It's optimized for 16K context length and performs exceptionally well with the Exllamav2_HF loader using specific configurations.
- Developed by juanako.ai using H100 GPUs
- Implements ChatML format and Alpaca System prompting
- Supports various framework versions including Transformers 4.35.0-UNA and PyTorch 2.1.0
Core Capabilities
- Exceptional performance in mathematics and logical reasoning (55.04% on GSM8K)
- Strong truthfulness metrics (64.63% on TruthfulQA)
- High accuracy in common sense reasoning (85.85% on HellaSwag)
- Robust performance on complex reasoning tasks (68.26% on ARC Challenge)
Frequently Asked Questions
Q: What makes this model unique?
The model's UNA technique sets it apart, offering superior performance without using traditional layer merging or SLERP/SLURP methods. It achieves top-tier performance across multiple benchmarks while maintaining efficiency.
Q: What are the recommended use cases?
The model excels in applications requiring strong mathematical reasoning, logical analysis, and complex problem-solving. It's particularly suitable for tasks requiring detailed comprehension and accurate response generation.
