Laserxtral
| Property | Value |
|---|---|
| Parameter Count | 24.2B |
| Model Type | Mixture of Experts (MoE) |
| License | CC-BY-NC-2.0 |
| Tensor Type | BF16 |
What is laserxtral?
Laserxtral is an innovative Mixture of Experts (MoE) model developed by cognitivecomputations that implements a novel 'lasering' technique to denoise and enhance model capabilities. Created by David, Fernando and Eric, and sponsored by VAGO Solutions, this model achieves comparable performance to Mixtral 8x7B Instruct while using only half the parameters.
Implementation Details
The model is built using mergekit and combines five carefully selected base models, including Dolphin-2.6-mistral, Marcoro14-7B-slerp, CodeNinja, MetaMath-Cybertron-Starling, and WizardMath. The innovative laserRMT technique is applied to control layers by identifying and treating those with lower signal-to-noise ratios.
- Utilizes advanced laserRMT implementation for noise reduction
- Combines expertise from multiple specialized models
- Implements Machenko Pastur calculations for ratio determination
- Achieves efficient parameter usage through expert mixture
Core Capabilities
- High truthfulness in responses
- Enhanced reasoning capabilities
- Efficient parameter utilization
- Comparable performance to larger models
Frequently Asked Questions
Q: What makes this model unique?
The model's uniqueness lies in its innovative use of laserRMT technology to denoise specific layers, allowing it to achieve performance comparable to larger models while using fewer parameters.
Q: What are the recommended use cases?
With its high truthfulness and reasoning capabilities, the model is well-suited for text generation tasks requiring accurate information processing and complex reasoning.
