EVA-Qwen2.5-32B-v0.2
| Property | Value |
|---|---|
| Parameter Count | 32.8B |
| Model Type | Text Generation/Roleplay |
| Base Model | Qwen2.5-32B |
| License | Apache 2.0 |
| Training Format | ChatML |
What is EVA-Qwen2.5-32B-v0.2?
EVA-Qwen2.5-32B-v0.2 is a specialized language model fine-tuned for roleplay and creative writing applications. It represents a significant improvement over its previous version, featuring cleaned and reprocessed datasets to eliminate non-unicode character issues and improve generation stability.
Implementation Details
The model was trained for 7 hours on 8xH100 SXM hardware, implementing full-parameter fine-tuning on the Qwen2.5-32B base model. It uses advanced training techniques including sample packing and a sequence length of 10240 tokens.
- Trained on 10 carefully curated datasets including Celeste 70B mixture and specialized roleplay content
- Implements ChatML format for consistent interaction
- Uses BF16 precision for optimal performance
Core Capabilities
- Creative writing and storytelling
- Character-based roleplay interactions
- Extended context handling up to 10K tokens
- Improved stability and coherence in outputs
Frequently Asked Questions
Q: What makes this model unique?
The model combines extensive parameter fine-tuning with a diverse dataset mixture, specifically optimized for creative and roleplay applications. Version 0.2 introduces significant improvements in data cleaning and processing, eliminating previous artifacts and ensuring more stable output.
Q: What are the recommended use cases?
The model excels at creative writing, character roleplay, and story generation. It's optimized with specific sampling parameters (Temperature: 1, Min-P: 0.05, Top-A: 0.2, Repetition Penalty: 1.03) for optimal creative output.
