Llama-3-Karamaru-v1
| Property | Value |
|---|---|
| Developer | Sakana AI |
| Base Model | Llama-3-ELYZA-JP-8B |
| License | Llama3 Community License |
| Training Data | 25M characters of Edo-period text |
What is Llama-3-Karamaru-v1?
Llama-3-Karamaru-v1 is an innovative language model that bridges historical and modern Japanese language. Developed by Sakana AI, it specializes in converting modern Japanese queries into responses styled in classical Edo-period Japanese. The model was trained on an extensive dataset of 25 million characters, combining human-transcribed text and AI-processed historical documents.
Implementation Details
The model utilizes a sophisticated architecture based on Llama-3-ELYZA-JP-8B, enhanced through continual pretraining on historical Japanese texts. The training data comprises 13 million characters of human-transcribed text and 12 million characters processed using AI-based kuzushiji OCR technology.
- Custom Edo-period dataset integration
- Advanced kuzushiji OCR processing using RURI model
- Specialized text refinement using Sakana AI's LLM-based classical Japanese OCR Refiner
- Pytorch implementation with bfloat16 precision support
Core Capabilities
- Modern to Edo-period Japanese language conversion
- Historical context-aware responses
- Support for research and educational applications
- Cultural and linguistic preservation
Frequently Asked Questions
Q: What makes this model unique?
The model's ability to process modern Japanese queries and respond in authentic Edo-period style, leveraging a massive historical text dataset and specialized OCR technology, makes it unique in the field of historical language processing.
Q: What are the recommended use cases?
The model is ideal for research, education, and cultural exploration, particularly in studying historical Japanese language and thought. It can be used in academic settings, cultural preservation projects, and educational programs focused on Japanese history and linguistics.
