GALACTICA 6.7B
| Property | Value |
|---|---|
| Parameter Count | 6.7 Billion |
| Model Type | Transformer (Decoder-only) |
| License | CC BY-NC 4.0 |
| Release Date | November 2022 |
| Author | Meta AI |
What is galactica-6.7b?
GALACTICA 6.7B is a specialized language model developed by Meta AI, designed specifically for scientific tasks. It represents the 'standard' version in the GALACTICA model family, trained on an extensive corpus of 106 billion tokens of scientific text, including papers, textbooks, and scientific websites. This model bridges the gap between general language models and scientific applications, offering specialized capabilities for academic and research tasks.
Implementation Details
The model implements a decoder-only transformer architecture with specific modifications for scientific text processing. It supports multiple modalities and can be deployed using PyTorch, with options for different precision levels (FP16, INT8) to optimize performance and memory usage.
- Supports both CPU and GPU deployment
- Compatible with Hugging Face's transformers library
- Offers flexible tokenization for different scientific formats
- Multiple precision options for resource optimization
Core Capabilities
- Citation prediction and reference handling
- Scientific question answering
- Mathematical reasoning
- Document summarization and generation
- Molecular property prediction
- Entity extraction from scientific text
Frequently Asked Questions
Q: What makes this model unique?
GALACTICA 6.7B stands out for its specialized training on scientific content, making it particularly effective for academic and research tasks. It demonstrates superior performance on knowledge-intensive scientific tasks compared to general-purpose language models while maintaining lower toxicity rates.
Q: What are the recommended use cases?
The model is primarily intended for researchers studying language models in scientific domains and developers building scientific tools. However, it's important to note that production use should include safeguards due to potential hallucination risks. Common applications include literature discovery, mathematical problem-solving, and scientific document generation.
