evo-1-131k-base
| Property | Value |
|---|---|
| Parameter Count | 6.45B |
| License | Apache 2.0 |
| Context Length | 131,072 tokens |
| Paper | Science Journal Publication |
What is evo-1-131k-base?
evo-1-131k-base is a cutting-edge biological foundation model designed for long-context genomic sequence modeling and design. Built on the innovative StripedHyena architecture, it represents a significant advancement in processing biological sequences at single-nucleotide resolution. Trained on the OpenGenome dataset containing approximately 300 billion tokens, this model demonstrates exceptional capability in handling sequences up to 650,000 tokens in length.
Implementation Details
The model utilizes a hybrid architecture combining multi-head attention and gated convolutions arranged in Hyena blocks. This design enables near-linear scaling of compute and memory relative to context length, making it particularly efficient for long sequence processing.
- Deep signal processing architecture with specialized layer classes
- Supports multiple parametrization options for different workloads
- Mixed precision implementation with float32 precision for poles and residues
- Efficient autoregressive generation capability
Core Capabilities
- Processing sequences at single-nucleotide, byte-level resolution
- Generation of sequences exceeding 650k length on a single 80GB GPU
- 3x faster training and finetuning at 131k context length
- Robust performance beyond traditional compute-optimal frontiers
- Flexible parametrization for various inference and finetuning scenarios
Frequently Asked Questions
Q: What makes this model unique?
The model's StripedHyena architecture sets it apart by enabling efficient processing of extremely long biological sequences while maintaining linear scaling characteristics. Its hybrid approach combining attention and convolution mechanisms makes it particularly suited for genomic-scale applications.
Q: What are the recommended use cases?
The model is specifically designed for biological sequence modeling at the genome scale, making it ideal for tasks involving whole-genome analysis, sequence design, and other genomics applications requiring long-context understanding.
