GIST-all-MiniLM-L6-v2
| Property | Value |
|---|---|
| Parameters | 22.7M |
| License | MIT |
| Paper | GISTEmbed Paper |
| Framework | PyTorch, Sentence-Transformers |
What is GIST-all-MiniLM-L6-v2?
GIST-all-MiniLM-L6-v2 is a specialized text embedding model that implements the Guided In-sample Selection of Training Negatives (GIST) technique. Built upon the all-MiniLM-L6-v2 architecture, this model has been fine-tuned using the MEDI dataset and additional triplets mined from MTEB Classification training data. The model excels at generating semantic embeddings without requiring specific instructions, making it particularly efficient for retrieval and similarity tasks.
Implementation Details
The model was trained with specific parameters including 40 epochs, a warmup ratio of 0.1, learning rate of 5e-6, and batch size of 16. It employs a contrastive loss temperature of 0.01 and implements checkpoint steps at 102,000 iterations. The architecture maintains the efficient design of MiniLM while incorporating GIST optimization techniques.
- No instruction required for embedding generation
- Optimized for both similarity and retrieval tasks
- Built on proven MiniLM architecture
- Fine-tuned with carefully curated datasets
Core Capabilities
- Semantic text similarity assessment
- Document retrieval and ranking
- Classification tasks
- Clustering applications
- Pair-wise text comparison
Frequently Asked Questions
Q: What makes this model unique?
The model's uniqueness lies in its GIST fine-tuning approach and ability to generate high-quality embeddings without requiring task-specific instructions, while maintaining competitive performance across various benchmarks.
Q: What are the recommended use cases?
The model is particularly well-suited for semantic search, document similarity matching, text classification, and information retrieval tasks. It performs exceptionally well in scenarios requiring semantic understanding without complex prompt engineering.
