stsb-bert-tiny-onnx
| Property | Value |
|---|---|
| Parameter Count | 4.39M |
| Tensor Type | F32 |
| Embedding Dimension | 128 |
| Framework | ONNX |
What is stsb-bert-tiny-onnx?
stsb-bert-tiny-onnx is a lightweight ONNX-optimized BERT model designed for sentence similarity tasks. It's capable of mapping sentences and paragraphs to 128-dimensional dense vector spaces, making it particularly efficient for applications like clustering and semantic search. The model has been optimized using the ONNX framework for improved inference performance while maintaining accuracy.
Implementation Details
The model utilizes a two-component architecture consisting of a Transformer layer followed by a Pooling layer. It was trained using CosineSimilarityLoss with AdamW optimizer and implements a WarmupLinear scheduler. The training process involved 10 epochs with a batch size of 16 and a learning rate of 8e-05.
- Maximum sequence length: 512 tokens
- Pooling configuration: Mean pooling enabled, other pooling modes disabled
- Warmup steps: 36
- Weight decay: 0.01
Core Capabilities
- Sentence and paragraph embedding generation
- Semantic similarity computation
- Efficient clustering operations
- Cross-lingual text comparison
- Text feature extraction
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient architecture, combining a tiny BERT implementation with ONNX optimization for faster inference. Its 4.39M parameter count makes it significantly lighter than standard BERT models while maintaining practical utility for sentence similarity tasks.
Q: What are the recommended use cases?
The model is ideal for applications requiring lightweight sentence embedding generation, such as semantic search systems, document clustering, and text similarity analysis. It's particularly suitable for resource-constrained environments where computational efficiency is crucial.
