ko-sroberta-nli
| Property | Value |
|---|---|
| Author | jhgan |
| Framework | PyTorch, Sentence-Transformers |
| Output Dimension | 768 |
| Citations | Ham et al. (2020), Reimers & Gurevych (2019, 2020) |
What is ko-sroberta-nli?
ko-sroberta-nli is a specialized Korean language model based on RoBERTa architecture, fine-tuned on Korean Natural Language Inference (KorNLI) dataset. It's designed to generate high-quality sentence embeddings, mapping Korean text to 768-dimensional dense vector spaces for advanced semantic analysis tasks.
Implementation Details
The model utilizes a RoBERTa backbone with mean pooling strategy and was trained using MultipleNegativesRankingLoss with a scale factor of 20.0. It employs the AdamW optimizer with a learning rate of 2e-05 and implements a WarmupLinear scheduler with 889 warmup steps.
- Maximum sequence length: 128 tokens
- Pooling strategy: Mean tokens pooling
- Batch size: 64
- Training epochs: 1
Core Capabilities
- Sentence similarity computation with strong performance (82.83% Cosine Pearson on KorSTS)
- Dense vector generation for Korean text
- Semantic search and clustering applications
- Cross-sentence semantic understanding
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its specialized optimization for Korean language understanding, demonstrated by its strong performance on the KorSTS benchmark and its efficient sentence embedding generation capabilities.
Q: What are the recommended use cases?
The model excels in semantic similarity tasks, document clustering, semantic search applications, and any NLP task requiring high-quality Korean sentence embeddings.
