luke-japanese-base-finetuned-ner
| Property | Value |
|---|---|
| Parameter Count | 279M |
| License | MIT |
| Framework | PyTorch |
| Language | Japanese |
| Task | Named Entity Recognition |
What is luke-japanese-base-finetuned-ner?
This is a specialized Japanese Named Entity Recognition (NER) model based on the LUKE (Language Understanding with Knowledge-based Embeddings) architecture. Fine-tuned on Wikipedia data, it's designed to identify and classify named entities in Japanese text with high accuracy, achieving an impressive micro-average F1 score of 0.84.
Implementation Details
The model is built upon luke-japanese-base and fine-tuned using a comprehensive Wikipedia-based Japanese NER dataset. It employs an entity-aware self-attention mechanism, extending the traditional transformer architecture to handle both words and entities as independent tokens.
- Uses SentencePiece tokenization
- Implements entity-aware self-attention
- Supports 8 entity categories including person names, locations, and organizations
- 279M parameters with F32 tensor type
Core Capabilities
- Person Name Recognition (F1: 0.90)
- Corporate Entity Detection (F1: 0.89)
- Event Name Recognition (F1: 0.87)
- Location Name Detection (F1: 0.83)
- Political Organization Recognition (F1: 0.82)
- Facility Name Detection (F1: 0.80)
Frequently Asked Questions
Q: What makes this model unique?
The model's unique strength lies in its entity-aware self-attention mechanism and its comprehensive training on Japanese Wikipedia data, making it particularly effective for Japanese NER tasks with state-of-the-art performance across multiple entity categories.
Q: What are the recommended use cases?
This model is ideal for Japanese text analysis tasks requiring entity extraction, including business document processing, content analysis, and automated information extraction from Japanese texts. It's particularly effective for applications requiring identification of organizations, people, locations, and events.
