ESG-BERT
| Property | Value |
|---|---|
| Parameter Count | 110M |
| Model Type | Language Model (BERT-based) |
| Primary Use | ESG Text Classification |
| Performance | 0.90 F1-Score |
| Authors | Mukut Mukherjee, Charan Pothireddi, Parabole.ai |
What is ESG-BERT?
ESG-BERT is a specialized language model designed for text mining in sustainable investing. Built on the BERT architecture, this model has been specifically trained to understand and classify Environmental, Social, and Governance (ESG) related content. The model demonstrates superior performance compared to general BERT models, achieving a 90% F1-score in classification tasks versus 79% for BERT-base.
Implementation Details
The model consists of 110M parameters and has undergone extensive training with domain-specific data. It achieved 100% accuracy in Next Sentence Prediction and 98% in Masked Language Modeling during training. The model supports 26 different ESG-related classification categories, ranging from Business Ethics to GHG Emissions.
- Advanced text classification capabilities for ESG content
- Supports deployment via TorchServe
- Includes comprehensive handler scripts for inference
- Compatible with PyTorch and Transformers library
Core Capabilities
- ESG-specific text classification
- Fine-tuning potential for downstream NLP tasks
- Sustainable investing analysis
- Environmental impact assessment
- Corporate responsibility evaluation
Frequently Asked Questions
Q: What makes this model unique?
ESG-BERT's specialization in sustainable investing sets it apart, with significantly better performance (90% F1-score) compared to general-purpose BERT models in ESG-related tasks.
Q: What are the recommended use cases?
The model is ideal for analyzing sustainability reports, corporate documents, and ESG-related content. It can classify text into 26 different ESG categories and can be fine-tuned for specific sustainable investing applications.
