TIPO-500M-ft
| Property | Value |
|---|---|
| Parameter Count | 500M |
| Architecture | LLaMA |
| Context Length | 1024 tokens |
| Training Data | Danbooru, GBC10M, Coyo11M |
| Training Hardware | 4x RTX 3090 |
| License | Kohaku License 1.0 |
| Paper | arXiv:2411.08127 |
What is TIPO-500M-ft?
TIPO-500M-ft is a specialized language model designed for Text-to-Image Prompt Optimization (TIPO). It's a fine-tuned version of the LLaMA architecture with 500M parameters, trained specifically to enhance the quality of text prompts for image generation systems. The model has been trained on a comprehensive dataset including Danbooru2023 and Coyo-HD-11M, processing approximately 42B tokens during training.
Implementation Details
The model implements the TIPO framework, which uses text presampling within the inference pipeline of text-to-image generative modeling. It's designed to work with various stable diffusion interfaces, including stable-diffusion-webui, stable-diffusion-webui-forge, and ComfyUI through the z-tipo-extension.
- Trained for 290 hours on 4x RTX 3090 GPUs
- Utilizes 1024 token context length
- Implements batch size of 3584
- Incorporates combined training data from multiple high-quality datasets
Core Capabilities
- Enhanced prompt generation for better image outputs
- Superior performance in scenery tag tests compared to alternatives
- Effective handling of both short and truncated long prompts
- Improved aesthetic scores while maintaining low FDD (Fréchet Distance Distribution)
- High AI corruption resistance (0.9195 score)
Frequently Asked Questions
Q: What makes this model unique?
TIPO-500M-ft stands out for its specialized text presampling approach, which enables it to refine and extend user input prompts automatically. It achieves better aesthetic scores and lower FDD compared to other prompt optimization methods while requiring minimal user effort.
Q: What are the recommended use cases?
The model is particularly effective for optimizing prompts in text-to-image generation systems. It excels in handling both simple scenario tags and complex descriptions, making it suitable for both novice users seeking better image generation results and professionals requiring refined prompt engineering.
