T2I-Adapter-Sketch-SDXL
| Property | Value |
|---|---|
| Parameter Count | 77M |
| Base Model | Stable Diffusion XL 1.0 |
| License | Apache 2.0 |
| Paper | arXiv:2302.08453 |
What is t2i-adapter-sketch-sdxl-1.0?
T2I-Adapter-Sketch-SDXL is a specialized adapter network developed by TencentARC that enhances Stable Diffusion XL's ability to generate images from sketches. It uses PidiNet edge detection to process input sketches and provides additional conditioning to the base SDXL model, enabling precise control over the generation process.
Implementation Details
The model is implemented as a lightweight adapter network with 77M parameters that integrates with the larger 2.6B parameter SDXL base model. It was trained on 3M high-resolution image-text pairs from LAION-Aesthetics V2 for 20,000 steps using a batch size of 256 and a constant learning rate of 1e-5.
- Trained with PidiNet edge detection for sketch processing
- Supports hand-drawn monochrome images with white outlines on black backgrounds
- Implements mixed precision training with fp16
- Uses EulerAncestralDiscreteScheduler for generation
Core Capabilities
- Convert simple sketches into detailed, photorealistic images
- Maintain structural fidelity to input sketches
- Support high-resolution image generation
- Controllable generation through adapter_conditioning_scale parameter
Frequently Asked Questions
Q: What makes this model unique?
This model is specifically designed to work with the SDXL architecture, offering superior sketch-to-image generation capabilities while requiring minimal additional parameters (77M) compared to the base model (2.6B).
Q: What are the recommended use cases?
The model excels at converting rough sketches into detailed images, making it ideal for artists, designers, and creators who want to quickly visualize their concepts. It's particularly useful for concept art, architectural visualization, and rapid prototyping of visual ideas.
