X-CLIP Base Patch16
| Property | Value |
|---|---|
| Parameter Count | 195M |
| License | MIT |
| Paper | View Paper |
| Top-1 Accuracy | 83.8% |
| Training Dataset | Kinetics-400 |
What is xclip-base-patch16?
X-CLIP base-patch16 is a powerful video-language understanding model that extends CLIP's capabilities to video processing. Developed by Microsoft, it processes videos using 16x16 pixel patches at 224x224 resolution, analyzing 8 frames per video for optimal performance. The model employs a contrastive learning approach to understand relationships between video content and textual descriptions.
Implementation Details
The model architecture builds upon CLIP's foundation, specifically designed for video-language tasks. It processes video inputs through a specialized pipeline that includes frame normalization using ImageNet statistics and sophisticated temporal analysis.
- Uses 16x16 patch resolution for video frame processing
- Processes 8 frames per video sequence
- Operates at 224x224 pixel resolution
- Implements contrastive learning between video and text pairs
Core Capabilities
- Zero-shot video classification
- Few-shot learning for video tasks
- Video-text retrieval
- Fully supervised video classification
- Achieves 95.7% top-5 accuracy on Kinetics-400
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely extends CLIP's architecture to handle video content, enabling sophisticated video-language understanding while maintaining relatively modest computational requirements at 195M parameters.
Q: What are the recommended use cases?
The model excels in video classification tasks, video-text matching, and retrieval applications. It's particularly effective for applications requiring understanding of temporal relationships in video content and their correlation with textual descriptions.
