OWL-ViT Large Patch14

What is owlvit-large-patch14?

OWL-ViT (Vision Transformer for Open-World Localization) is an advanced zero-shot text-conditioned object detection model that combines CLIP's multimodal capabilities with object detection functionality. It features a ViT-L/14 architecture and can process multiple text queries to locate objects in images without requiring prior training on specific object classes.

Implementation Details

The model architecture consists of two main components: a CLIP backbone featuring a ViT-L/14 Transformer for image encoding and a masked self-attention Transformer for text encoding. The model removes CLIP's final token pooling layer and adds lightweight classification and box heads to each transformer output token, enabling object detection capabilities.

• CLIP backbone trained from scratch on image-caption data
• Fine-tuned on COCO and OpenImages datasets
• Supports multiple text queries per image
• Uses bipartite matching loss during training

Core Capabilities

• Zero-shot object detection without class-specific training
• Open-vocabulary classification using text embeddings
• Multiple object detection with confidence scores
• Precise bounding box predictions
• Text-conditioned search within images