North Rhine-Westphalia AI, Machine Learning and Computer Vision Meetup

Deploying computer vision models in production comes with unique challenges—tight deadlines, noisy data, and the trade-off between robustness and performance. This talk explores key differences between academic research and real-world applications, highlighting practical strategies for overcoming constraints and ensuring reliable, scalable CV systems.

We present a novel approach that integrates the broad image description capabilities of Vision Language Models (VLMs) with the pixel-level feature extraction of open-vocabulary detection (OVD), instance segmentation, and tracking. We rely entirely on the use of existing foundation models and focus on extracting rich and task-relevant object attributes in a machine-readable format. This results in a vision system that is well suited for robotics use cases in dynamic unknown environments with non-standard objects, as well as for automatic annotation of datasets and benchmarking of VLMs.