Vision Testing for Robustness

What makes our vison systems fail?

Computer vision algorithms performance is affected by many aspects: e.g. shadows, reflections, low contrasts, or occlusion.
Let's deal with them!

Participate

Particles
Unexpected Objects
Shadows and Complexity
Transparent Objects
Compound Repetitive Objects
Noise

The Vision

In particular for safety-related applications such as robotics or medical imaging, it is crucial to assure robustness – the insensibility against visual challenges. Testing with recorded images is a common approach, but has its limits.
We want to assure robustness of computer vision algorithms and will

cover all typical and challenging situations
avoid wasting time and money with redundant testing
allow testing of dangerous situations without risks

See for Yourself

See how VITRO works in detail
Download free test data sets
See our reference projects

Model-Based Catalogue of Challenges

We developed an open, widely accepted catalog of visual challenges and potential hazards (e.g. misdetection of obstacles). It will be collaboratively extended and improved by the community. We will maintain and moderate it, and we invite all who know improvements to contribute.

Recent Publications

Analyzing Computer Vision Data – The Good, the Bad and the Ugly. CVPR 2017

Ground Truth Accuracy and Performance of the Matching Pipeline CVPR WS 2017

How Good Is My Test Data? Introducing Safety Analysis for Computer Vision. IJCV 2017

Our Solution

VITRO provides synthetic test data – both stimuli and expected results – that allow for systematic assessment of application-specific robustness concerning the relevant challenges. Generated test data minimize test overhead and risk of missing important constellations.

Stop Wasting Time

… with redundant test data, but make every test run count! Start using VITRO already during development to facilitate all the benefits of test-driven development. Test data sets provided by VITRO are perfectly tailored to your application with maximum expressiveness regarding its robustness. Generated from models, the data is intrinsically consistent and precisely evaluable.