Our catalog of challenges for CV algorithms creates a basis for referencing criticalities by name and allows the calculation of criticality coverage. It is a simple truth from set theory: We cannot calculate or reference individual parts of a whole without defining the full set itself. This catalogue is the first and only approach to define what constitutes a full set of critical situations for computer vision and it opens up a whole new quality for way we evaluate computer vision.

The catalog shall be collaboratively be extended and improved by the community. We will maintain it and moderate its modifications, but we invite all who know improvements to contribute.