Comprehension
Comprehension is whether an AI agent can operate an API from its documentation: find the facts an integration depends on, construct correct requests, and refuse to invent capabilities the API does not support. It is the test an API takes after being found — documentation an agent can locate but cannot work from still loses the integration.
Comprehension failure has a recognizable shape in the wild: a coding agent misreads the auth scheme, guesses at endpoint paths, or confidently scaffolds against capabilities that do not exist — and the resulting support ticket blames the API, not the agent. Every scaffold attempt is an agent reading the docs in anger, and what it can extract from them decides whether the integration works.
Comprehension also varies with the model doing the reading. Documentation that a frontier-tier model can operate from may defeat a smaller one, and since routing routine work to lower-cost models is standard industry practice, how far down the capability range your docs remain operable is a commercial property, captured by the capability floor.
Discry measures comprehension as an experiment rather than a checklist. Models across capability tiers answer quizzes with the fetched docs as their only source, drawn from published task banks: factual questions about auth, errors, pagination, and limits; request-construction tasks; and trap tasks where the right answer is a refusal. Every answer is graded mechanically against ground truth that must cite a specific location in the API's own documentation, and a closed-book pass excludes anything a model already knew.
What does an AI agent make of your API?
Find out in about a minute — no signup.
Discry your API — free