◎ Discry Score
buildkite.com
devtools · API
B
0 / 100
DISCOVERY0
COMPREHENSION0
Category leader: 96 (A)
Discry your API →
DEVTOOLS · RANK #18 OF 39

Buildkite is Good to agents.

Discry independently scored how well an AI agent can discover and understand the Buildkite API from what’s public — not whether it’s usable. Below: every signal we checked, what’s costing the score, and what to change.

Discry your API — freeView the docs ↗

SCORED UNDER RUBRIC 1.2 · A full re-launch under Discry Score 2.5 — a new behavioral instrument, not comparable to these scores — is in progress.

Discovery

45% of score · 57/100
OpenAPI specA machine-readable OpenAPI/Swagger spec agents can parse.Fail
llms.txtAn llms.txt index that points agents to the docs that matter.Pass
llms.txt qualityThe llms.txt is focused, current, and well under the size budget.Partial
llms-full.txtA full-text bundle agents can load in one request.Fail
AGENTS.mdAn AGENTS.md that tells coding agents how to build on the API.Pass
.well-known/mcp.jsonA discoverable MCP manifest at a well-known path.Fail
MCP registryThe API is listed in a public MCP registry.Pass
robots.txt AI directivesrobots.txt allows (or explicitly guides) AI crawlers.Pass
SitemapA sitemap so agents can enumerate the docs surface.Partial

Comprehension

55% of score · 93/100
Task-oriented descriptionsEndpoints described by what they accomplish, not just their shape.Pass
Realistic examplesRunnable, real-world request/response examples.Partial
Multi-step workflowsDocs that chain calls into complete jobs an agent can follow.Pass
Error-recovery guidanceDocumented failure modes and how to recover from them.Pass
Answer-first formatThe answer leads; preamble does not bury it.Pass
Capability boundariesClear limits — what the API can and cannot do.Pass
Naming consistencyConsistent, predictable naming across endpoints.Pass
Heading hierarchyClean heading structure agents can navigate.Pass
Markdown docsDocs available as clean markdown, not JS-rendered HTML only.Pass
Token efficiencyDocs are concise enough to fit an agent context window.Pass

What we found

  • An agent can reach Buildkite the way it's designed to be reached: Buildkite ships an official MCP server (managed remote + local, on the Docker MCP catalog, Smithery and Glama), a dedicated coding-agents skills repo, and a 'Getting started with coding agents' guide — very few APIs document agent onboarding this deliberately.
  • Documentation is near-perfect for comprehension: every doc page is available as clean Markdown (append .md), a rich 107KB llms.txt indexes the whole corpus with per-section descriptions, headings nest cleanly, and endpoint pages lead with method/URL/purpose — an agent can parse it without loss.
  • The single biggest gap is discovery-side: there is no official machine-readable OpenAPI/Swagger spec for the REST API, so an agent that wants a typed contract must scrape the HTML/Markdown reference or rely on unofficial community specs.
  • Error recovery is genuinely agent-usable: the listing endpoint returns a 400 that literally tells you how to fix it ('narrow your query using filters such as branch or created_from'), 422s return a structured errors array, and the rate-limit docs specify exact backoff behavior.
  • robots.txt places no restrictions on AI crawlers (GPTBot, ClaudeBot, CCBot, etc. all pass), and the API surface is discoverable — but llms-full.txt returns 403 and there is no .well-known/mcp.json, leaving two low-weight discovery signals unmet.

What to change

Prioritized by impact on discoverability. You (or your docs platform) deploy these — Discry never touches your API.

  1. 01Publish an official, versioned OpenAPI 3.x spec for the REST API at a stable URL (e.g. buildkite.com/docs/apis/rest-api/openapi.json) and link it from the API overview. This is the highest-impact discovery fix — it's the single largest rubric weight and currently scores zero.
  2. 02Serve a working llms-full.txt (or fix the 403 on the existing path). Buildkite already generates per-section llms.txt files, so a consolidated agent-optimized dump is a small step that closes a discovery gap.
  3. 03Trim or split the root llms.txt: at 107KB it exceeds the ~50KB/12K-token comfort zone for a single agent read. Keep the high-value API-focused index and rely on the per-section files for depth so llms.txt quality moves from partial to pass.
  4. 04Add per-endpoint code examples in at least one additional language (e.g. Python or JavaScript) alongside curl. Examples are highly realistic and copy-pasteable today, but curl-only limits polyglot agents.
  5. 05Ensure documentation pages (docs.buildkite.com / /docs paths) are explicitly enumerated in the sitemap index so crawlers that rely on sitemaps — not just llms.txt — can discover the full reference.

Execution coverage · INFORMATIONAL, UNSCORED

Whether an agent can actually complete a call and recover from errors is the deeper Audit layer — documented here, but not part of the Discry Score.

API access token (Bearer)OAuth (remote MCP server) Error format documented Rate limits documented Pagination documented Idempotency documented

Execution characteristics are documented to a high standard: Bearer-token auth with scoped, revocable tokens; machine-parseable JSON error bodies with per-endpoint error tables (400/422 with actionable messages); a dedicated rate-limits page with RateLimit-* and RateLimit-User-* headers plus 429/Retry-After backoff guidance; and page/per_page pagination surfaced via the Link header. Idempotency keys are not documented (build creation is noted as asynchronous instead).

See your own Discry Score.

Drop your API docs URL. See what an agent sees — in 60 seconds, free.

Discry your API — free