# Cormorant — complete documentation > Cormorant is a web search and fetch API for research projects and AI agents: SearXNG-first search with a Brave fallback, and page/PDF/document fetching through a pipeline of routes (a shared cache, HTTP with a browser fingerprint, a headless browser, a residential network when blocked, archives and alternative copies), with exact bytes, batch jobs of up to 50,000 items and per-request cost reporting. Base URL: `https://www.cormorant.ch`. Interactive reference: https://www.cormorant.ch/docs/reference. OpenAPI: https://www.cormorant.ch/openapi.json. # Endpoint reference (generated from the OpenAPI document) ## POST /v1/search — Search the web Auth: Bearer key. Operation id: `search`. Runs a web search and waits for the answer (typically 1–5 s, up to 150 s). Order: the shared search cache (7 days; `cache_ttl_s` sets the maximum acceptable age, `0` bypasses it) → Cormorant's SearXNG instances → the Brave Search API when SearXNG errors, times out, reports unresponsive engines or returns fewer than 3 usable results (`fallback_reason`). Results are deduplicated by normalised URL, keeping every snippet and engine. `cost_usd` is non-zero only when Brave was called. For many queries use `POST /v1/batch/search`. Request body (SearchBody): - `query` (string, required): Search query (whitespace and case are normalised for caching). - `language` (string | null): Result language, ISO 639-1 (optionally with region, e.g. `de-CH`). - `country` (string | null): Country bias, ISO 3166-1 alpha-2. - `max_results` (integer, default `20`): Maximum number of results. - `freshness` (`day` | `week` | `month` | `year` | `pd` | `pw` | `pm` | `py` | null): Only results from the last day, week, month or year (`pd`/`pw`/`pm`/`py` are accepted as aliases). - `cache_ttl_s` (integer | null): Maximum acceptable age of a cached answer in seconds; `0` bypasses the cache. Default: the cache's own TTL. - `project_ref` (string | null): Your own text stored with your usage (e.g. the entity this request is for). Responses: - `200` → SearchResponse: Merged results; also returned (with `error`) when nothing was found. - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## POST /v1/fetch — Fetch a URL Auth: Bearer key. Operation id: `fetch`. Fetches one URL and returns its text, metadata and provenance. A fresh entry of the shared page cache answers at once. Otherwise the ladder runs: the standard routes `direct` (HTTP with a Chrome fingerprint) → `browser` (Chromium) → `archive` (Wayback Machine), then the premium routes `proxy` (residential network) → `proxy_render` (rendering service), which are priced higher. It escalates only on blocking (403/429/451/5xx, bot challenges, network errors); thin text escalates `direct → browser` only. Definitive answers such as 404 are returned as is (`error: http_404`). `text` is the retained main text (per page for PDFs in `pdf_text_pages`), `content_kind` classifies the document, and `body_url` downloads the exact bytes. Control cost and latency with `allow_premium`, `tiers` and `timeout_s`. Request body (FetchBody): - `url` (string, required): http(s) URL to fetch. Private, loopback and link-local destinations are refused (`blocked_destination`). - `render` (`auto` | `never` | `always`, default `auto`): `auto`: start with plain HTTP and use a browser when needed; `never`: no browser (direct → archive → proxy); `always`: browser only (browser → proxy_render). - `max_bytes` (integer, default `15728640`): Maximum body size in bytes (enforced while streaming); larger bodies give `too_large`. Default 15 MB, max 50 MB. Batch items that leave it unset accept PDFs up to 200 MB. - `accept_pdf` (boolean, default `true`): Accept PDF documents; `false` turns them into `pdf_not_accepted`. - `cache_ttl_s` (integer | null): Maximum acceptable age of a cached answer in seconds; `0` bypasses the cache. Default: the cache's own TTL. - `min_text_chars` (integer, default `500`): HTML main text shorter than this counts as thin and escalates direct → browser (never to the premium routes). - `project_ref` (string | null): Your own text stored with your usage (e.g. the entity this request is for). - `allow_archive` (boolean, default `true`): Allow a recent Wayback Machine snapshot (≤ 365 days) when the live site blocks Cormorant's own tiers, answers 404/410, or its host name does not exist (`served_from_archive_because`). - `allow_alternates` (boolean, default `true`): When the page is a paywall teaser, disallowed by robots.txt, gone (404/410) or blocked on every tier, look for a verified open copy elsewhere (publisher API, open-access version via DOI, the same wire story on another site). The result then carries `alternate_of`, `alternate_source` and `match_score`. Never circumvents a paywall. - `allow_premium` (boolean, default `true`): `false`: never use the premium routes (`proxy`: residential network, `proxy_render`: rendering service), which are priced higher. - `tiers` (`direct` | `browser` | `archive` | `proxy` | `proxy_render`[] | null): Allowlist of tiers to try (in ladder order). An empty intersection with the plan gives `no_tier_allowed`; cache hits from other tiers are not used. - `timeout_s` (number | null): Deadline for the whole fetch across tiers, 5–150 s. When it runs out the result is `error: timeout` (or the best page so far). - `country` (string | null): Geotargeting for the premium routes (ISO 3166-1 alpha-2). - `domain_concurrency` (integer | null): How many requests to this URL's domain may run at the same time, counting this one (default 2 per Cormorant egress address). Up to 4 (the operator's `POLITENESS_BURST_MAX`) for a short, polite burst on a site you fetch several pages of, e.g. a company's own site; higher values are capped. Ignored while the site is pushing back (429s, challenges) and for hosts with their own site-wide limits (e.g. Wikimedia). - `user_agent` (string | null): User-Agent for `direct`, `browser` and `proxy` (not Web Render). Must be on the project's allowlist (set by the operator), else `403 user_agent_not_allowed`. - `accept_language` (string | null): Accept-Language header / browser locale. - `include_html` (boolean | null): Return the decoded HTML in `html`. Default `true` for `/v1/fetch`, `false` for batch items (the exact bytes are always available via `body_url`). Responses: - `200` → FetchResponse: The fetch result. HTTP 200 also for unsuccessful fetches: check `error`. - `503` → FetchResponse: `error: domain_busy`: the domain's polite slots are taken for longer than a synchronous call waits; retry after `Retry-After` / `retry_after_s` seconds, lower the parallelism per domain, or use a batch. - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## POST /v1/batch/search — Create a batch search job Auth: Bearer key. Operation id: `batchSearch`. Queues up to 50,000 searches as one job and returns its `job_id` at once. Items run with the same logic as `POST /v1/search`, admitted fairly across running jobs and retried with backoff on transient problems. Poll `GET /v1/jobs/{job_id}` or pass `callback_url`. Re-posting the same `idempotency_key` returns the existing job instead of creating a new one. Request body (BatchSearchBody): - `items` (SearchBody[], required): Search requests, same fields as `POST /v1/search`. - `idempotency_key` (string, required): Your unique key for this job. Re-posting the same key returns the existing job (`200`, `created: false`); the same key with a different kind or item count is `409`. - `callback_url` (string | null): Optional http(s) URL that receives a `POST` with the job summary when the job completes (3 attempts; private addresses are refused). Responses: - `202` → JobCreated: Job created. - `200` → JobCreated: Existing job (same idempotency key). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `409` → Error: The `idempotency_key` was already used for a job of a different kind or size. - `413` → Error: Too many items for one job (at most 50,000). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## POST /v1/batch/fetch — Create a batch fetch job Auth: Bearer key. Operation id: `batchFetch`. Queues up to 50,000 fetches as one job and returns its `job_id` at once. Items run with the same logic as `POST /v1/fetch`. Admission is domain-aware: a domain that dominates the batch drains at its polite rate while other domains proceed. Results are stored without HTML unless an item sets `include_html: true`; the exact bytes stay available via `body_url`. A batch item blocked on the standard routes is retried once after a cool-down before a premium route is used. Request body (BatchFetchBody): - `items` (FetchBody[], required): Fetch requests, same fields as `POST /v1/fetch` (`include_html` defaults to `false` here). - `idempotency_key` (string, required): Your unique key for this job. Re-posting the same key returns the existing job (`200`, `created: false`); the same key with a different kind or item count is `409`. - `callback_url` (string | null): Optional http(s) URL that receives a `POST` with the job summary when the job completes (3 attempts; private addresses are refused). Responses: - `202` → JobCreated: Job created. - `200` → JobCreated: Existing job (same idempotency key). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `409` → Error: The `idempotency_key` was already used for a job of a different kind or size. - `413` → Error: Too many items for one job (at most 50,000). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/jobs/{job_id} — Get a job's status and results Auth: Bearer key. Operation id: `getJob`. Returns the job's state and item counts; with `include=items`, also one page of items with their stored results (same shape as the single endpoints). Page with `offset`/`limit` until `offset >= counts.total`, or filter with `status=failed`. Jobs are visible only to their project and kept 30 days after they finish. Parameters: - `job_id` (path, string): - `include` (query, string | null): `items` to include a page of item results. - `offset` (query, integer): Index of the first item of the page. - `limit` (query, integer): Items per page (max 1000). - `status` (query, string | null): Only items in this state: `pending`, `queued`, `done`, `failed`, `cancelled`. Responses: - `200` → JobStatus: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## DELETE /v1/jobs/{job_id} — Cancel a job Auth: Bearer key. Operation id: `cancelJob`. Stops admitting items (within a few seconds): pending items become `cancelled`, queued items skip themselves, finished results are kept. Returns the job status. Parameters: - `job_id` (path, string): Responses: - `200` → JobStatus: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/bodies/{sha256} — Download the exact retrieved bytes Auth: Bearer key. Operation id: `getBody`. Streams the exact bytes of a fetch (`body_url` in fetch results) with the origin `Content-Type`, `Content-Length`, `ETag: ""`, `X-Body-Kind` and `Cache-Control: immutable`. `If-None-Match` gives `304`; `HEAD` returns the headers only. A project can download only bodies it has retrieved itself (fresh or from the shared cache); anything else is `404`. Bodies are kept while a cache entry points to them or a project retrieved them within the last 30 days. Parameters: - `sha256` (path, string): Responses: - `200`: The bytes, with the origin `Content-Type`. - `304`: Not modified (`If-None-Match` matched). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `503` → Error: A dependency is unavailable (authentication not configured, object storage down). Transient: retry with backoff. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## HEAD /v1/bodies/{sha256} — Body metadata (headers only) Auth: Bearer key. Operation id: `headBody`. Streams the exact bytes of a fetch (`body_url` in fetch results) with the origin `Content-Type`, `Content-Length`, `ETag: ""`, `X-Body-Kind` and `Cache-Control: immutable`. `If-None-Match` gives `304`; `HEAD` returns the headers only. A project can download only bodies it has retrieved itself (fresh or from the shared cache); anything else is `404`. Bodies are kept while a cache entry points to them or a project retrieved them within the last 30 days. Parameters: - `sha256` (path, string): Responses: - `200`: The bytes, with the origin `Content-Type`. - `304`: Not modified (`If-None-Match` matched). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `503` → Error: A dependency is unavailable (authentication not configured, object storage down). Transient: retry with backoff. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/sec/companies/resolve — Resolve a ticker, CIK or name to an SEC company Auth: Bearer key. Operation id: `secResolveCompany`. Looks the company up in SEC's `company_tickers.json` and `company_tickers_exchange.json` (cached 12 h) and returns its `cik`, `cik_padded`, `title`, `tickers` and `exchanges`. Give exactly one of `ticker`, `cik`, `name` or `isin`. A name can match several companies: then `ambiguous` is `true`, the company fields are `null` and `candidates` lists the matches. A CIK without a listed ticker is resolved from its submissions. Not an SEC identifier: `error: not_sec_filer` when the company is known otherwise (a non-US listing, the company register, GLEIF) with `hint` (`company`, `listing`, and `sec_candidates`: SEC filers with its name, e.g. the ADR), else `error: unknown_identifier`. Parameters: - `ticker` (query, string | null): US ticker, e.g. `TSN` (`BRK.B` = `BRK-B`); for a foreign private issuer its ADR ticker (`NVO`, `TM`). A listing elsewhere (`NESN.SW`, `XSWX:NESN`) answers `not_sec_filer` with what is known. - `cik` (query, string | null): CIK, e.g. `100493` or `0000100493`. - `name` (query, string | null): Company name or its beginning. - `isin` (query, string | null): ISIN: the issuer (company register, GLEIF), then the SEC filer with its name, e.g. Novo Nordisk's `DK0062498333` → `NVO`. - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecResolve: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## GET /v1/sec/companies/{cik}/submissions — A company's filings (normalised submissions) Auth: Bearer key. Operation id: `secSubmissions`. Company details and filings from `data.sec.gov/submissions/CIK##########.json` (cached 12 h, revalidated with conditional requests), including the pages of older filings it lists under `files` (only those that can hold filings after `since`). Filings are newest first with `accession`, `form`, `filing_date`, `report_date`, `primary_document`, `primary_document_url`, `index_url` and `is_amendment`. The raw SEC JSON: `body_url`. Parameters: - `cik` (path, string): CIK: `100493` or `0000100493`. - `forms` (query, string | null): Comma-separated form types, e.g. `10-K,20-F,40-F,10-K/A`. - `since` (query, string | null): Filed on or after (YYYY-MM-DD). - `until` (query, string | null): Filed on or before (YYYY-MM-DD). - `limit` (query, integer): Filings returned at most. - `history` (query, boolean): Also read the pages of older filings (`files`); `false` = SEC's recent list only (the last ~1000 filings). - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecSubmissions: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## GET /v1/sec/companies/{cik}/filings/latest — A company's latest annual report (or other form) Auth: Bearer key. Operation id: `secLatestFiling`. The newest filing of `forms` (by filing date): `accession`, `form`, `filing_date`, `report_date`, `primary_document`, `primary_document_url`, `index_url`, `is_amendment`, with the company's `cik`, `name`, `tickers`. Looks through older filings when the recent list has none. None at all: `error: not_found`. Parameters: - `cik` (path, string): CIK: `100493` or `0000100493`. - `forms` (query, string | null): Comma-separated form types; default `10-K,20-F,40-F` (annual reports; add e.g. `10-K/A` to include amendments). - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecLatestFiling: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## GET /v1/sec/companies/{cik}/facts — A company's XBRL facts Auth: Bearer key. Operation id: `secCompanyFacts`. All XBRL facts SEC holds for the company (`data.sec.gov/api/xbrl/companyfacts/CIK##########.json`, cached 12 h). The answer summarises it (`name`, concepts per taxonomy); download the complete JSON (often several MB) from `body_url`. Parameters: - `cik` (path, string): CIK: `100493` or `0000100493`. - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecFacts: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## GET /v1/sec/filings/{cik}/{accession} — The documents of a filing Auth: Bearer key. Operation id: `secFilingIndex`. The filing index: form, dates and every document (`sequence`, `description`, `document`, `type` such as `10-K` or `EX-21.1`, `size`, `url`), from the `-index.htm` page (or the folder's `index.json`). Filings never change, so this is cached for a year. Parameters: - `cik` (path, string): CIK: `100493` or `0000100493`. - `accession` (path, string): Accession number, with or without dashes. - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecFilingIndex: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## GET /v1/sec/search — EDGAR full-text search Auth: Bearer key. Operation id: `secSearch`. Searches the text of filings since 2001 with EDGAR full-text search (`efts.sec.gov`, cached 1 h): `total` and up to 100 `hits` per page, each with the filing's `accession`, `ciks`, `form`, the matching document and its `url`. Parameters: - `q` (query, string): Query (EDGAR syntax: `"exact phrase"`, `OR`, `-exclude`). - `forms` (query, string | null): Comma-separated form types, e.g. `10-K,20-F,40-F,10-K/A`. - `dateRange` (query, string | null): `all`, `10y`, `5y`, `1y`, `30d`, or `custom` with `startdt`/`enddt`. - `startdt` (query, string | null): Filed on or after (YYYY-MM-DD). - `enddt` (query, string | null): Filed on or before (YYYY-MM-DD). - `from` (query, integer): Offset of the first hit (pages of 100). - `project_ref` (query, string | null): Your own reference, kept in usage. - `timeout_s` (query, number | null): Overall deadline in seconds. Responses: - `200` → SecSearch: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## POST /v1/sec/documents/fetch — Fetch a filing document Auth: Bearer key. Operation id: `secFetchDocument`. Fetches one document of a filing — the primary HTML/iXBRL document, a PDF or an exhibit — and returns exactly what `POST /v1/fetch` returns (`text`, `pdf_text_pages`, `content_kind`, `body_url`, `sha256`, …), so the same artifact handling applies. Only documents under `/Archives/edgar/data/{cik}/{accession}/` can be fetched; filing documents are immutable and cached for a year. Request body (SecDocumentFetchBody): - `cik` (integer | string, required): CIK: 100493 or "0000100493". - `accession` (string, required): Accession number, with or without dashes. - `document` (string, required): File name within the filing (`primary_document` or a document of the filing index). - `max_bytes` (integer, default `52428800`): Largest body accepted. - `include_html` (boolean, default `true`): Return `html` for HTML documents. - `accept_pdf` (boolean, default `true`): Accept PDF documents. - `cache_ttl_s` (integer | null): Maximum acceptable age of a cached copy; filing documents are immutable and cached a year, so set this only to force a revalidation. - `timeout_s` (number | null): Overall deadline. - `project_ref` (string | null): Your own reference, kept in usage. Responses: - `200` → FetchResponse: The fetch result (same shape as `POST /v1/fetch`). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `504` → ErrorWithCode: No result within the synchronous wait (150 s, or `timeout_s` + 15 s). The request may still complete in the background; retry later (it is often a cache hit then) or use the batch endpoints. - `422` → HTTPValidationError: Validation Error ## POST /v1/batch/sec — Create a batch SEC EDGAR job Auth: Bearer key. Operation id: `batchSec`. Queues up to 50,000 SEC EDGAR operations as one job (kind `sec`) and returns its `job_id` at once: `resolve`, `submissions`, `latest`, `facts` and `filing` (results as from the matching `/v1/sec/...` endpoint, plus `op`) and `document` (the result of `POST /v1/sec/documents/fetch`, without `html` unless `include_html`). `submissions`, `latest` and `facts` take a `ticker` instead of a `cik`. Items share the cluster-wide SEC rate limit (at most 10 requests per second per egress, as SEC allows) with every other SEC request: a full limiter or a SEC backoff puts an item back in the queue instead of failing it. Each item is metered once as kind `sec` (counted like a fetch for key limits). Identifiers are checked when the job is created (`422 invalid_item`). Poll `GET /v1/jobs/{job_id}` or pass `callback_url`. Request body (BatchSecBody): - `items` (SecBatchItem[], required): Operations. - `idempotency_key` (string, required): Re-posting returns the same job. - `callback_url` (string | null): POSTed when the job finishes. Responses: - `202` → JobCreated: Job created. - `200` → JobCreated: Existing job (same idempotency key). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `409` → Error: The `idempotency_key` was already used for a job of a different kind or size. - `413` → Error: Too many items for one job (at most 50,000). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/account — Account, plan and current estimate Auth: Bearer key. Operation id: `getAccount`. The key's project usage this month by billing category, and — when the project belongs to a customer — the customer's usage, price plan and an estimate of the month's charges (rated now, not persisted). Usage is aggregated every 5 minutes (`usage_refreshed_at`). Projects without a customer get `customer`, `plan` and `estimate` as `null`. `keys` breaks the month down per API key: with an account admin key (`cma_…`) every key of the customer's projects (or of this project without a customer) plus usage from before per-key attribution; with an additional key (`cmr_…`) only that key. Prefix, label and role only, usage by category and the key's share of the usage charges. Responses: - `200` → Account: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/account/invoices — List invoices Auth: Bearer key. Operation id: `listInvoices`. The customer's issued, paid and void invoices (never drafts); an empty list for projects without a customer. Responses: - `200` → InvoiceList: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/account/invoices/{number} — Get an invoice with its lines Auth: Bearer key. Operation id: `getInvoice`. One invoice with its priced lines and the volume tiers applied. Another customer's invoice number, or a draft, is `404`. Parameters: - `number` (path, string): Responses: - `200` → InvoiceDetail: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/account/invoices/{number}/pdf — Download an invoice as PDF Auth: Bearer key. Operation id: `getInvoicePdf`. The invoice as a PDF file. Another customer's invoice number, or a draft, is `404`. Parameters: - `number` (path, string): Responses: - `200`: The PDF (`Content-Disposition: attachment`). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/status — Service status Auth: none. Operation id: `getStatus`. Overall status, every component's status with aggregate metrics, and open or recent incidents and maintenance. No authentication. Refreshed at most every 15 s (`Cache-Control: public, max-age=15`; `ETag` / `If-None-Match` supported). Always `200` while Cormorant answers: alert on `status`, not on the HTTP code. See the monitoring guide. Responses: - `200` → app__api__models__Status: Successful Response - `429` → Error: More than `STATUS_RATE_LIMIT_PER_MIN` (default 120) status requests per minute from this IP address. Retry after `Retry-After` seconds. ## GET /v1/status/summary — Service status (summary) Auth: none. Operation id: `getStatusSummary`. Only the overall `status` and `updated_at`: a tiny body for uptime monitors (match on `"status":"operational"`). No authentication; always `200` while Cormorant answers. Responses: - `200` → StatusSummary: Successful Response - `429` → Error: More than `STATUS_RATE_LIMIT_PER_MIN` (default 120) status requests per minute from this IP address. Retry after `Retry-After` seconds. ## GET /v1/status/components/{component_id} — One component's status Auth: none. Operation id: `getStatusComponent`. The status, reasons and metrics of one component, with the incidents and maintenance that name it. No authentication. Parameters: - `component_id` (path, string): Component id, e.g. `fetch` (see `GET /v1/status`). Responses: - `200` → StatusComponentDetail: Successful Response - `404` → Error: No such component in this deployment (see the component list of `GET /v1/status`). - `429` → Error: More than `STATUS_RATE_LIMIT_PER_MIN` (default 120) status requests per minute from this IP address. Retry after `Retry-After` seconds. - `422` → HTTPValidationError: Validation Error ## GET /v1/status/history — Status history Auth: none. Operation id: `getStatusHistory`. The worst status of every component per time bucket, oldest first, and its uptime over the period (`partial_outage` counts 30 % downtime, `major_outage` 100 %; buckets without data are `null` and not counted). The overall status is the component `_overall`. Kept 7 days. No authentication. Parameters: - `hours` (query, integer): How far back, in hours (1–168). Up to 24 h the buckets are 5 minutes; beyond, 1 hour (the worst status within the hour). Responses: - `200` → StatusHistory: Successful Response - `429` → Error: More than `STATUS_RATE_LIMIT_PER_MIN` (default 120) status requests per minute from this IP address. Retry after `Retry-After` seconds. - `422` → HTTPValidationError: Validation Error ## GET /v1/keys — API keys visible to this key Auth: Bearer key. Operation id: `listKeys`. The keys the caller may see, by prefix, label, role and limits — never the keys themselves. An account admin key (`cma_…`) sees every key of its customer's projects (or of its project without a customer), revoked keys included; a standard key (`cmr_…`) sees only itself. Responses: - `200` → ApiKeyList: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/keys/self — The calling key and its quota Auth: Bearer key. Operation id: `getKeySelf`. Prefix, label, role, project, status and limits of the calling key, with its current quota state (`quota`: each limit's use this month, as in `GET /v1/quota`). Responses: - `200` → ApiKeySelf: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/usage — Usage, charges and statistics Auth: Bearer key. Operation id: `getUsage`. Usage of the caller's scope with what it costs: requests, successes, cache hits, failures by error, latency p50/p95, bytes, the share of premium-route fetches and — for projects with a customer — billable quantities, included units used and the **charge at the customer's plan price** in the invoice currency (the same rating as the invoice; groups add up exactly to the estimate's usage lines). A standard key (`cmr_…`) sees only its own usage; an account admin key (`cma_…`) sees every key and project of the customer, revoked keys and usage from before per-key attribution (`key: null`) included. Combine `group_by` dimensions and filter by `key`, `project` and `category`. Figures come from aggregates refreshed every 5 minutes (`as_of`); `final` marks closed days. Provider cost (`cost_usd`, `budget`) appears only for projects without an external customer. `Accept: text/csv` returns the groups as CSV. Parameters: - `from` (query, string | null): First day, `YYYY-MM-DD` (UTC). Default: the first of the current month. - `to` (query, string | null): Last day (inclusive), `YYYY-MM-DD`. Default: today. - `group_by` (query, string | null): Comma-separated dimensions to group by, combinable: `key`, `project`, `category`, `day`, `hour` (hourly data is kept 35 days). Example: `group_by=key,day`. - `key` (query, string | null): Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). A key the caller cannot see is `404`. - `project` (query, string | null): Only these projects (comma-separated names). - `category` (query, string | null): Only these billing categories (comma-separated): `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. - `format` (query, string | null): `csv` instead of `Accept: text/csv`; default `json`. Responses: - `200` → Usage: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/usage/timeseries — Usage over time, for charts Auth: Bearer key. Operation id: `getUsageTimeseries`. The metrics of `GET /v1/usage` (requests, failures, latency, premium share, charges, …) per hour or day, one point per bucket — buckets without usage are zero — for charts. Same scope and filters as `GET /v1/usage`; `final` per point marks closed days. CSV with `Accept: text/csv`. Parameters: - `granularity` (query, string): `hour` (last 35 days) or `day`. - `from` (query, string | null): First day, `YYYY-MM-DD` (UTC). Default: today with `granularity=hour`, the first of the month with `day`. - `to` (query, string | null): Last day (inclusive), `YYYY-MM-DD`. Default: today. - `group_by` (query, string | null): One series per value of these dimensions (comma-separated): `key`, `project`, `category`. Default: one series. - `key` (query, string | null): Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). A key the caller cannot see is `404`. - `project` (query, string | null): Only these projects (comma-separated names). - `category` (query, string | null): Only these billing categories (comma-separated): `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. - `format` (query, string | null): `csv` instead of `Accept: text/csv`; default `json`. Responses: - `200` → UsageTimeseries: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/usage/domains — Top fetched domains Auth: Bearer key. Operation id: `getUsageDomains`. The busiest fetched domains of the caller's keys: requests, successes, success rate, premium-route fetches and premium share. Built from each key's 100 busiest domains per day, kept 90 days. Usage recorded before per-key attribution is not included. CSV with `Accept: text/csv`. Parameters: - `from` (query, string | null): First day, `YYYY-MM-DD` (UTC). Default: the first of the current month (90 days kept). - `to` (query, string | null): Last day (inclusive), `YYYY-MM-DD`. Default: today. - `group_by` (query, string | null): `key`: a top list per key instead of one list. - `key` (query, string | null): Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). A key the caller cannot see is `404`. - `project` (query, string | null): Only these projects (comma-separated names). - `limit` (query, integer): Domains per list (max 500). - `format` (query, string | null): `csv` instead of `Accept: text/csv`; default `json`. Responses: - `200` → UsageDomains: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/requests — Request ledger Auth: Bearer key. Operation id: `listRequests`. Every request of the caller's scope, newest first, in real time: request id, time, key, project, `project_ref`, kind, domain, tier, status, error, latency, cache hit, billing category and — for customers — its `charge` (its share of the day's charges for its key and category; `null` until aggregated, within 5 minutes). Page with `next_cursor` (also in the `Link` header). Kept as long as the request log (90 days). A standard key sees only its own requests. CSV with `Accept: text/csv`. Parameters: - `from` (query, string | null): First day, `YYYY-MM-DD` (UTC). Default: 29 days before today (kept 90 days). - `to` (query, string | null): Last day (inclusive), `YYYY-MM-DD`. Default: today. - `key` (query, string | null): Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). A key the caller cannot see is `404`. - `project` (query, string | null): Only these projects (comma-separated names). - `category` (query, string | null): Only these billing categories (comma-separated): `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. - `kind` (query, string | null): `search`, `fetch`, `company`, `sec` (SEC EDGAR) or `company_profile` (extended company profiles). - `outcome` (query, string | null): `succeeded` or `failed`. - `error` (query, string | null): Only this error code, e.g. `http_404`. - `domain` (query, string | null): Only fetches of this host. - `cached` (query, boolean | null): Only cache hits (`true`) or misses. - `job_id` (query, string | null): Only items of this batch job. - `project_ref` (query, string | null): Only this `project_ref`. - `limit` (query, integer): Rows per page (max 1000). - `cursor` (query, string | null): `next_cursor` of the previous page. - `format` (query, string | null): `csv` instead of `Accept: text/csv`; default `json`. Responses: - `200` → RequestLedger: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/requests/{request_id} — One request of the ledger Auth: Bearer key. Operation id: `getRequest`. One request by its `request_id` (`req_…`, or `:` for batch items) with the tiers tried (`attempts`). A request of another key (for a standard key) or another customer is `404`. Parameters: - `request_id` (path, string): Responses: - `200` → LedgerRequestDetail: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/quota — Limits, allowances and what is left Auth: Bearer key. Operation id: `getQuota`. Every limit that applies to the caller, with `limit`, `used`, `reserved`, `remaining`, `resets_at`, `projected_end_of_period` and `status` (`ok`, `warning` from 80 %, `exhausted`): the key's monthly spend cap (in the invoice currency) and request quotas, its and the project's requests per second, and — for account admin keys — every key of the account and the plan's included units. A standard key sees only itself. Responses: - `200` → Quota: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/alerts — List alert webhooks Auth: Bearer key. Operation id: `listAlerts`. The webhooks the caller may see (never their secrets): a standard key its own, an account admin key every webhook of the account. Responses: - `200` → AlertHookList: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## POST /v1/alerts — Register an alert webhook Auth: Bearer key. Operation id: `createAlert`. Registers a webhook that receives an event whenever a key, project or the account crosses 50, 80 or 100 % of a limit or allowance this month (spend cap, request quotas, provider budget, plan included units). The response contains the signing `secret` **once**; deliveries carry `X-Cormorant-Signature: t=,v1=.">`. Standard keys can register alerts for themselves only; account admin keys for any key, project or the whole account (`403 account_admin_required`). Request body (AlertIn): - `url` (string, required): HTTPS endpoint that receives `POST` deliveries (public address). - `scope` (`key` | `project` | `customer` | null): `key` (default for standard keys; their own key only), `project` or `customer` (default for account admin keys: the whole account). - `key` (string | null): With `scope=key`: the key's prefix (account admin keys); default the calling key. - `project` (string | null): With `scope=project`: project name. - `thresholds` (integer[] | null): Any of 50, 80, 100 (percent); default all three. Responses: - `201` → AlertHookCreated: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## DELETE /v1/alerts/{alert_id} — Delete an alert webhook Auth: Bearer key. Operation id: `deleteAlert`. Deletes a webhook the caller can see; anything else is `404`. Parameters: - `alert_id` (path, integer): Responses: - `204`: Deleted. - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/companies/search — Search companies Auth: Bearer key. Operation id: `searchCompanies`. Searches the register (trigram and full-text over current, former and trading names; exact on identifiers) plus the live sources covering `country` on the first page. Live hits (`in_register: false`) carry a `ref` to fetch them with `GET /v1/companies/{ref}`, which adds them to the register. Needs `q` or `identifier`. Live sources are asked concurrently and waited for at most 2.5 s; `live_sources` says per source what happened (`ok`, `failed`, `timeout`, or `skipped` while a source backs off or its canaries fail) and how long it took. A near-exact register match (score ≥ 0.95) answers without asking the live sources unless `live=true`; `live=false` searches the register only. Parameters: - `q` (query, string | null): Name (any language, legal-form words ignored). - `country` (query, string | null): ISO 3166-1 alpha-2; also routes live search to that country's sources. - `identifier` (query, string | null): `scheme:value` exact match. - `status` (query, string | null): Register status, e.g. `active`. - `legal_form` (query, string | null): Legal-form category, e.g. `corporation`. - `limit` (query, integer): Results per page. - `cursor` (query, string | null): `next_cursor` of the previous page. - `live` (query, boolean | null): Also search the live sources (first page). Default: unless the register already has a near-exact match (score ≥ 0.95); `true` always, `false` never. Responses: - `200` → CompanySearch: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## POST /v1/companies/resolve — Resolve a company (entity resolution) Auth: Bearer key. Operation id: `resolveCompany`. Best match plus ranked candidates. Identifiers are tried first (exact, `match_score` 1); then name, country and city are scored over the register and the live sources. `match` is set only for a confident match, and is added to the register unless `persist` is `false`: its `id` is the `cmp_` id and `in_register` is `true`, so no second call is needed. A confident register match answers without asking any source; otherwise the live sources are asked concurrently (at most 2.5 s, `live_sources` as for search) and the other sources' data about a newly stored match is fetched in the background. Request body (ResolveBody): - `name` (string | null): Company name. - `country` (string | null): ISO 3166-1 alpha-2. - `city` (string | null): City. - `identifiers` (string[]): `scheme:value` identifiers (exact; tried first). - `isin` (string | null): ISIN of a security the company issued; the same as adding `isin:` to `identifiers` (check digit verified, `422 invalid_identifier`). - `limit` (integer, default `5`): Candidates to return. - `persist` (boolean, default `true`): Add the `match` to the register when it was found only at a live source, so it comes back with its `cmp_` id and `in_register: true` (as `GET /v1/companies/{ref}` would). `false`: leave the register untouched; the match then may have `id: null` and a `scheme:value` `ref`. Responses: - `200` → CompanyResolved: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/companies/changes — Change feed of the register Auth: Bearer key. Operation id: `companyChanges`. Every change of the register, oldest first: which company and which top-level fields (fetch the record for the values). Poll with `next_cursor`; or register a webhook with `POST /v1/companies/watches`. Parameters: - `since` (query, string | null): ISO 8601 time to start from (first call). - `cursor` (query, string | null): `next_cursor` of the previous call. - `country` (query, string | null): Only this country. - `limit` (query, integer): Changes per page. Responses: - `200` → CompanyChanges: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/companies/watches — List company watches Auth: Bearer key. Operation id: `listCompanyWatches`. The project's watches (without secrets). Responses: - `200` → CompanyWatches: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## POST /v1/companies/watches — Watch companies (webhook) Auth: Bearer key. Operation id: `createCompanyWatch`. Registers a webhook: changes of these companies are POSTed as `company.changed` (signed like usage alerts: `X-Cormorant-Signature: t=,v1=.">`). Identifiers not yet in the register are looked up first; unknown ones are `404`. The `secret` is returned only here. Request body (WatchBody): - `companies` (string[], required): `cmp_…` ids or `scheme:value` identifiers. - `url` (string, required): HTTPS endpoint receiving `company.changed` POSTs. Responses: - `201` → CompanyWatch: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `404` → Error: Not found, or not visible to this project. - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## DELETE /v1/companies/watches/{watch_id} — Delete a company watch Auth: Bearer key. Operation id: `deleteCompanyWatch`. Stops deliveries and removes the watch. Parameters: - `watch_id` (path, string): Watch id (`wch_…`). Responses: - `204`: Deleted. - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/companies/sources — Company data sources Auth: Bearer key. Operation id: `companySources`. Every source with countries, identifier schemes, capabilities, precedence, licence/terms URL, freshness classes and live health (the same status as the status page's "Company data" group). Not metered. Responses: - `200` → CompanySources: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## POST /v1/companies/batch — Create a company batch job Auth: Bearer key. Operation id: `batchCompanies`. Queues companies as a job of kind `company`: `get` (records, refreshing stale data), `refresh` (re-fetch everything) or `resolve`. Each item is metered as one `company` request when it runs. Poll `GET /v1/jobs/{job_id}` (item results have the shape of `GET /v1/companies/{id}` under `record`, or of `POST /v1/companies/resolve`), or download `GET /v1/companies/batch/{job_id}.csv`. CSV input: `POST /v1/companies/batch/csv`. Request body (CompanyBatchBody): - `action` (`get` | `refresh` | `resolve`, default `get`): `get`: the record (refreshing stale data), `refresh`: re-fetch from every source, `resolve`: entity resolution of `name`/`country`/`city`/`identifiers`. - `items` (BatchItem[], required): Up to 50,000 items. - `include` (`people` | `ownership` | `filings` | `financials` | `events`[]): Sub-resources to include in each record (`get`/`refresh`). - `max_age` (integer | null): As `max_age` of `GET /v1/companies/{id}`. - `persist` (boolean, default `true`): `resolve`: add confident matches to the register (as `POST /v1/companies/resolve`). - `idempotency_key` (string, required): Re-posting returns the same job. - `callback_url` (string | null): POSTed when the job finishes. Responses: - `202` → JobCreated: Successful Response - `200` → JobCreated: Existing job (same idempotency key). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `409` → Error: The `idempotency_key` was already used for a job of a different kind or size. - `413` → Error: Too many items for one job (at most 50,000). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## POST /v1/companies/batch/csv — Create a company batch job from CSV Auth: Bearer key. Operation id: `batchCompaniesCsv`. The same job as `POST /v1/companies/batch`, from a CSV upload. Parameters: - `idempotency_key` (query, string): Re-posting returns the same job. - `action` (query, `get` | `refresh` | `resolve`): What to do per row. - `include` (query, string | null): Comma-separated sub-resources (`get`/`refresh`). - `max_age` (query, integer | null): As for `GET /v1/companies/{id}`. - `persist` (query, boolean): `resolve`: add confident matches to the register. - `callback_url` (query, string | null): POSTed when the job finishes. Responses: - `202` → JobCreated: Successful Response - `200` → JobCreated: Existing job (same idempotency key). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `409` → Error: The `idempotency_key` was already used for a job of a different kind or size. - `413` → Error: Too many items for one job (at most 50,000). - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/companies/batch/{job_id}.csv — Company batch results as CSV Auth: Bearer key. Operation id: `companyBatchCsv`. One row per item: `index, input, status, company_id, name, country, company_status, legal_form, match_score, identifiers, error`. Parameters: - `job_id` (path, string): Job id. Responses: - `200`: CSV - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `404` → Error: Not found, or not visible to this project. - `422` → HTTPValidationError: Validation Error - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/schemas/company — JSON Schema of company/v1 Auth: Bearer key. Operation id: `companySchema`. The JSON Schema (2020-12) of the `company/v1` record, with the identifier scheme catalogue in `x-identifier-schemes`. Responses: - `200`: JSON Schema (2020-12). - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: The key's customer account is suspended (`customer_suspended`). ## GET /v1/companies/{company_ref} — Get a company record Auth: Bearer key. Operation id: `getCompany`. The full record from the register. A company not yet in the register is fetched from the sources that know the identifier and added. Each source's data older than its freshness class (or `max_age`) is re-fetched first, concurrently, for at most 2 s: a slower source, or one whose canaries fail, finishes in the background (`pending`; the record has its data on the next call) and a source backing off is not asked (`skipped`), so the stored record is served at once with its `freshness`. `refresh` says per source and group what was refreshed, fresh, pending, skipped or failed (a failing source never fails the answer). Fields carry `provenance`; `include=claims` adds every claim. A merged `cmp_` id answers with the surviving record (`redirected_from`). Parameters: - `company_ref` (path, string): `cmp_…` (merged ids redirect) or `scheme:value`, e.g. `ch_uid:CHE-101.329.561`, `lei:BFM8T61CT2L1QCEMIK50`, `us_cik:320193`, `isin:CH0038863350` (the issuer, via GLEIF's ISIN-to-LEI mapping). - `max_age` (query, integer | null): Maximum acceptable age of each source's data in seconds; older data is re-fetched first. Default: the source's freshness class (e.g. registry core 24 h, people 7 days). - `refresh` (query, boolean): `true` re-fetches from every source now. - `include` (query, string | null): Comma-separated: `people`, `ownership`, `filings`, `financials`, `events`, `claims`. - `sources` (query, string | null): Comma-separated source names: use (and show values from) only these. Responses: - `200` → CompanyRecord: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `404` → Error: Not found, or not visible to this project. - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/companies/{company_ref}/history — Field history of a company (claims timeline) Auth: Bearer key. Operation id: `companyHistory`. Every value each source asserted, newest observation first: when it was first seen (`observed_at`), confirmed, and when it stopped being current (`valid_to`, `current: false`). Claims are never overwritten. Parameters: - `company_ref` (path, string): `cmp_…` (merged ids redirect) or `scheme:value`, e.g. `ch_uid:CHE-101.329.561`, `lei:BFM8T61CT2L1QCEMIK50`, `us_cik:320193`, `isin:CH0038863350` (the issuer, via GLEIF's ISIN-to-LEI mapping). - `field` (query, string | null): Only this field, e.g. `names.legal` or `addresses`. - `limit` (query, integer): Claims per page. - `cursor` (query, string | null): `next_cursor` of the previous page. Responses: - `200` → CompanyPage: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `404` → Error: Not found, or not visible to this project. - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /v1/companies/{company_ref}/{resource} — A company's events, people, ownership, relationships, filings, financials or sources Auth: Bearer key. Operation id: `companySubresource`. One sub-resource with paging, refreshed like the record (the field group of the sub-resource, e.g. `people` 7 days). `sources` lists the raw documents with `body_url`. Parameters: - `company_ref` (path, string): `cmp_…` (merged ids redirect) or `scheme:value`, e.g. `ch_uid:CHE-101.329.561`, `lei:BFM8T61CT2L1QCEMIK50`, `us_cik:320193`, `isin:CH0038863350` (the issuer, via GLEIF's ISIN-to-LEI mapping). - `resource` (path, `events` | `people` | `ownership` | `relationships` | `filings` | `financials` | `sources`): Sub-resource. - `max_age` (query, integer | null): Maximum acceptable age of each source's data in seconds; older data is re-fetched first. Default: the source's freshness class (e.g. registry core 24 h, people 7 days). - `refresh` (query, boolean): `true` re-fetches from every source now. - `include_former` (query, boolean): `people`: also former officers. - `limit` (query, integer): Items per page. - `cursor` (query, string | null): `next_cursor` of the previous page. Responses: - `200` → CompanyPage: Successful Response - `401` → Error: Missing or invalid API key (`WWW-Authenticate: Bearer`). - `403` → ErrorWithCode: Not allowed: the key's customer account is suspended (`customer_suspended`, any `/v1` endpoint), a `user_agent` is not on the project's allowlist (`user_agent_not_allowed`, fetch endpoints), or the key may not use this category (`key_category_not_allowed`, search and fetch endpoints). - `404` → Error: Not found, or not visible to this project. - `429`: A limit is reached. The project's rate limit (default 200 requests/s, one-second window; plain `detail`) or the key's own (`key_rate_limited`): retry after `Retry-After` (1 s). The key's monthly request quota (`key_quota_exhausted`) or spend cap (`key_budget_exhausted`): `Retry-After` is the time until the month resets; see `GET /v1/quota`. - `422` → HTTPValidationError: Validation Error ## GET /health — Liveness Auth: none. Operation id: `health`. `200 {"status": "ok"}` while the API process is up. Responses: - `200` → Health: Successful Response ## GET /ready — Readiness Auth: none. Operation id: `ready`. `200` when PostgreSQL and Redis answer and at least one SearXNG instance is registered, else `503`. Object storage is reported but does not gate readiness. Responses: - `200` → Ready: Successful Response - `503` → Ready: A required dependency is unavailable. ## Response objects ### SearchResponse Answer of `POST /v1/search` (and of each batch search item). - `query` (string, required): The query as sent. - `results` (SearchResult[], required): Merged, deduplicated results; empty when nothing was found or on error. - `provider` (`searxng` | `brave` | null, required): Who answered: Cormorant's SearXNG instances, or the Brave Search API fallback. `null` when no provider answered. - `fallback_reason` (string | null, required): Why the Brave fallback was considered (`searxng_error`, `searxng_timeout`, `no_searxng_instance`, `unresponsive_engines`, `captcha_engines`, `too_few_results`); `null` when SearXNG's answer was good enough. - `unresponsive_engines` (string[], required): SearXNG engines that failed for this query. - `cached` (boolean, required): True when served from the shared search cache. - `cost_usd` (number | null, required): Cormorant's provider spend caused by this request in USD (search fallback and premium routes; cache hits and standard routes cause no provider cost). `null` for keys of an external customer: what you are charged is in `GET /v1/usage` and `GET /v1/requests`. - `billing_category` (string, required): The billing category the request counts toward (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`): it maps the request to the meters and charges of your plan. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `error` (string | null, required): Set only when the providers could not answer (e.g. `budget_exhausted`, `brave_http_429`, `searxng_timeout`). A search that found nothing has `results: []` and `error: null`. - `request_id` (string, required): Identifier of this request (`req_…`; batch items use `:`). Quote it when asking the operator about a request. - `debug_url` (string | null): Only in the answer to a debug request (header `X-Cormorant-Debug: 1` with a key enabled for debugging; available on request): a link to this request's diagnostic record for Cormorant support. Quote it when you report a problem; it cannot be opened with an API key. ### FetchResponse Answer of `POST /v1/fetch` (and of each batch fetch item). Returned with HTTP 200 also for unsuccessful fetches: check `error`. - `url` (string, required): The URL as requested. - `final_url` (string | null, required): URL after redirects. For `tier = archive` the archived copy's URL; `null` when unknown (`proxy_render`). - `redirects` (Redirect[], required): Redirect hops before `final_url`. - `status` (integer | null, required): HTTP status of the returned page (on `all_tiers_failed`: the last status seen). - `content_type` (string | null, required): What the body is (rendered pages: `text/html`). - `origin_content_type` (string | null, required): Content type the origin sent; `null` when unknown. - `headers` (map, required): Allowlisted headers of the origin's final response (content type/length/language/disposition, ETag, Last-Modified, Cache-Control, Expires, Date, Link, X-Robots-Tag). Never cookies. Empty for `proxy_render`. - `tier` (`direct` | `browser` | `archive` | `proxy` | `proxy_render` | `source` | null, required): Tier that produced the page; `null` when none did. - `attempts` (Attempt[], required): Every tier tried, in order. - `timing` (FetchTiming, required): Latency components: queue, polite per-domain waits, upstream. - `html` (string | null): Decoded HTML for HTML pages. Present on `/v1/fetch` unless `include_html=false`; omitted from batch item results unless the item set `include_html=true`. - `text` (string, required): Retained text (main text for HTML, pages joined for PDFs, decoded document for JSON/XML/plain text, empty for binary). Identical on cache hits. - `pdf_text_pages` (PdfPage[] | null, required): Per-page text for PDFs; `null` otherwise. Stops early when `text_truncated`. - `pdf_page_count` (integer | null, required): Number of pages of a PDF (all of them, also when the text stops early); `null` otherwise. - `text_truncated` (boolean, required): True when PDF text extraction stopped at its limits (pages or characters): `text` and `pdf_text_pages` cover the first pages only. - `title` (string | null, required): Document title, if found. - `meta` (ArticleMeta | null, required): Article metadata for HTML pages (also on cache hits and batch results); `null` for other content kinds and on errors. - `sha256` (string | null, required): SHA-256 of the retrieved bytes (see `body_kind`). - `bytes` (integer, required): Size of the retrieved bytes. - `cached` (boolean, required): True when served from the shared page cache. - `revalidated` (boolean, required): True when an expired cache entry was confirmed unchanged by the origin (conditional GET on the direct tier answered 304, attempt reason `not_modified`): the cached content is returned with `cached: true` and no cost, and its freshness is renewed. - `retrieved_at` (string | null, required): When the bytes were retrieved (ISO 8601, UTC). - `etag` (string | null, required): Origin ETag, if any. - `last_modified` (string | null, required): Origin Last-Modified, if any. - `cost_usd` (number | null, required): Cormorant's provider spend caused by this request in USD (search fallback and premium routes; cache hits and standard routes cause no provider cost). `null` for keys of an external customer: what you are charged is in `GET /v1/usage` and `GET /v1/requests`. - `billing_category` (string, required): The billing category the request counts toward (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`): it maps the request to the meters and charges of your plan. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `archived_at` (string | null, required): Snapshot time when `tier = archive`; capture time of a `common_crawl` copy (`tier = source`). - `served_from_archive_because` (`domain_dead` | `not_found` | null, required): Set when the archived copy (`tier = archive`) stands in for a page the live site cannot give: `domain_dead` (the host name does not exist) or `not_found` (the site answered 404/410; `attempts` shows it). Only with `allow_archive`. `null` otherwise. - `body_url` (string | null, required): Where the exact retrieved bytes can be downloaded with the same key (`GET /v1/bodies/{sha256}`); `null` when not stored. - `body_kind` (`origin` | `rendered_dom` | null, required): What `sha256` covers: `origin` = the entity body the origin (or archive) sent; `rendered_dom` = the document serialized by a browser after scripts ran. - `content_kind` (`html` | `pdf` | `text` | `json` | `xml` | `binary` | null, required): Classification of the content (from the content type, sniffed when missing or generic). - `paywall` (`none` | `metered` | `hard` | `unknown` | null, required): Paywall verdict for HTML pages (`null` otherwise): `hard` = `text` is only the teaser; `metered` = a meter or a declared paywall, but the full text came through; `unknown` = weak signals only; `none`. Cormorant detects paywalls and never circumvents them. - `paywall_evidence` (string[], required): Signals behind the verdict, e.g. `jsonld_not_free`, `vendor_piano`, `paywall_markup`, `truncation_markup`, `phrase_de`, `metered_fr`, `length_mismatch`. - `alternate_of` (string | null, required): Set when the content is a verified open copy found elsewhere (`tier: source`): the URL that was asked for. `final_url` is the copy's URL. - `alternate_source` (`publisher_feed` | `guardian_api` | `open_access` | `syndication` | `common_crawl` | null, required): Where the copy came from: `publisher_feed` (the item in the publisher's own full-text RSS/Atom feed; `final_url` is the feed), `guardian_api` (Guardian Content API), `open_access` (open-access version of the DOI via OpenAlex), `syndication` (the same story on another site, e.g. wire copy), `common_crawl` (batch fetches only: the newest Common Crawl capture of a gone or blocked page; `archived_at` is the capture time). - `match_score` (number | null, required): How the copy was verified: 1.0 for exact identifiers (DOI, Guardian path, feed item link, Common Crawl capture of the same URL); for `syndication` and title-matched `publisher_feed` items the share of the original lead's word 5-grams found in the copy (≥ 0.5). - `primary_sources` (PrimarySource[], required): For paywalled or unavailable pages: links on the page to primary documents (court decisions, government/regulator pages, press releases, DOIs, PDFs). - `hint` (string | null): For known hard domains (sites that block automated access, see Fetching → Known hard domains): the recommended route for the site, on errors and when an archive or alternative copy stands in. `null` otherwise. - `alternative_urls` (string[]): With `hint` on errors: other places to look for this page (e.g. its Wayback Machine URL, an OpenAlex search for a ResearchGate paper). - `text_complete` (boolean | null, required): `false` when `text` is only the teaser of a paywalled article (`paywall: hard`); `null` for non-HTML content. - `error` (string | null, required): `null` on success, else an error code such as `invalid_url`, `blocked_destination`, `robots_disallowed`, `too_large`, `pdf_not_accepted`, `http_`, `all_tiers_failed`, `domain_unavailable`, `domain_dead`, `timeout`, `no_tier_allowed`, `politeness_timeout`, `domain_busy` (synchronous calls, HTTP 503: the domain is busy for longer than a synchronous call waits; see `retry_after_s`); for SEC EDGAR URLs also `sec_not_configured`, `rate_limited`, `sec_unavailable`, `redirect_not_allowed`. - `error_detail` (string | null, required): With an error: one line listing every attempt (tier, reason, premium bytes) and which premium routes were tried, or why none was (e.g. skipped because the domain refused them on recent fetches). For humans and logs; match on `error` and `attempts` in code. - `premium_routes_tried` (string[], required): Premium routes (`proxy`, `proxy_render`) actually tried for this request, in order (skipped ones excluded). Their bytes are in `attempts[].premium_bytes`. - `retry_after_s` (integer | null, required): With `error: domain_unavailable`: seconds until the domain is tried again. Retrying earlier answers the same error at once. With `domain_busy`: the expected wait for one of the domain's polite slots. Also set with `rate_limited` and `sec_unavailable` (SEC EDGAR). - `request_id` (string, required): Identifier of this request (`req_…`; batch items use `:`). Quote it when asking the operator about a request. - `debug_url` (string | null): Only in the answer to a debug request (header `X-Cormorant-Debug: 1` with a key enabled for debugging; available on request): a link to this request's diagnostic record for Cormorant support. Quote it when you report a problem; it cannot be opened with an API key. ### JobCreated Answer of `POST /v1/batch/search` and `POST /v1/batch/fetch`. - `job_id` (string, required): Job identifier (UUID). - `created` (boolean, required): `true` (HTTP 202) for a new job, `false` (HTTP 200) when the `idempotency_key` matched an existing job, which is returned instead. ### JobStatus Status of a batch job; with `?include=items` also a page of items. - `job_id` (string, required): Job identifier (UUID). - `kind` (string, required): Kind of items. - `status` (`queued` | `running` | `completed` | `cancelled`, required): Job state. - `counts` (JobCounts, required): Number of items in each state. - `created_at` (string, required): Creation time (ISO 8601). - `updated_at` (string, required): Last change (ISO 8601). - `finished_at` (string | null, required): When the job completed or was cancelled. - `callback_url` (string | null, required): Callback URL given at creation. - `callback_status` (string | null, required): Outcome of the completion callback (`delivered_`, `http_`, `blocked_destination`, `error_`); `null` before delivery. - `items_url` (string, required): Relative URL of the first page of items. - `items` (JobItem[] | null): Present with `include=items`: one page of items. - `offset` (integer | null): Present with `include=items`: offset of this page. - `limit` (integer | null): Present with `include=items`: page size. ### SecResolve Answer of `GET /v1/sec/companies/resolve`. The company fields are those of the single match; `null` when the query is ambiguous. - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `title` (string | null): Company name (SEC's conformed name). - `tickers` (string[]): Ticker symbols, primary first. - `exchanges` (string[]): Exchanges of the tickers (from company_tickers_exchange.json). - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `ambiguous` (boolean, default `false`): More than one company matches (see `candidates`). - `candidates` (SecCompany[]): Every match (at most 10), best first. ### SecSubmissions Answer of `GET /v1/sec/companies/{cik}/submissions`. - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `title` (string | null): Company name (SEC's conformed name). - `tickers` (string[]): Ticker symbols, primary first. - `exchanges` (string[]): Exchanges of the tickers (from company_tickers_exchange.json). - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `name` (string | null): Company name. - `entity_type` (string | null): Entity type. - `sic` (string | null): SIC code. - `sic_description` (string | null): SIC description. - `category` (string | null): Filer category. - `fiscal_year_end` (string | null): Fiscal year end (MMDD). - `state_of_incorporation` (string | null): State or country of incorporation. - `ein` (string | null): Employer identification number. - `lei` (string | null): Legal entity identifier. - `website` (string | null): Website, when SEC lists one. - `phone` (string | null): Business phone. - `business_address` (map | null): `street1`, `street2`, `city`, `state_or_country`, `state_or_country_description`, `zip_code`. - `former_names` (map[]): Earlier names: `name`, `from`, `to`. - `filings` (SecFiling[]): Filings matching the filters, newest first (at most `limit`), from the recent list and the history pages. - `filings_total` (integer, default `0`): Matching filings before `limit`. - `truncated` (boolean, default `false`): `limit` cut the list. - `files` (SecHistoryFile[]): Pages of older filings and whether they were read. ### SecLatestFiling Answer of `GET /v1/sec/companies/{cik}/filings/latest`: the newest filing of the requested forms, with its company. - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `accession` (string | null): Accession number. - `form` (string | null): Form type. - `filing_date` (string | null): Filing date. - `report_date` (string | null): Period of report. - `acceptance_datetime` (string | null): When EDGAR accepted it. - `act` (string | null): Securities act. - `file_number` (string | null): SEC file number. - `film_number` (string | null): Film number. - `items` (string | null): 8-K items. - `core_type` (string | null): Core form type. - `size` (integer | null): Size of the submission in bytes. - `is_xbrl` (boolean | null): Has XBRL data. - `is_inline_xbrl` (boolean | null): Primary document is inline XBRL. - `primary_document` (string | null): File name of the primary document. - `primary_document_description` (string | null): Its description. - `primary_document_url` (string | null): URL of the primary document. - `index_url` (string | null): Filing index page. - `is_amendment` (boolean | null): An amendment (`…/A` form). - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `name` (string | null): Company name. - `tickers` (string[]): Ticker symbols. - `exchanges` (string[]): Exchanges. ### SecFacts Answer of `GET /v1/sec/companies/{cik}/facts`: a summary; the complete XBRL company facts JSON is at `body_url`. - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `name` (string | null): Entity name. - `taxonomies` (map): Concepts per taxonomy. - `concepts` (integer, default `0`): Concepts in total. - `bytes` (integer, default `0`): Size of the JSON. ### SecFilingIndex Answer of `GET /v1/sec/filings/{cik}/{accession}`. - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `accession` (string | null): Accession number. - `accession_nodash` (string | null): Accession number without dashes. - `company` (string | null): Filer name from the index page. - `form` (string | null): Form type. - `filing_date` (string | null): Filing date. - `report_date` (string | null): Period of report. - `accepted` (string | null): Acceptance time as SEC shows it. - `is_amendment` (boolean, default `false`): An amendment (`…/A` form). - `index_url` (string | null): Filing index page. - `folder_url` (string | null): Filing folder. - `index_source` (string | null): `index_htm` (with types and descriptions) or `index_json` (the folder listing, when the index page has no document table). - `documents` (SecDocument[]): Documents of the filing. ### SecSearch Answer of `GET /v1/sec/search` (EDGAR full-text search, 100 hits per page). - `error` (string | null): `null` on success, else `unknown_identifier` (SEC knows no company by this ticker, CIK, name or ISIN, and neither does Cormorant), `not_sec_filer` (the company is known — a non-US listing, the company register, GLEIF — but not registered with the SEC under it; see `hint`), `not_found` (no such filing, document or filing of the requested forms), `ambiguous`, `invalid_identifier`, `rate_limited`, `sec_unavailable`, `sec_not_configured`, or a fetch error (`timeout`, `too_large`, `robots_disallowed`, …). - `message` (string | null): Human-readable detail of `error`. - `hint` (SecHint | null): With `not_sec_filer` (and on a resolve by ISIN): what Cormorant knows — `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing, `MIC:SYMBOL`) and `sec_candidates` (SEC filers with the company's name, e.g. the ADR of a foreign private issuer: retry with that ticker). - `retry_after_s` (integer | null): With `rate_limited` / `sec_unavailable`: seconds before a retry makes sense. - `cached` (boolean, default `false`): `true` when every SEC document came from Cormorant's cache. - `retrieved_at` (string | null): When the oldest SEC document used was retrieved (ISO 8601). - `source_url` (string | null): The main SEC document the answer is built from. - `body_url` (string | null): Exact bytes of that document (the raw SEC JSON/HTML), like `body_url` of a fetch. - `sha256` (string | null): SHA-256 of those bytes. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `request_id` (string, required): Request id. - `query` (string | null): The query. - `total` (integer | null): Matching documents. - `total_relation` (string | null): `eq` (exact) or `gte` (at least). - `hits` (SecSearchHit[]): This page of hits. ### Account Answer of `GET /v1/account`. - `project` (AccountProject, required): Project of the API key. - `period` (AccountPeriod, required): Current billing period. - `usage_refreshed_at` (string | null, required): When usage was last aggregated (every 5 minutes). - `project_usage` (map, required): This project's usage this period, by billing category. - `customer` (AccountCustomer | null, required): The owning customer; `null` for projects without one. - `customer_usage` (map | null): The customer's usage (all its projects); absent without a customer. - `plan` (PlanSummary | null, required): Price plan of the period; `null` without one. - `estimate` (Estimate | null, required): Charges so far this period; `null` without a customer. - `keys` (AccountKey[], required): This period per API key. Account admin key: all keys of the customer's projects (or of this project without a customer), plus usage recorded before per-key attribution (`prefix` null). Additional key: only itself. ### InvoiceList Answer of `GET /v1/account/invoices`. - `invoices` (Invoice[], required): Issued, paid and void invoices of the customer. ### InvoiceDetail An invoice with its lines. - `number` (string, required): Invoice number. - `status` (`issued` | `paid` | `void`, required): State. - `period` (string, required): Billed month. - `period_start` (string, required): First day. - `period_end` (string, required): Last day. - `currency` (string, required): Currency. - `plan_label` (string | null, required): Plan name. - `complimentary` (boolean, required): Invoice of a complimentary plan (total 0). - `subtotal` (string, required): Decimal as a string, in the invoice currency. - `discount_total` (string, required): Decimal as a string, in the invoice currency. - `net_total` (string, required): Decimal as a string, in the invoice currency. - `vat_rate` (string, required): VAT rate in percent. - `vat_amount` (string, required): Decimal as a string, in the invoice currency. - `total` (string, required): Decimal as a string, in the invoice currency. - `vat_note` (string | null, required): VAT note printed on the invoice. - `issued_at` (string | null, required): Issue time. - `due_at` (string | null, required): Due date. - `paid_at` (string | null, required): Payment date. - `voided_at` (string | null, required): Void time. - `pdf_url` (string, required): Relative URL of the PDF. - `lines` (InvoiceLine[], required): Priced lines. ### app__api__models__Status Service status for monitoring (no authentication). - `status` (`operational` | `maintenance` | `degraded` | `partial_outage` | `major_outage`, required): Overall status: the worst component status, where a major outage of a non-core component (anything but api, search, fetch, storage) counts as a partial outage; `maintenance` when the only components not operational are under maintenance. - `description` (string, required): Human-readable overall status. - `updated_at` (string, required): When this snapshot was computed (ISO 8601, UTC; at most ~15 s old). - `window_s` (integer, required): Window of the recent-request metrics, in seconds. - `components` (StatusComponent[], required): Every component, in display order. - `incidents` (StatusIncident[], required): Unresolved incidents and those resolved within 7 days, newest first. - `maintenance` (StatusMaintenance[], required): Scheduled and in-progress maintenance, and windows completed within 7 days, soonest first. ### StatusSummary Overall status only, for uptime monitors. - `status` (`operational` | `maintenance` | `degraded` | `partial_outage` | `major_outage`, required): Overall status. - `description` (string, required): Human-readable overall status. - `updated_at` (string, required): When the status was computed (ISO 8601, UTC). ### StatusComponentDetail One component with the notes that concern it. - `id` (string, required): Stable component id: `api`, `search`, `fetch`, `browser_rendering`, `archive`, `premium_routes`, `batch_jobs`, `storage`, and `company_` for each active company data source (e.g. `company_gleif`). - `group` (string | null): Set for grouped components: `company_data` for the company data sources. A company source counts at most as `degraded` for the overall status. - `name` (string, required): Display name. - `description` (string, required): What the component covers. - `status` (`operational` | `maintenance` | `degraded` | `partial_outage` | `major_outage`, required): Current status. - `reasons` (string[], required): Stable codes saying why the component is not operational (empty when it is): `no_workers`, `queue_lag`, `low_success_rate`, `high_latency`, `engines_reduced`, `engines_unavailable`, `tier_errors`, `scheduler_stalled`, `jobs_waiting`, `database_unreachable`, `cache_unreachable`, `object_storage_unreachable`, `sec_backoff`, `throttled`, `incident`, `maintenance`; company data sources also `canary_failed`, `source_backoff`, `schema_drift`, `parse_failures`. - `metrics` (object, required): Aggregate numbers behind the status (rates 0–1). Counts and rates cover the last `window_s` seconds (600); a component with fewer than its minimum sample there uses the last hour instead, and its `metrics.window_s` (or `_window_s`) says which. `success_rate` is `null` only when even that window is short of the minimum; `requests`/`attempts` and `failures` are always given. Keys per component are listed in the API contract; new keys may be added. - `updated_at` (string, required): When the status was computed (ISO 8601, UTC). - `incidents` (StatusIncident[], required): Incidents naming this component. - `maintenance` (StatusMaintenance[], required): Maintenance naming this component, or the whole service. ### StatusHistory Status history per component. - `hours` (integer, required): Period covered, in hours. - `bucket_s` (integer, required): Bucket size in seconds (300 or 3600). - `start` (string | null, required): Start of the first bucket (ISO 8601, UTC). - `components` (map, required): Per component id, plus `_overall` for the overall status. ### ApiKeyList Answer of `GET /v1/keys`. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `keys` (ApiKeyInfo[], required): Account admin key: every key of the customer's projects (or of the project without a customer), revoked keys included. Standard key: only itself. ### ApiKeySelf Answer of `GET /v1/keys/self`: the calling key and its quota state. - `prefix` (string, required): First 10 characters; `cma_…` = account admin key, `cmr_…` = standard key. - `label` (string | null, required): Label set by the operator. - `role` (`account_admin` | `standard`, required): `account_admin` sees the whole customer account; `standard` only itself. - `project` (string, required): Project the key belongs to. - `status` (`active` | `revoked`, required): Key status. - `created_at` (string, required): Creation time. - `revoked_at` (string | null, required): Revocation time. - `last_used_at` (string | null, required): Last authenticated request (accurate to about a minute). - `is_self` (boolean, required): This is the calling key. - `limits` (KeyLimitsInfo | null, required): Limits set by the operator; `null` = none (only the project's). - `usage` (QuotaKeyUsage | null, required): Use this month. - `quota` (QuotaEntry[], required): Each limit of the key, as in `GET /v1/quota`. ### Usage Answer of `GET /v1/usage`. - `project` (string, required): Project of the API key. - `key` (string, required): Prefix of the calling key. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `scope` (UsageScope, required): The customer, projects and keys the figures cover (after filters). - `from` (string, required): First day (inclusive, UTC). - `to` (string, required): Last day (inclusive, UTC). - `as_of` (string | null, required): The figures include every request recorded before this time (aggregates are refreshed every 5 minutes); `null` before the first refresh. - `final` (boolean, required): Every day of the range is closed: the figures will not change. - `currency` (string | null, required): Currency of `charge` (the customer's invoice currency); `null` without a customer. - `totals` (UsageMetrics, required): Everything in the scope and filters. - `search` (UsageKind, required): Search usage (compatibility block). - `fetch` (UsageKind, required): Fetch usage, sync and batch (compatibility block). - `company` (UsageKind | null): Company data requests (`/v1/companies/*`, category and meter `company`); present when there are any. - `sec` (UsageKind, required): SEC EDGAR usage: `/v1/sec/…` and fetches of SEC hosts (kind `sec`; categories `sec`, `sec_cached`). Empty when there was none. - `company_profile` (UsageKind, required): Extended company profile usage (kind `company_profile`; categories `company_profile`, `company_profile_cached`). Empty when there was none. - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer. - `budget` (Budget | null): The project's monthly provider budget — only for projects without an external customer. - `spend_budget` (SpendBudget | null): This month's spend against the spend caps, per key and per customer (`cap: null` when no cap is set). - `group_by` (string[] | null): The dimensions of `groups`. - `groups` (UsageGroup[] | null): One entry per combination of the `group_by` dimensions with usage, oldest first for time dimensions, then busiest first. - `keys` (UsageKey[] | null): With `group_by` containing `key`: per API key, busiest first (compatibility list; `groups` has the full metrics). - `account` (UsageAccountPeriod[] | null): Account admin keys of a customer: per billing period of the range, the account-level lines and totals. ### UsageTimeseries Answer of `GET /v1/usage/timeseries`. - `project` (string, required): Project of the API key. - `key` (string, required): Prefix of the calling key. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `scope` (UsageScope, required): The customer, projects and keys covered. - `from` (string, required): First day. - `to` (string, required): Last day. - `granularity` (`hour` | `day`, required): Bucket size. - `as_of` (string | null, required): Every request recorded before this time is included. - `final` (boolean, required): Every bucket is closed. - `currency` (string | null, required): Currency of `charge`. - `group_by` (string[], required): Dimensions of the series. - `series` (UsageSeries[], required): One series per `group_by` value. ### UsageDomains Answer of `GET /v1/usage/domains`. - `project` (string, required): Project of the API key. - `key` (string, required): Prefix of the calling key. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `scope` (UsageScope, required): The customer, projects and keys covered. - `from` (string, required): First day. - `to` (string, required): Last day. - `domains` (DomainUsage[] | null): Without `group_by`: the busiest domains, busiest first. - `keys` (KeyDomains[] | null): With `group_by=key`: a top list per key, busiest key first. ### RequestLedger Answer of `GET /v1/requests`. - `project` (string, required): Project of the API key. - `key` (string, required): Prefix of the calling key. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `scope` (UsageScope, required): The customer, projects and keys covered. - `from` (string, required): First day. - `to` (string, required): Last day. - `as_of` (string, required): Now: the ledger is real time. - `currency` (string | null, required): Currency of `charge`. - `limit` (integer, required): Page size. - `next_cursor` (string | null, required): Pass as `cursor` for the next (older) page; `null` on the last page. - `requests` (LedgerRequest[], required): Newest first. ### LedgerRequestDetail Answer of `GET /v1/requests/{request_id}`. - `request_id` (string, required): `req_…`, or `:` for batch items. - `time` (string, required): When it was recorded. - `key` (string | null, required): Prefix of the key that made it; `null` before per-key attribution. - `key_role` (`account_admin` | `standard` | null, required): Role of that key. - `project` (string | null, required): Project. - `project_ref` (string | null, required): Your `project_ref`. - `job_id` (string | null, required): Batch job of the item. - `kind` (`search` | `fetch` | `company` | `sec` | `company_profile`, required): Request kind (`company_profile`: extended company profiles). - `domain` (string | null, required): Fetched host. - `tier` (string | null, required): Fetch: the tier that answered; search: the provider. - `status` (integer | null, required): HTTP status of the origin. - `ok` (boolean, required): Succeeded. - `error` (string | null, required): Error code. - `latency_ms` (integer | null, required): Latency in ms. - `cached` (boolean, required): Served from the shared cache. - `bytes` (integer, required): Bytes retrieved. - `premium` (boolean, required): Served by (or charged for) a premium route. - `category` (string, required): Billing category. - `charge` (string | null): Customers: the request's share of its key's charges for that day and category (their average price; 6 decimals, `currency`). `null` until aggregated (within 5 minutes); settles when the day is final. - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer. - `reason` (string | null, required): Why it ended as it did. - `attempts` (LedgerAttempt[], required): The tiers tried, in order. ### Quota Answer of `GET /v1/quota`. - `key` (string, required): Prefix of the calling key. - `role` (`account_admin` | `standard`, required): Role of the calling key. - `period` (string, required): Current month (`YYYY-MM`, UTC). - `resets_at` (string, required): Start of the next month. - `as_of` (string, required): Now. - `usage_refreshed_at` (string | null, required): Last aggregation (spend is exact up to it). - `keys` (QuotaKey[], required): The calling key (standard key) or every active key of the account (account admin key). - `project` (QuotaProject, required): Limits of the calling key's project. - `customer` (QuotaCustomer | null): Account admin keys of a customer: plan allowances. - `events` (AlertEvent[], required): Thresholds (50 / 80 / 100 %) crossed this month in the caller's scope, newest first. - `spend_budget` (SpendBudget | null): This month's spend against the spend caps, per key and per customer (`cap: null` when no cap is set). - `alert_email` (QuotaAlertEmail | null): Alert e-mail deliverability ("alert e-mail undeliverable" after a hard bounce or complaint). Webhooks are not affected. ### AlertHookList Answer of `GET /v1/alerts`. - `alerts` (AlertHook[], required): Webhooks visible to the caller. ### AlertHookCreated Answer of `POST /v1/alerts`. - `id` (integer, required): Webhook id. - `url` (string, required): Delivery URL. - `scope` (`key` | `project` | `customer`, required): What it watches: one key, one project (and its keys) or the whole account. - `key` (string | null, required): With `scope=key`: key prefix. - `project` (string | null, required): With `scope=project`: project name. - `thresholds` (integer[], required): Percentages delivered. - `created_at` (string, required): Registration time. - `last_delivery_at` (string | null, required): Last delivery attempt. - `last_status` (string | null, required): `delivered_`, `http_`, `error_` or `blocked_destination`. - `secret` (string, required): Signing secret (`whsec_…`), shown **only here**: verify `X-Cormorant-Signature` with it. ### CompanySearch Answer of `GET /v1/companies/search`. - `results` (CompanySummary[], required): Ranked by `match_score`. - `next_cursor` (string | null, required): Pass as `cursor` for more register results; `null` at the end. - `live_sources` (LiveSource[], required): Sources searched live (first page only, routed by `country`). - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). ### CompanyResolved Answer of `POST /v1/companies/resolve`. - `match` (CompanySummary | null, required): The best candidate when it is a confident match (score ≥ 0.85 and clearly ahead of the next), else `null`. Added to the register unless `persist` was `false`: `id` is then the `cmp_` id and `in_register` is `true`. - `candidates` (CompanySummary[], required): Ranked candidates. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). - `live_sources` (LiveSource[]): Sources searched live (empty when the register or an identifier answered). ### CompanyChanges Answer of `GET /v1/companies/changes`. - `items` (CompanyChange[], required): Oldest first. - `next_cursor` (string, required): Pass as `cursor` to continue (poll with it). - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). ### CompanyWatches Answer of `GET /v1/companies/watches`. - `watches` (CompanyWatch[], required): The project's watches. ### CompanyWatch A webhook watching a list of companies. - `id` (string, required): Watch id. - `url` (string, required): Your HTTPS endpoint. - `companies` (string[], required): Watched Cormorant ids. - `created_at` (string, required): ISO 8601. - `last_delivery_at` (string | null, required): Last delivery attempt. - `last_status` (string | null, required): `delivered`, `http_` or an error class. - `secret` (string | null): HMAC secret to verify `X-Cormorant-Signature`; returned only when the watch is created. ### CompanySources Answer of `GET /v1/companies/sources`. - `sources` (CompanySource[], required): Every source, active first. ### CompanyRecord A company record (`company/v1`) plus what this request refreshed. - `schema_version` (`company/v1`, default `company/v1`): Always `company/v1`. - `id` (string, required): Stable Cormorant id (`cmp_`); survives merges (redirects). - `identifiers` (app__companies__model__Identifier[]): Identifiers in every known scheme. - `names` (Names): Names. - `legal_form` (LegalForm | null): Legal form. - `status` (app__companies__model__Status | null): Register status. - `jurisdiction` (Jurisdiction | null): Jurisdiction and register. - `addresses` (Address[]): Addresses. - `dates` (Dates): Incorporation, registration and deregistration dates. - `purpose` (map): Purpose text per language (ISO 639-1). - `activities` (Activity[]): Activity codes. - `capital` (Capital[]): Share capital. - `size` (Size | null): Size (official/filing data only). - `people` (Person[] | null): Officers (`include=people`). - `ownership` (Ownership[] | null): Ownership (`include=ownership`). - `relationships` (Relationship[]): Links to other companies. - `filings` (Filing[] | null): Filings (`include=filings`). - `financials` (Financials[] | null): Financials (`include=financials`). - `events` (Event[] | null): Timeline (`include=events`). - `securities` (Security[]): Listed securities. - `web` (Web | null): Web presence. - `sources` (SourceDocument[]): Raw documents behind the record (newest first). - `freshness` (SourceFreshness[]): Per source and field group. - `quality` (Quality, required): Completeness score and source conflicts. - `provenance` (map): Claim behind each resolved field. - `claims` (Claim[] | null): Every claim (`include=claims`). - `redirected_from` (string | null): Id that was asked for, when it was merged into this record. - `updated_at` (string, required): Last change of the record (ISO 8601). - `refresh` (CompanyRefresh[]): Per source and field group: refreshed, fresh or failed. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). ### CompanyPage A page of a company sub-resource or of its history. - `company_id` (string, required): Cormorant id. - `items` (object[], required): Items in the shape of the matching company/v1 field (`events`, `people`, `ownership`, …); history items are claims. - `next_cursor` (string | null, required): Pass as `cursor` for the next page. - `refresh` (CompanyRefresh[], required): What this request refreshed. - `billing` (Billing | null): What this request is charged under your plan (`charge`, `currency`, `within_allowance`, units per meter), priced like the invoice. Absent on answers that were not recorded (invalid input, limits). ### Health Liveness. - `status` (`ok`, required): Always `ok` when the process answers. ### Ready Readiness: PostgreSQL, Redis and at least one SearXNG instance. - `status` (`ready` | `not_ready`, required): Overall readiness. - `checks` (map, required): Result per dependency (never secrets). - `version` (string, required): API version. ### SearchResult One deduplicated result. Every distinct snippet and engine seen for the URL is kept, in order of first appearance. - `url` (string, required): Result URL as returned by the engines. - `title` (string | null, required): Result title (whitespace-normalised). - `snippets` (string[], required): All distinct snippets for this URL across engines and providers. - `engines` (string[], required): Engines that returned this URL (`brave_api` for the Brave Search API). - `published_at` (string | null, required): Publication date reported by an engine (ISO 8601), if any. - `rank` (integer, required): 1-based position after merging. ### Billing What this request is charged under your plan (never provider cost). - `category` (string, required): Billing category (same as `billing_category`). - `billable` (boolean, required): The request counts on the invoice (failed requests do not, unless the plan bills them). - `currency` (string | null, required): Invoice currency; `null` for projects without a customer plan. - `charge` (string | null, required): Charge for this request at your plan price: tiers, included units, contract discount (decimal string, 6 places). `0.000000` inside the allowance. Excludes the monthly base fee, minimum top-up and VAT. `null` without a customer plan. - `within_allowance` (boolean | null, required): `true` when every unit fell inside the plan's included units (charge 0); `null` when nothing was priced (not billable, or no meter counts this category). - `units` (BillingUnit[], required): Per plan meter. ### Redirect One redirect hop before `final_url`. - `url` (string, required): URL that answered with a redirect. - `status` (integer, required): Redirect status code. - `location` (string, required): Resolved target of the `Location` header. ### Attempt One tier tried while answering the request, in order. - `tier` (string, required): Tier name (`direct`, `browser`, `archive`, `proxy`, `proxy_render`), `source` for the alternative-source steps, or a pseudo-step such as `request_timeout` or `dns` (the host name does not exist, see `domain_dead`). - `status` (integer | null, required): HTTP status seen, if any. - `reason` (string, required): Outcome: `ok`, `thin_content`, `not_modified`, `challenge_`, `http_`, `timeout`, `tls_error`, `dns_error`, `dns_nxdomain`, `domain_dead`, `network_*`, `not_configured`, `budget_exhausted`, `premium_skipped_domain_refuses`, `hard_domain_skip`, `archive_not_found`, premium-route reasons (`…_captcha_rejected`, `…_no_credits`, `…_render_503`), `paywall`, `paywall_skip_premium`, `archive_blocked_paywall`, `alternate_`, `syndication_candidates_`, … - `ms` (integer, required): Time spent on this attempt in milliseconds. - `premium_bytes` (integer, default `0`): Bytes the residential network billed for this attempt (premium route `proxy`; `0` for standard routes and `proxy_render`, which bills per call). - `wait_ms` (integer, default `0`): Part of `ms` spent waiting for a polite per-domain slot (busy domain). - `upstream_ms` (integer, default `0`): Part of `ms` spent on the tier itself: the site, the browser, the archive or the premium route's provider (`ms - wait_ms`). - `detail` (string | null): Present only when set: for `proxy_render`, the rendering difficulty used (`low`, `medium`, `high`); for `alternate_`, the copy's URL. ### FetchTiming Where the time of a fetch went (milliseconds). - `queue_ms` (integer | null, required): Waiting in Cormorant's queues before a worker ran the request (synchronous: from the API to a worker; batch: since the item was admitted, politeness requeues included; plus the render queue after a browser handoff). `null` for cache hits answered by the API. - `politeness_wait_ms` (integer, required): Waiting for polite per-domain slots (sum of `attempts[].wait_ms`). - `upstream_ms` (integer, required): Time in tier attempts and alternative sources (sum of `attempts[].upstream_ms`). ### PdfPage Extracted text of one PDF page. - `page` (integer, required): 1-based page number. - `text` (string, required): Text of the page. ### ArticleMeta Article metadata of an HTML page, from JSON-LD, OpenGraph/`article:*`, `citation_*` meta tags and the HTML itself. Each field is `null` when the page does not say. - `author` (string | null, required): Author name(s); several are separated by `; `. - `published_at` (string | null, required): Publication date, ISO 8601: `YYYY-MM-DDTHH:MM:SSZ` (UTC) when the page gives a time zone, without zone when it gives a time but no zone, `YYYY-MM-DD` (or `YYYY-MM`, `YYYY`) when it gives only a date. - `modified_at` (string | null, required): Last modification date (same format as `published_at`). - `language` (string | null, required): Language tag (BCP 47), e.g. from JSON-LD `inLanguage` or ``. - `canonical_url` (string | null, required): Canonical URL the page declares (``, else JSON-LD `url` / `og:url`), absolute. - `publisher` (string | null, required): Publishing organisation. - `site_name` (string | null, required): Name of the site or periodical (`og:site_name`, JSON-LD `WebSite`, `citation_journal_title`). - `description` (string | null, required): Summary the page gives of itself (at most 2,000 characters). - `section` (string | null, required): Section or category (first `articleSection` / `article:section`). - `word_count` (integer | null, required): Words in `text` (what was retrieved, not what the page declares). ### PrimarySource - `url` (string, required): Absolute URL of the linked document. - `kind` (`court` | `government` | `press_release` | `doi` | `pdf`, required): What the link points to. - `text` (string | null, required): The link text, if any. ### JobCounts Item counts by state. - `total` (integer, required): Items in the job. - `pending` (integer, required): Waiting for admission to the queues. - `queued` (integer, required): Admitted and queued or running. - `done` (integer, required): Finished successfully. - `failed` (integer, required): Finished with an error (result and error kept). - `cancelled` (integer, required): Cancelled before running. ### JobItem One item of a job with its stored result. - `index` (integer, required): 0-based position in the submitted `items`. - `status` (`pending` | `queued` | `done` | `failed` | `cancelled`, required): Item state. - `input` (object, required): The item as submitted (defaults omitted). - `result` (object | null, required): The search or fetch response (same shape as the single endpoint); `null` while not finished or when the stored result is unavailable. - `error` (string | null, required): Error code of a failed item; `result_unavailable` when the stored result cannot be read right now (retry later); `cancelled` for cancelled items. ### SecHint What Cormorant knows about a company SEC does not know by the given identifier. - `listing` (string | null): The home listing (`MIC:SYMBOL`), when known. - `company` (SecHintCompany | null): The company in Cormorant's company register. - `sec_candidates` (object[]): SEC filers with the company's name (`cik`, `cik_padded`, `title`, `tickers`, `exchanges`), e.g. the ADR of a foreign private issuer: retry with that ticker. - `same_symbol_on_sec` (object[] | null): A non-US listing whose symbol SEC also lists (maybe, not surely, the same company). - `candidates` (object[] | null): With `ambiguous`: the SEC companies that match. ### SecCompany A company as SEC lists it. - `cik` (integer | null): Central Index Key. - `cik_padded` (string | null): CIK as 10 digits. - `title` (string | null): Company name (SEC's conformed name). - `tickers` (string[]): Ticker symbols, primary first. - `exchanges` (string[]): Exchanges of the tickers (from company_tickers_exchange.json). ### SecFiling One filing of a company. - `accession` (string, required): Accession number. - `form` (string | null, required): Form type. - `filing_date` (string | null, required): Filing date. - `report_date` (string | null, required): Period of report (fiscal period end). - `acceptance_datetime` (string | null, required): When EDGAR accepted it. - `act` (string | null, required): Securities act. - `file_number` (string | null, required): SEC file number. - `film_number` (string | null, required): Film number. - `items` (string | null, required): 8-K items. - `core_type` (string | null, required): Core form type. - `size` (integer | null, required): Size of the submission in bytes. - `is_xbrl` (boolean | null, required): Has XBRL data. - `is_inline_xbrl` (boolean | null, required): Primary document is inline XBRL. - `primary_document` (string | null, required): File name of the primary document. - `primary_document_description` (string | null, required): Its description. - `primary_document_url` (string | null, required): URL of the primary document (fetch it with `POST /v1/sec/documents/fetch`). - `index_url` (string, required): Filing index page. - `is_amendment` (boolean, required): An amendment (`…/A` form). ### SecHistoryFile A page of a company's older filings (SEC's `filings.files`). - `name` (string, required): Page file name. - `filing_count` (integer | null, required): Filings on the page. - `filing_from` (string | null, required): Oldest filing date on the page. - `filing_to` (string | null, required): Newest filing date on the page. - `url` (string, required): The page on data.sec.gov. - `fetched` (boolean, required): Whether its filings are included in this answer. - `body_url` (string | null, required): Exact bytes of the page, when fetched. ### SecDocument A document of a filing. - `sequence` (integer | null, required): Sequence number in the filing. - `description` (string | null, required): Description. - `document` (string, required): File name. - `type` (string | null, required): Document type (form or exhibit). - `size` (integer | null, required): Size in bytes. - `url` (string, required): URL (fetch it with `POST /v1/sec/documents/fetch`). ### SecSearchHit A document found by EDGAR full-text search. - `accession` (string | null, required): Accession number. - `ciks` (integer[], required): Filer CIKs. - `display_names` (string[], required): Filer names as EDGAR shows them. - `form` (string | null, required): Form type of the filing. - `root_forms` (string[], required): Base form types. - `file_type` (string | null, required): Type of the matching document. - `file_description` (string | null, required): Its description. - `file_date` (string | null, required): Filing date. - `period_ending` (string | null, required): Period of report. - `document` (string | null, required): File name of the matching document. - `url` (string | null, required): URL of the matching document. - `index_url` (string | null, required): Filing index page. - `score` (number | null, required): Relevance score. ### AccountProject The key's project. - `id` (integer, required): Project id. - `name` (string, required): Project name. ### AccountPeriod The current billing period (calendar month, UTC). - `period` (string, required): `YYYY-MM`. - `from` (string, required): First day. - `to` (string, required): Last day. ### CategoryUsage Metered usage of one billing category (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`). - `requests` (integer, required): Requests. - `succeeded` (integer, required): Successful requests. - `bytes` (integer, required): Bytes retrieved. - `bytes_ok` (integer, required): Bytes of successful requests. - `premium_bytes` (integer, required): Bytes through premium routes. - `premium_bytes_ok` (integer, required): Bytes through premium routes, successful requests only. - `cost_usd` (string | null, required): Provider cost in USD (decimal string); `null` for projects of an external customer, who see their charges instead. - `cost_ok_usd` (string | null, required): Provider cost of successful requests in USD (decimal string); `null` for projects of an external customer. ### AccountCustomer The billed account that owns the project. - `name` (string, required): Customer name. - `company_name` (string | null, required): Legal company name. - `currency` (`USD` | `CHF` | `EUR`, required): Invoice currency. - `status` (`active` | `suspended`, required): Account status; keys of suspended accounts get `403 customer_suspended`. - `complimentary` (boolean, required): Complimentary accounts are rated at list price and then discounted 100 %. ### PlanSummary The price plan version that prices the period. - `id` (integer, required): Plan version id. - `code` (string, required): Plan code. - `version` (integer, required): Plan version. - `name` (string, required): Plan name. - `currency` (string, required): Plan currency. - `complimentary` (boolean, required): A complimentary plan. ### Estimate Current-period charges, rated now and not persisted. - `period` (string, required): `YYYY-MM`. - `from` (string, required): First day. - `to` (string, required): Last day. - `currency` (string, required): Currency. - `lines` (EstimateLine[], required): Priced lines. - `subtotal` (string, required): Decimal as a string, in the invoice currency. - `discount_total` (string, required): Decimal as a string, in the invoice currency. - `net_total` (string, required): Decimal as a string, in the invoice currency. - `vat_rate` (string, required): VAT rate in percent. - `vat_amount` (string, required): Decimal as a string, in the invoice currency. - `total` (string, required): Decimal as a string, in the invoice currency. - `complimentary` (boolean, required): Complimentary account or plan (billed at list price, then discounted 100 %). - `list_total` (string, required): Net before the complimentary discount: the list value of what was used. ### AccountKey Month-to-date usage of one API key of the customer (or of the project without a customer). Keys are identified by prefix and label only. - `prefix` (string | null, required): First 10 characters of the key; `null` for usage recorded before per-key attribution. - `label` (string | null, required): Label set by the operator. - `role` (`account_admin` | `standard` | null, required): `account_admin` (prefix `cma_`) or `standard` (`cmr_`); `null` for usage recorded before per-key attribution. - `project` (string | null, required): Project of the key. - `active` (boolean, required): The key is not revoked. - `requests` (integer, required): Requests this period. - `succeeded` (integer, required): Successful requests. - `usage` (map, required): Usage by billing category. - `amount` (string | null, required): The key's share of the period's usage charges at list price (before account-level fees, discounts and the complimentary credit; Decimal as a string, in the invoice currency.) — `null` without a customer. ### Invoice An issued, paid or void invoice (drafts are never shown). - `number` (string, required): Invoice number. - `status` (`issued` | `paid` | `void`, required): State. - `period` (string, required): Billed month. - `period_start` (string, required): First day. - `period_end` (string, required): Last day. - `currency` (string, required): Currency. - `plan_label` (string | null, required): Plan name. - `complimentary` (boolean, required): Invoice of a complimentary plan (total 0). - `subtotal` (string, required): Decimal as a string, in the invoice currency. - `discount_total` (string, required): Decimal as a string, in the invoice currency. - `net_total` (string, required): Decimal as a string, in the invoice currency. - `vat_rate` (string, required): VAT rate in percent. - `vat_amount` (string, required): Decimal as a string, in the invoice currency. - `total` (string, required): Decimal as a string, in the invoice currency. - `vat_note` (string | null, required): VAT note printed on the invoice. - `issued_at` (string | null, required): Issue time. - `due_at` (string | null, required): Due date. - `paid_at` (string | null, required): Payment date. - `voided_at` (string | null, required): Void time. - `pdf_url` (string, required): Relative URL of the PDF. ### InvoiceLine One line of an invoice. - `position` (integer, required): Line number. - `kind` (string, required): Line kind (see the estimate). - `description` (string, required): Line text. - `quantity` (string | null, required): Quantity (decimal string). - `unit` (string | null, required): Unit. - `unit_price` (string | null, required): Flat unit price, if any. - `amount` (string, required): Decimal as a string, in the invoice currency. - `tiers` (object[], required): Volume tiers applied (`from`, `to`, `quantity`, `unit_price`, `amount`). ### StatusComponent One subsystem. Components that are not part of the deployment (e.g. premium routes when none is configured) are not listed. - `id` (string, required): Stable component id: `api`, `search`, `fetch`, `browser_rendering`, `archive`, `premium_routes`, `batch_jobs`, `storage`, and `company_` for each active company data source (e.g. `company_gleif`). - `group` (string | null): Set for grouped components: `company_data` for the company data sources. A company source counts at most as `degraded` for the overall status. - `name` (string, required): Display name. - `description` (string, required): What the component covers. - `status` (`operational` | `maintenance` | `degraded` | `partial_outage` | `major_outage`, required): Current status. - `reasons` (string[], required): Stable codes saying why the component is not operational (empty when it is): `no_workers`, `queue_lag`, `low_success_rate`, `high_latency`, `engines_reduced`, `engines_unavailable`, `tier_errors`, `scheduler_stalled`, `jobs_waiting`, `database_unreachable`, `cache_unreachable`, `object_storage_unreachable`, `sec_backoff`, `throttled`, `incident`, `maintenance`; company data sources also `canary_failed`, `source_backoff`, `schema_drift`, `parse_failures`. - `metrics` (object, required): Aggregate numbers behind the status (rates 0–1). Counts and rates cover the last `window_s` seconds (600); a component with fewer than its minimum sample there uses the last hour instead, and its `metrics.window_s` (or `_window_s`) says which. `success_rate` is `null` only when even that window is short of the minimum; `requests`/`attempts` and `failures` are always given. Keys per component are listed in the API contract; new keys may be added. ### StatusIncident An incident posted by the operator. - `id` (string, required): Incident id. - `title` (string, required): Short title. - `status` (`investigating` | `identified` | `monitoring` | `resolved`, required): Incident status. - `impact` (`none` | `degraded` | `partial_outage` | `major_outage`, required): Impact while unresolved: the named components show at least this status. - `components` (string[], required): Affected component ids. - `created_at` (string, required): Opened (ISO 8601, UTC). - `updated_at` (string, required): Last update (ISO 8601, UTC). - `resolved_at` (string | null, required): Resolved (ISO 8601, UTC), or `null`. - `updates` (StatusUpdate[], required): Timeline, newest first. ### StatusMaintenance A maintenance window posted by the operator. - `id` (string, required): Maintenance id. - `title` (string, required): Short title. - `status` (`scheduled` | `in_progress` | `completed`, required): `scheduled` before `starts_at`, `in_progress` until `ends_at` (or until completed early), then `completed`. - `components` (string[], required): Affected component ids (empty: the whole service). - `starts_at` (string, required): Planned start (ISO 8601, UTC). - `ends_at` (string, required): Planned end (ISO 8601, UTC). - `created_at` (string, required): Announced (ISO 8601, UTC). - `updated_at` (string, required): Last update (ISO 8601, UTC). - `completed_at` (string | null, required): Completed (ISO 8601, UTC), or `null`. - `updates` (StatusUpdate[], required): Timeline, newest first. ### StatusSeries History of one component. - `uptime` (number | null, required): Uptime over the period, 0–1 (`partial_outage` buckets count 30 % downtime, `major_outage` 100 %); `null` without data. - `statuses` (`operational` | `maintenance` | `degraded` | `partial_outage` | `major_outage` | null[], required): Worst status per bucket, oldest first; `null` where nothing was recorded. ### ApiKeyInfo An API key as its customer sees it: prefix, label and role, never the key. - `prefix` (string, required): First 10 characters; `cma_…` = account admin key, `cmr_…` = standard key. - `label` (string | null, required): Label set by the operator. - `role` (`account_admin` | `standard`, required): `account_admin` sees the whole customer account; `standard` only itself. - `project` (string, required): Project the key belongs to. - `status` (`active` | `revoked`, required): Key status. - `created_at` (string, required): Creation time. - `revoked_at` (string | null, required): Revocation time. - `last_used_at` (string | null, required): Last authenticated request (accurate to about a minute). - `is_self` (boolean, required): This is the calling key. - `limits` (KeyLimitsInfo | null, required): Limits set by the operator; `null` = none (only the project's). ### KeyLimitsInfo Limits the operator set on a key (`null` fields: no such limit). - `monthly_spend_cap` (string | null, required): Charges per calendar month (UTC) in `currency` (provider cost in USD for projects without a customer). - `currency` (string, required): Currency of the cap. - `monthly_requests` (map | null, required): Requests per month per category (`search`, `fetch`, `fetch_premium`, `company`). - `rate_limit_rps` (integer | null, required): Requests per second. - `allowed_categories` (string[] | null, required): What the key may do (`search`, `fetch`, `fetch_premium`, `company`); `null` = all. Without `fetch_premium`, fetches run without the premium routes. ### QuotaKeyUsage What a key used this month. - `requests` (map, required): Requests per category (`search`, `fetch`, `fetch_premium`, `company`). - `spend` (string, required): Charges (or provider cost without a customer) including the estimate since the last aggregation. - `currency` (string, required): Currency of `spend`. ### QuotaEntry One limit or allowance and how much of it is used. - `name` (string, required): `monthly_spend`, `monthly_requests.`, `rate_limit`, `monthly_provider_budget` or `included.`. - `unit` (string, required): Currency, `requests`, `requests/s` or the meter's unit. - `limit` (integer | string, required): The limit (money as a decimal string). - `used` (integer | string, required): Used this period (or this second). - `reserved` (integer | string, required): Spend estimated since the last aggregation (5 minutes), or provider spend held for calls in flight. - `remaining` (integer | string, required): `limit - used - reserved`, at least 0. - `resets_at` (string, required): When the period (or the one-second window) resets. - `projected_end_of_period` (integer | string | null, required): `used + reserved` extrapolated linearly to the end of the month; `null` for rate limits. - `status` (`ok` | `warning` | `exhausted`, required): `warning` from 80 %, `exhausted` at 100 %. ### UsageScope What the figures cover. - `customer` (string | null, required): Customer name; `null` for projects without a customer. - `projects` (string[], required): Projects covered. - `keys` (`self` | `all`, required): `self`: only the calling key (standard keys); `all`: every key of the account, revoked keys and unattributed usage included (account admin keys). ### UsageMetrics Metrics of a group (or of the totals). - `requests` (integer, required): All requests (cache hits included). - `succeeded` (integer, required): Successful requests. - `failed` (integer, required): `requests - succeeded`. - `success_rate` (number | null, required): `succeeded / requests`. - `cached` (integer, required): Served from the shared cache. - `cache_hit_rate` (number | null, required): `cached / requests`. - `errors` (map, required): Failures by error code (top 50), e.g. `http_404`, `all_tiers_failed`. - `latency_ms_p50` (integer | null, required): Median latency of uncached requests in ms, estimated from the latency histogram (exact to the bucket); `null` without uncached requests. - `latency_ms_p95` (integer | null, required): 95th percentile latency of uncached requests in ms (estimated). - `bytes` (integer, required): Bytes retrieved. - `premium_requests` (integer, required): Fetches served by (or charged for) the premium routes. - `premium_share` (number | null, required): `premium_requests / uncached fetches`: how often the premium routes were needed. - `charge` (string | null): Charges at the customer's plan price in `currency` (decimal string): the group's share of the usage lines of the estimate or invoice. Account-level lines (base fee, minimum, discounts, complimentary credit) are under `account`. Only for projects with a customer. - `billable` (map | null): Per price-plan meter (`search`, `fetch_own`, `fetch_premium`, `premium_gb`, …): quantity, included units used, billable quantity and amount. Only for projects with a customer. - `cost_usd` (number | null): Cormorant's provider spend in USD — only for projects without an external customer (internal use). ### UsageKind Usage of one request kind (search or fetch) in the caller's scope. Empty object when unused. - `requests` (integer | null): All requests. - `cached` (integer | null): Cache hits. - `cache_hit_rate` (number | null): `cached / requests`. - `succeeded` (integer | null): Successful requests. - `success_rate` (number | null): `succeeded / requests`. - `bytes` (integer | null): Bytes retrieved. - `premium_bytes` (integer | null): Bytes through premium routes. - `latency_ms_p50` (integer | null): Median latency of uncached requests (estimated from a latency histogram, see the guide). - `latency_ms_p95` (integer | null): 95th percentile latency of uncached requests (estimated). - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer (internal use). - `fallbacks` (integer | null): Search only: requests answered by the fallback. - `fallback_rate` (number | null): Search only: `fallbacks / uncached requests`. - `providers` (map | null): Search only: breakdown by provider (`searxng`, `brave`). - `tiers` (map | null): Fetch only: breakdown by tier. - `latency_components` (map | null): p50/p95 (ms, estimated) of the latency components of uncached requests: `queue_ms` (Cormorant's queues), `politeness_wait_ms` (polite per-domain slots), `upstream_ms` (the site or provider). Requests from before these were recorded are not included. - `by_outcome` (map | null): The same requests split by outcome: `succeeded` and `failed`, each with requests, bytes, premium bytes and (fetch) premium requests. Shows what failed fetches cost in premium bytes. - `errors` (map | null): Error codes and their counts (top 50). ### Budget The project's monthly budget for third-party providers (calendar month, UTC). - `period` (string, required): Budget month (`YYYY-MM`). - `budget_usd` (number, required): Monthly budget. - `spent_usd` (number, required): Settled spend this month. - `reserved_usd` (number, required): Reserved for provider calls in flight. - `remaining_usd` (number, required): Budget left. ### SpendBudget This month against the spend caps, per key and per customer. - `period` (string, required): Current month. - `currency` (string, required): Currency of the amounts. - `basis` (`charges` | `provider_cost`, required): `charges` at your plan price (projects with a customer), or `provider_cost` in USD (projects without one). - `resets_at` (string, required): Start of the next month. - `keys` (SpendBudgetKey[], required): The calling key (standard key) or every active key (account admin key). - `customer` (SpendBudgetCustomer | null): Account admin keys of a customer: the whole account. `cap` is always `null`: spend caps are set per key. ### UsageGroup One group of `GET /v1/usage?group_by=…`: the dimension values, then the metrics. - `requests` (integer, required): All requests (cache hits included). - `succeeded` (integer, required): Successful requests. - `failed` (integer, required): `requests - succeeded`. - `success_rate` (number | null, required): `succeeded / requests`. - `cached` (integer, required): Served from the shared cache. - `cache_hit_rate` (number | null, required): `cached / requests`. - `errors` (map, required): Failures by error code (top 50), e.g. `http_404`, `all_tiers_failed`. - `latency_ms_p50` (integer | null, required): Median latency of uncached requests in ms, estimated from the latency histogram (exact to the bucket); `null` without uncached requests. - `latency_ms_p95` (integer | null, required): 95th percentile latency of uncached requests in ms (estimated). - `bytes` (integer, required): Bytes retrieved. - `premium_requests` (integer, required): Fetches served by (or charged for) the premium routes. - `premium_share` (number | null, required): `premium_requests / uncached fetches`: how often the premium routes were needed. - `charge` (string | null): Charges at the customer's plan price in `currency` (decimal string): the group's share of the usage lines of the estimate or invoice. Account-level lines (base fee, minimum, discounts, complimentary credit) are under `account`. Only for projects with a customer. - `billable` (map | null): Per price-plan meter (`search`, `fetch_own`, `fetch_premium`, `premium_gb`, …): quantity, included units used, billable quantity and amount. Only for projects with a customer. - `cost_usd` (number | null): Cormorant's provider spend in USD — only for projects without an external customer (internal use). - `key` (string | null): With `group_by=key`: key prefix (`cma_…` account admin, `cmr_…` standard); `null` for usage recorded before per-key attribution. - `key_label` (string | null): With `group_by=key`: the key's label. - `key_role` (`account_admin` | `standard` | null): With `group_by=key`: the key's role. - `key_status` (`active` | `revoked` | null): With `group_by=key`: `revoked` keys stay in the history. - `project` (string | null): With `group_by=project` (or `key`): project name. - `category` (string | null): With `group_by=category`: billing category (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`). - `day` (string | null): With `group_by=day`: UTC day. - `hour` (string | null): With `group_by=hour`: start of the UTC hour. - `final` (boolean | null): With `group_by=day` or `hour`: the day is closed and its figures will not change. ### UsageKey Usage of one API key (`GET /v1/usage?group_by=key`, compatibility list). - `prefix` (string | null, required): First 10 characters of the key (`cma_…` account admin, `cmr_…` standard); `null` for requests recorded before per-key attribution. - `label` (string | null, required): Label set by the operator. - `role` (`account_admin` | `standard` | null, required): `account_admin` (prefix `cma_`) or `standard` (`cmr_`); `null` for requests recorded before per-key attribution. - `project` (string | null, required): Project of the key. - `active` (boolean, required): The key is not revoked. - `requests` (integer, required): All requests. - `cached` (integer, required): Cache hits. - `succeeded` (integer, required): Successful requests. - `search_requests` (integer, required): Searches. - `fetch_requests` (integer, required): Fetches. - `company_requests` (integer, required): Company data requests (`/v1/companies/*`). - `sec_requests` (integer, required): SEC EDGAR requests (`/v1/sec/…` and fetches of SEC hosts). - `company_profile_requests` (integer, required): Extended company profile requests (kind `company_profile`). - `premium_bytes` (integer, required): Bytes through premium routes. - `charge` (string | null): Charges at the customer's plan price (decimal string, `currency`); only for projects with a customer. - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer. ### UsageAccountPeriod One billing period of the range at account level (account admin keys). - `period` (string, required): `YYYY-MM`. - `from` (string, required): First day rated. - `to` (string, required): Last day rated (today while open). - `currency` (string, required): Currency. - `complimentary` (boolean, required): Complimentary account or plan (billed at list price, then discounted 100 %). - `usage_charges` (string, required): Sum of the usage lines: what the keys' `charge` add up to (for the whole period, all keys). - `account_level_amount` (string, required): Sum of `account_lines`. - `account_lines` (AccountLine[], required): Base fee, minimum, discounts, credit. - `list_total` (string, required): Net before the complimentary credit (what the period is worth at list price). - `net_total` (string, required): Net total before VAT. ### UsageSeries One series: the `group_by` values, then one point per bucket. - `key` (string | null): With `group_by=key`: key prefix. - `key_label` (string | null): With `group_by=key`: label. - `key_role` (`account_admin` | `standard` | null): With `group_by=key`: role. - `key_status` (`active` | `revoked` | null): With `group_by=key`: status. - `project` (string | null): With `group_by=project` or `key`. - `category` (string | null): With `group_by=category`. - `points` (UsagePoint[], required): Every bucket of the range up to now, oldest first; zero where unused. ### DomainUsage Fetches of one domain. - `domain` (string, required): Host name. - `requests` (integer, required): Fetches. - `succeeded` (integer, required): Successful fetches. - `success_rate` (number | null, required): `succeeded / requests`. - `premium_requests` (integer, required): Fetches served by the premium routes. - `premium_share` (number | null, required): `premium_requests / requests`. - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer. ### KeyDomains Top domains of one key (`group_by=key`). - `key` (string, required): Key prefix. - `key_label` (string | null, required): Label. - `key_role` (`account_admin` | `standard`, required): Role. - `key_status` (`active` | `revoked`, required): Status. - `project` (string | null, required): Project. - `domains` (DomainUsage[], required): Busiest first. ### LedgerRequest One request of the ledger. - `request_id` (string, required): `req_…`, or `:` for batch items. - `time` (string, required): When it was recorded. - `key` (string | null, required): Prefix of the key that made it; `null` before per-key attribution. - `key_role` (`account_admin` | `standard` | null, required): Role of that key. - `project` (string | null, required): Project. - `project_ref` (string | null, required): Your `project_ref`. - `job_id` (string | null, required): Batch job of the item. - `kind` (`search` | `fetch` | `company` | `sec` | `company_profile`, required): Request kind (`company_profile`: extended company profiles). - `domain` (string | null, required): Fetched host. - `tier` (string | null, required): Fetch: the tier that answered; search: the provider. - `status` (integer | null, required): HTTP status of the origin. - `ok` (boolean, required): Succeeded. - `error` (string | null, required): Error code. - `latency_ms` (integer | null, required): Latency in ms. - `cached` (boolean, required): Served from the shared cache. - `bytes` (integer, required): Bytes retrieved. - `premium` (boolean, required): Served by (or charged for) a premium route. - `category` (string, required): Billing category. - `charge` (string | null): Customers: the request's share of its key's charges for that day and category (their average price; 6 decimals, `currency`). `null` until aggregated (within 5 minutes); settles when the day is final. - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer. ### LedgerAttempt One tier tried. - `tier` (string | null, required): Tier or provider. - `status` (integer | null, required): HTTP status. - `reason` (string | null, required): Outcome. - `ms` (integer | null): Duration. ### QuotaKey One key's limits. - `key` (string, required): Key prefix. - `key_label` (string | null, required): Label. - `key_role` (`account_admin` | `standard`, required): Role. - `project` (string | null, required): Project. - `is_self` (boolean, required): The calling key. - `allowed_categories` (string[] | null, required): What the key may do; `null` = all. - `usage` (QuotaKeyUsage, required): Use this month. - `limits` (QuotaEntry[], required): Each limit set on the key. ### QuotaProject The calling key's project. - `project` (string, required): Project name. - `limits` (QuotaEntry[], required): Requests per second; the provider budget for projects without an external customer. ### QuotaCustomer The customer account (account admin keys). - `customer` (string, required): Customer name. - `currency` (string, required): Invoice currency. - `allowances` (QuotaEntry[], required): The plan's included units per meter this month (`included.`). ### AlertEvent A threshold crossed: in `GET /v1/quota` `events` and the body of webhook deliveries. - `id` (string, required): Event id (`evt_…`); deliveries may repeat it. - `created_at` (string, required): When it fired. - `type` (`usage.threshold`, required): Event type. - `scope` (`key` | `project` | `customer`, required): What crossed the threshold. - `customer` (string | null, required): Customer. - `project` (string | null, required): Project (key and project events). - `key` (string | null, required): Key prefix (key events). - `key_role` (`account_admin` | `standard` | null, required): Role of that key. - `limit` (AlertLimit, required): The limit and its use. ### QuotaAlertEmail Whether usage alert e-mails reach the account's alert address. - `status` (`deliverable` | `partially_undeliverable` | `undeliverable` | `not_set`, required): `undeliverable`: every alert address hard-bounced or reported a mail as spam; `partially_undeliverable`: some did. No further alert mails go to those addresses until they are changed or cleared (contact us). - `addresses` (integer, default `0`): Alert addresses set. - `undeliverable` (integer, default `0`): How many of them are undeliverable. - `reason` (`bounced` | `complained` | null): With `undeliverable`: why. - `since` (string | null): With `undeliverable`: since when. ### AlertHook A registered alert webhook (never its secret). - `id` (integer, required): Webhook id. - `url` (string, required): Delivery URL. - `scope` (`key` | `project` | `customer`, required): What it watches: one key, one project (and its keys) or the whole account. - `key` (string | null, required): With `scope=key`: key prefix. - `project` (string | null, required): With `scope=project`: project name. - `thresholds` (integer[], required): Percentages delivered. - `created_at` (string, required): Registration time. - `last_delivery_at` (string | null, required): Last delivery attempt. - `last_status` (string | null, required): `delivered_`, `http_`, `error_` or `blocked_destination`. ### CompanySummary A search or resolve hit. - `id` (string | null, required): Cormorant id when the company is in the register, else `null`. - `ref` (string, required): What to pass to `GET /v1/companies/{id}`: the `cmp_` id, or `scheme:value` for a live hit not yet in the register. - `in_register` (boolean, required): Whether the company is already in the register. - `match_score` (number, required): 0–1 relevance (name similarity). - `name` (string | null, required): Legal name. - `country` (string | null, required): ISO 3166-1 alpha-2. - `city` (string | null, required): City of the first address. - `status` (string | null, required): Register status. - `legal_form` (string | null, required): Legal-form category. - `identifiers` (object[], required): `{scheme, value}` identifiers. - `source` (string | null, required): Source of a live hit (`null` for register hits). - `reasons` (object | null): `POST /v1/companies/resolve`: what the score is based on (`name`, `country`, `city`, `identifier`). ### LiveSource A source searched live for this request. - `source` (string, required): Source name. - `ok` (boolean, required): Whether it answered. - `error` (string | null, required): Error class when it did not (`timeout`, `http_503`, `backoff`, `canary_failed`, …). - `status` (`ok` | `failed` | `timeout` | `skipped` | null): `ok`; `failed` (source error); `timeout` (slower than the answer's deadline — it finishes in the background); `skipped` (not asked: backing off after repeated failures, or its canaries fail). - `ms` (integer | null): Time the source took (ms). ### CompanyChange One change of the register. - `seq` (integer, required): Position in the feed (use as cursor). - `company_id` (string, required): Cormorant id. - `kind` (`created` | `updated` | `merged` | `split`, required): `merged`: the record was merged into another (follow the id: it redirects); `split`: a merge was undone. - `fields` (string[], required): Top-level fields that changed. - `country` (string | null, required): Country of the company. - `changed_at` (string, required): When (ISO 8601). ### CompanySource A company data source. - `name` (string, required): Source name (as in `sources[]` and `provenance`). - `label` (string, required): Display name. - `description` (string, required): What it provides. - `countries` (string[], required): ISO 3166-1 alpha-2 (`*` = global). - `schemes` (string[], required): Identifier schemes it can be asked by. - `capabilities` (string[], required): `search`, `get`, `people`, `filings`, `events`, `relationships`, `ownership`, `financials`, `changes`. - `source_class` (string, required): Precedence class: `official_register` > `official_gazette` > `gleif` > `regulatory` > `crosswalk` > `commercial` > `other`. - `precedence` (integer, required): Numeric precedence (higher wins conflicts). - `licence_url` (string, required): Licence/terms of the source's data. - `freshness` (map, required): Freshness class per field group in seconds: older data is re-fetched when requested. - `active` (boolean, required): Enabled and configured by the operator. - `health` (CompanySourceHealth, required): Live health. ### app__companies__model__Identifier An identifier of the company in some scheme (canonical form). - `scheme` (string, required): Identifier scheme (`ch_uid`, `lei`, `us_cik`, …). - `value` (string, required): Canonical value. - `issuer` (string | null): Body that issues identifiers of this scheme. - `status` (string, default `active`): `active` or `inactive` (retired, lapsed). - `parts` (object | null): Structured parts (`de_register`: court, type, number; `ticker`: mic, symbol). - `sources` (string[]): Sources that reported it. ### Names Legal name and other names. - `legal` (string | null): Current legal name: the official national register's primary entry in its official form and language when a register source has the company (in Switzerland Zefix, then the UID register), else GLEIF's, else other sources'. Never depends on the order of refreshes. - `previous` (Name[]): Former legal names. - `trading` (Name[]): Trading names, brands, `doing business as`. - `translations` (Name[]): The legal name in other languages (e.g. Swiss multilingual names). - `alternates` (Name[]): Other sources' current forms of the legal name that differ from `legal` (e.g. GLEIF's `NESTLÉ S.A.` next to the commercial register's `Nestlé AG`), best-ranked source first, each with its `source`. ### LegalForm Legal form, mapped to ISO 20275 (ELF) and a category. - `local_code` (string | null): Local abbreviation. - `local_name` (string | null): Local name. - `elf_code` (string | null): ISO 20275 Entity Legal Form code. - `category` (`corporation` | `partnership` | `sole_trader` | `cooperative` | `foundation` | `association` | `public_body` | `branch` | `other`, default `other`): Normalised category. ### app__companies__model__Status Register status. - `value` (`active` | `inactive` | `in_liquidation` | `dissolved` | `merged` | `struck_off` | `unknown`, required): Normalised status. - `local_value` (string | null): Status as the source words it. - `since` (string | null): Date the status took effect. - `reason` (string | null): Reason, e.g. merger or bankruptcy. ### Jurisdiction Legal jurisdiction. - `country` (string | null): ISO 3166-1 alpha-2. - `subdivision` (string | null): ISO 3166-2 (US state, Swiss canton …). - `register` (Register | null): Register entry. ### Address A postal address of the company. - `type` (`registered` | `business` | `postal` | `branch` | `other`, required): Kind of address. - `lines` (string[]): Address lines as published. - `street` (string | null): Street. - `house_number` (string | null): House number. - `postal_code` (string | null): Postal code. - `city` (string | null): City or locality. - `municipality_code` (string | null): Official municipality code (e.g. Swiss BFS number). - `region` (string | null): Region (ISO 3166-2 when known). - `country` (string | null): ISO 3166-1 alpha-2. - `geo` (Geo | null): Coordinates, if the source gives them. - `valid_from` (string | null): Start of validity (date). - `valid_to` (string | null): End of validity (date). ### Dates Key dates. - `incorporated` (string | null): Founding/incorporation date. - `registered` (string | null): Entry into the register. - `deregistered` (string | null): Deletion from the register. - `fiscal_year_end` (string | null): Fiscal year end (`MM-DD`). ### Activity An economic activity classification. - `scheme` (`nace` | `noga` | `wz` | `sic` | `naics` | `isic`, required): Classification. - `code` (string, required): Code as published. - `label` (string | null): Label of the code (English). - `primary` (boolean, default `false`): Main activity. - `nace` (string | null): NACE Rev. 2 class of a NOGA/WZ code. ### Capital Share capital. - `type` (`nominal` | `paid_in` | `authorised`, required): Nominal, paid-in or authorised capital. - `amount` (number, required): Amount. - `currency` (string, required): ISO 4217. - `since` (string | null): Date of the amount. ### Size Size, only from official or filing data. - `employees_range` (string | null): Employee range, e.g. `50-249`. - `employees` (integer | null): Employee count, when filed. - `revenue` (number | null): Revenue, when filed. - `currency` (string | null): Currency of the revenue. - `as_of` (string | null): Reference date. - `source` (string | null): Source of the figures. ### Person An officer, signatory or other function holder as published by the register. - `name` (string, required): Name as published. - `role` (string, required): Normalised role: `director`, `chair`, `ceo`, `managing_director`, `secretary`, `auditor`, `liquidator`, `partner`, `signatory`, `member`, `other`. - `local_role` (string | null): Role as worded by the source. - `function` (string | null): Further function text. - `signing_authority` (string | null): `individual`, `joint_two`, `none`, or the local wording. - `since` (string | null): Start (date). - `until` (string | null): End (date). - `residence_city` (string | null): Place of residence, as published. - `residence_country` (string | null): ISO 3166-1 alpha-2. - `nationality` (string | null): Nationality, as published. - `origin` (string | null): Place of origin (Swiss registers). - `company` (CompanyRef | null): When the function holder is a company. - `source` (string, required): Source of the entry. - `person_id` (string, required): Stable key of this person within Cormorant (`prs_…`). ### Ownership An ownership link. - `type` (`shareholder` | `ultimate_parent` | `direct_parent` | `beneficial_owner`, required): Kind of ownership. - `holder` (CompanyRef | null): Holder, when a company. - `holder_person` (string | null): Holder's name, when a natural person. - `share_pct` (number | null): Share in percent. - `since` (string | null): Start (date). - `source` (string | null): Source of the link. ### Relationship A link to another company. - `type` (`subsidiary` | `branch_of` | `branch` | `head_office` | `merged_into` | `absorbed` | `predecessor` | `successor` | `auditor` | `fund_manager` | `other`, required): Kind of link. - `company` (CompanyRef, required): The other company. - `since` (string | null): Start (date). - `until` (string | null): End (date). ### Filing A filed document. - `type` (string, required): Normalised type (`annual_report`, `financial_statements`, `10-K`, …). - `date` (string | null): Filing date. - `period` (string | null): Reporting period. - `title` (string | null): Title. - `url` (string | null): Where the source publishes it. - `document_ref` (DocumentRef | null): Stored copy, if retrieved. ### Financials Figures of one period, only when published by an official source/XBRL. - `period` (string, required): Period, e.g. `2025` or `2025-12-31`. - `currency` (string | null): ISO 4217. - `revenue` (number | null): Revenue. - `net_income` (number | null): Net income. - `total_assets` (number | null): Total assets. - `equity` (number | null): Equity. - `employees` (integer | null): Employees. - `source_filing` (string | null): Filing the figures come from. ### Event A dated event of the company's history (from gazettes and registers). - `type` (string, required): `incorporation`, `name_change`, `seat_change`, `purpose_change`, `capital_change`, `officer_change`, `merger`, `liquidation`, `deletion`, `registration`, `other`. - `date` (string | null): Date of the event or publication. - `title` (string | null): Short description. - `url` (string | null): Publication link. - `source` (string, required): Source of the event. - `data` (object): Source-specific details (publication id …). ### Security A listed security of the company. - `isin` (string | null): ISIN. - `ticker` (string | null): Ticker symbol. - `mic` (string | null): ISO 10383 market identifier. - `listing_status` (string | null): `listed` or `delisted`. ### Web Web presence (official sources or verified). - `website` (string | null): Main website. - `domains` (string[]): Domains of the company. ### SourceDocument A raw document a source served about the company. - `source` (string, required): Source name. - `url` (string | null): Source URL. - `retrieved_at` (string, required): Retrieval time (ISO 8601). - `sha256` (string | null): SHA-256 of the exact bytes. - `body_url` (string | null): Download link of the exact bytes. - `content_type` (string | null): Content type. - `licence` (string | null): Licence/terms URL of the source. - `parser_version` (string | null): Parser version that read it. ### SourceFreshness Freshness of one source's data about the company. - `source` (string, required): Source name. - `group` (string, required): Field group (`core`, `people`, `relationships`, `filings`, `events`, …). - `last_checked` (string | null): Last fetch from the source. - `last_changed` (string | null): Last time the source's data changed. - `next_refresh_due` (string | null): When it counts as stale. - `error` (string | null): Error of the last attempt, if it failed. ### Quality Completeness and conflicts. - `completeness` (number, required): 0–1 share of core fields filled. - `conflicts` (Conflict[]): Fields where sources disagree. ### Provenance Which claim the resolved value of a field comes from. - `source` (string, required): Source of the chosen claim. - `observed_at` (string, required): When the value was first observed. - `confirmed_at` (string | null): Last confirmation by the source. - `sha256` (string | null): Raw document of the claim, if any. - `conflict` (boolean, default `false`): True when another source currently claims a different value. ### Claim One source's statement about one field, with its validity. - `field` (string, required): Field the claim is about (`names.legal`, `addresses`, …). - `key` (string | null): Item key within a list field (e.g. the address type). - `value` (any, required): The claimed value (shape of the field or list item). - `source` (string, required): Source making the claim. - `observed_at` (string, required): First time Cormorant saw this value (ISO 8601). - `confirmed_at` (string | null): Last time the source still said so (ISO 8601). - `valid_from` (string | null): Start of validity the source states (date). - `valid_to` (string | null): End of validity: stated by the source, or the day a newer observation replaced it. - `current` (boolean, required): True while the source still asserts this value. - `confidence` (number, required): 0–1; parsing and matching certainty. - `document_ref` (DocumentRef | null): Raw document behind the claim. ### CompanyRefresh What one source did for this request (refresh-on-request). - `source` (string, required): Source name. - `group` (string, required): Field group: `core`, `people`, `ownership`, `relationships`, `filings`, `financials`, `events`. - `status` (`refreshed` | `fresh` | `failed` | `not_found` | `in_progress` | `pending` | `skipped`, required): `refreshed` (fetched now), `fresh` (register data within the source's freshness class or your `max_age`), `failed` (source error; the register still answers), `not_found` (the source does not know the company), `in_progress` (another request is refreshing it right now), `pending` (too slow for this answer or the source's canaries fail: it finishes in the background, the next request sees the data; `error` `deadline` or `canary_failed`), `skipped` (the source is backing off after repeated failures; `error` `backoff`). - `error` (string | null): Error class when `failed` (`http_503`, `timeout`, …); `cached` with `not_found` when the source said so recently and was not asked again. - `at` (string | null): When it was refreshed (ISO 8601). ### BillingUnit One meter of your plan that this request counts toward. - `meter` (string, required): Plan meter (`search`, `fetch_own`, `fetch_premium`, `premium_gb`, `delivered_gb`, `sec`, `company_profile`, `company`, …). - `unit` (string | null, required): Unit of the meter (`search`, `fetch`, `request`, `GB`); `null` for pass-through meters. - `quantity` (string | null, required): Units this request adds (decimal string); `null` for pass-through meters. - `included` (string | null, required): Of those, units inside the plan's included allowance (charged 0). - `charge` (string, required): Charge for these units (decimal string, 6 places, plan currency). ### SecHintCompany The company as Cormorant's company register knows it. - `id` (string, required): Register id. - `name` (string | null, required): Legal name. - `other_names` (string[]): Other forms of the name (other sources, languages). - `country` (string | null, required): ISO 3166-1 alpha-2. - `listings` (string[]): Listings as `MIC:SYMBOL`. - `lei` (string | null): LEI. - `isins` (string[]): ISINs. ### EstimateLine One priced line of an estimate. - `kind` (string, required): `usage`, `pass_through`, `base_fee`, `minimum`, `discount` or `complimentary`. - `component` (string | null, required): Meter the line prices, e.g. `fetch_own`. - `description` (string, required): Human-readable line. - `quantity` (string | null, required): Quantity (decimal string). - `unit` (string | null, required): Unit, e.g. `fetch`, `GB`. - `unit_price` (string | null, required): Unit price when flat; `null` for tiered lines. - `amount` (string, required): Decimal as a string, in the invoice currency. - `detail` (object | null, required): Breakdown: `included`, `billable` and the volume `tiers` applied. ### StatusUpdate One entry of an incident's or maintenance's timeline. - `at` (string, required): When it was posted (ISO 8601, UTC). - `status` (string, required): Status set by this update. - `message` (string, required): What the operator said. ### MeterUsage One price-plan meter of a group: how much counted, how much of it was covered by the plan's included units, and what it costs. - `quantity` (string | null, required): Measured quantity (decimal string: requests or GB); `null` for meters billed on provider cost. - `included` (string | null, required): Included units used by this group: its pro-rata share of the account's included units of the period. - `billable` (string | null, required): `quantity - included`. - `amount` (string, required): Charge of this meter for the group, at the plan price in the customer's currency (decimal string). ### UsageBreakdown Requests of one provider (search) or tier (fetch); cache hits excluded. - `requests` (integer, required): Uncached requests. - `succeeded` (integer, required): Successful requests. - `premium_bytes` (integer, required): Bytes transferred through premium routes. - `cost_usd` (number | null): Cormorant's provider spend in USD — only for projects without an external customer (internal use); customers see `charge` instead. - `latency_ms_p50` (integer | null): Median latency (estimated, see the guide). - `latency_ms_p95` (integer | null): 95th percentile latency (estimated). - `queue_ms_p50` (integer | null): Median queue time. - `queue_ms_p95` (integer | null): 95th percentile queue time. - `politeness_wait_ms_p50` (integer | null): Median wait for polite per-domain slots. - `politeness_wait_ms_p95` (integer | null): 95th percentile wait for polite per-domain slots. - `upstream_ms_p50` (integer | null): Median time in tier attempts (site / provider). - `upstream_ms_p95` (integer | null): 95th percentile time in tier attempts. ### Percentiles - `p50` (integer | null, required): Median (ms, estimated). - `p95` (integer | null, required): 95th percentile (ms, estimated). ### UsageOutcome Requests of one kind that ended with this outcome (cache hits included). - `requests` (integer, required): Requests. - `bytes` (integer, required): Bytes retrieved. - `premium_bytes` (integer, required): Bytes billed by the premium routes (residential proxy) for these requests. On `failed`: what blocked pages cost before the fetch gave up. - `premium_requests` (integer | null): Fetch only: requests that used a premium route (billing category `fetch_premium`). - `cost_usd` (number | null): Provider spend in USD — only for projects without an external customer (internal use). ### SpendBudgetKey - `cap` (string | null, required): Monthly spend cap (set by the operator per key); `null`: no cap is set. - `spent` (string, required): Spent this month: charges at your plan price as of the last refresh plus the estimate since (`pending`). Provider cost in USD for projects without a customer. - `pending` (string, required): Per-request estimate since the last refresh (part of `spent`). - `remaining` (string | null, required): `cap - spent` (never below 0); `null` without a cap. - `status` (`ok` | `warning` | `exhausted` | `no_cap`, required): `warning` from 80 % of the cap, `exhausted` at the cap (requests of the key answer `429 key_budget_exhausted`), `no_cap` when none is set. - `by_kind` (SpendByKind, required): `spent` by request kind (without `pending`). - `key` (string, required): Key prefix. - `key_label` (string | null, required): Key label. - `project` (string | null, required): Project of the key. - `is_self` (boolean, required): The calling key. - `currency` (string, required): Currency of the amounts. ### SpendBudgetCustomer - `cap` (string | null, required): Monthly spend cap (set by the operator per key); `null`: no cap is set. - `spent` (string, required): Spent this month: charges at your plan price as of the last refresh plus the estimate since (`pending`). Provider cost in USD for projects without a customer. - `pending` (string, required): Per-request estimate since the last refresh (part of `spent`). - `remaining` (string | null, required): `cap - spent` (never below 0); `null` without a cap. - `status` (`ok` | `warning` | `exhausted` | `no_cap`, required): `warning` from 80 % of the cap, `exhausted` at the cap (requests of the key answer `429 key_budget_exhausted`), `no_cap` when none is set. - `by_kind` (SpendByKind, required): `spent` by request kind (without `pending`). - `customer` (string, required): Customer name. - `currency` (string, required): Invoice currency. - `account_charges` (string, required): Account-level lines this month: base fee, minimum top-up, contract discount, complimentary credit (not part of `spent`). - `net_estimate` (string, required): The month's estimate before VAT (`spent` without `pending`, plus `account_charges`). ### AccountLine An account-level line of a period (not attributed to keys). - `kind` (string, required): `base_fee`, `minimum`, `discount` or `complimentary`. - `description` (string, required): Line text. - `amount` (string, required): Decimal string in `currency`. ### UsagePoint One bucket of a time series: its start, whether it is final, the metrics. - `requests` (integer, required): All requests (cache hits included). - `succeeded` (integer, required): Successful requests. - `failed` (integer, required): `requests - succeeded`. - `success_rate` (number | null, required): `succeeded / requests`. - `cached` (integer, required): Served from the shared cache. - `cache_hit_rate` (number | null, required): `cached / requests`. - `errors` (map, required): Failures by error code (top 50), e.g. `http_404`, `all_tiers_failed`. - `latency_ms_p50` (integer | null, required): Median latency of uncached requests in ms, estimated from the latency histogram (exact to the bucket); `null` without uncached requests. - `latency_ms_p95` (integer | null, required): 95th percentile latency of uncached requests in ms (estimated). - `bytes` (integer, required): Bytes retrieved. - `premium_requests` (integer, required): Fetches served by (or charged for) the premium routes. - `premium_share` (number | null, required): `premium_requests / uncached fetches`: how often the premium routes were needed. - `charge` (string | null): Charges at the customer's plan price in `currency` (decimal string): the group's share of the usage lines of the estimate or invoice. Account-level lines (base fee, minimum, discounts, complimentary credit) are under `account`. Only for projects with a customer. - `billable` (map | null): Per price-plan meter (`search`, `fetch_own`, `fetch_premium`, `premium_gb`, …): quantity, included units used, billable quantity and amount. Only for projects with a customer. - `cost_usd` (number | null): Cormorant's provider spend in USD — only for projects without an external customer (internal use). - `t` (string, required): Start of the hour or day (UTC). - `final` (boolean, required): The bucket's day is closed. ### AlertLimit The limit an alert is about. - `name` (string, required): `monthly_spend`, `monthly_requests.`, `monthly_provider_budget` or `included.`. - `unit` (string, required): Currency, `requests` or the meter's unit. - `limit` (integer | string, required): The limit (money as a decimal string). - `used` (integer | string, required): Used when the event fired. - `threshold` (integer, required): Percent crossed: 50, 80 or 100. - `period` (string, required): Month (`YYYY-MM`, UTC). - `resets_at` (string, required): When the limit resets. ### CompanySourceHealth Live health of a company data source. - `status` (string, required): Status-page status of the source. - `success_rate` (number | null, required): Last hour; `null` with too few calls. - `latency_p95_ms` (integer | null, required): Recent p95 latency. - `canary_ok` (boolean | null, required): Last canary result (known companies). - `last_success_at` (string | null, required): Last successful call (ISO 8601). ### Name A name with its validity and language. - `name` (string, required): The name as registered. - `lang` (string | null): ISO 639-1 language, when the source says. - `valid_from` (string | null): Start of use (date). - `valid_to` (string | null): End of use (date). - `source` (string | null): Source that published this form (`translations`, `alternates`). ### Register Register the company is entered in. - `authority` (string | null): Register authority (name or GLEIF RA code). - `court` (string | null): Register court (Germany) or cantonal office. - `entity_id` (string | null): The company's id in that register. ### Geo - `lat` (number, required): Latitude (WGS 84). - `lon` (number, required): Longitude (WGS 84). ### CompanyRef Another company, by Cormorant id and/or identifiers. - `id` (string | null): Cormorant id, when the company is in the register. - `name` (string | null): Name as the source gives it. - `identifiers` (app__companies__model__Identifier[]): Identifiers given by the source. - `country` (string | null): ISO 3166-1 alpha-2. ### DocumentRef The raw document a claim was read from. - `source` (string, required): Source that served the document. - `url` (string | null): Where the source published it. - `retrieved_at` (string | null): When Cormorant retrieved it (ISO 8601). - `sha256` (string | null): SHA-256 of the exact bytes. - `body_url` (string | null): Download link of the exact bytes (`GET /v1/bodies/{sha256}`). ### Conflict A field where sources currently disagree. - `field` (string, required): Field. - `values` (object[], required): `{source, value}` per source. ### SpendByKind `spent` (aggregated part) by request kind. - `search` (string, required): Decimal string in `currency`. - `fetch` (string, required): Decimal string in `currency`. - `sec` (string, required): Decimal string in `currency`. - `company_profile` (string, required): Decimal string in `currency`. Extended company profiles. - `company` (string, required): Decimal string in `currency`. # Overview Cormorant is a web **search** and **fetch** API shared by research projects and AI agents. You send a query or a URL; Cormorant does the hard parts — engines that rate-limit, sites that block bots, PDFs, caching, politeness, cost control — and answers with clean text, metadata and provenance. ## What you can do | You want to… | Use | Typical latency | |---|---|---| | Find pages about a topic | `POST /v1/search` | 1–5 s (cache hit: ms) | | Read one page, PDF or document | `POST /v1/fetch` | 0.5–15 s (cache hit: ms) | | Search or fetch hundreds to 50,000 items | `POST /v1/batch/search`, `POST /v1/batch/fetch` | minutes, asynchronous | | Get the exact bytes behind a fetch | `GET /v1/bodies/{sha256}` | streaming | | See your plan, this month's estimate and invoices | `GET /v1/account`, `GET /v1/account/invoices` | ms | | See requests, cache hits, spend and budget | `GET /v1/usage` | ms | ## Base URL and authentication ```text https://api.cormorant.ch ``` Every `/v1` request carries your project's key: ```text Authorization: Bearer cmr_… ``` Keys are issued by the operator, one project per key. See [Authentication & keys](/docs/authentication.md). ## Concepts in one minute - **Search** asks Cormorant's own SearXNG engines first and falls back to the Brave Search API only when they fail or find too little. Results from all engines are merged; no snippet is lost. - **Fetch** climbs a ladder of **tiers**: the standard routes `direct` (HTTP with a browser fingerprint), `browser` (real Chromium) and `archive` (Wayback Machine), then the premium routes `proxy` (residential network) and `proxy_render` (rendering service), which are priced higher — and escalates only when a site blocks. A page that is simply short never reaches a premium route. - **The cache is shared** across projects: a page or search someone fetched recently is answered at once, billed as a cache hit. - **Text first, bytes on demand.** `text` is the extracted main text; the exact retrieved bytes are one request away at `body_url`. - **Every response says what it cost** (`cost_usd`) and carries a `request_id` you can quote to the operator. - **Fetch problems are results, not HTTP errors.** `/v1/fetch` answers `200` with `error` set (for example `http_404` or `robots_disallowed`). HTTP error statuses mean something is wrong with the request itself (key, limits, validation, timeouts). ## Versioning The API is `v1`. Within v1, fields are only ever **added** — nothing is renamed or removed — so ignore fields you do not know. Changes are listed in the [changelog](/docs/changelog.md). ## Where next - [Quickstart](/docs/quickstart.md): a first search and fetch in two minutes. - [Sync vs batch](/docs/sync-vs-batch.md): pick the right endpoint for your volume. - [Using Cormorant from AI agents](/docs/agents.md) and the [MCP server](/docs/mcp.md). - The [interactive API reference](/docs/reference) and the [API contract](/docs/api.md). # Quickstart From zero to a first search and fetch in two minutes. ## 1. Get a key Ask the Cormorant operator for a project API key (`cmr_…`). Keep it out of source control and pass it through an environment variable: ```sh export CORMORANT_API_KEY=cmr_... ``` ## 2. Search ```sh curl -sS https://api.cormorant.ch/v1/search \ -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query": "Holcim annual report 2025", "max_results": 5}' ``` ```python import os import httpx client = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, timeout=180, # sync calls can take up to 150 s ) found = client.post("/v1/search", json={"query": "Holcim annual report 2025", "max_results": 5}) found.raise_for_status() for hit in found.json()["results"]: print(hit["rank"], hit["title"], hit["url"]) ``` ```ts const BASE = "https://api.cormorant.ch"; const headers = { Authorization: `Bearer ${process.env.CORMORANT_API_KEY}`, "Content-Type": "application/json", }; const res = await fetch(`${BASE}/v1/search`, { method: "POST", headers, body: JSON.stringify({ query: "Holcim annual report 2025", max_results: 5 }), }); if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`); const { results } = await res.json(); for (const hit of results) console.log(hit.rank, hit.title, hit.url); ``` Each result has `url`, `title`, every distinct `snippets` entry and the `engines` that found it. The response also tells you the `provider` (`searxng` or `brave`), whether it was `cached`, and its `cost_usd`. ## 3. Fetch a page ```sh curl -sS https://api.cormorant.ch/v1/fetch \ -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://www.holcim.com/investors", "include_html": false}' ``` ```python page = client.post( "/v1/fetch", json={"url": "https://www.holcim.com/investors", "include_html": False} ).json() if page["error"] is None: print(page["title"], page["tier"], len(page["text"]), "characters") else: print("could not fetch:", page["error"], page["request_id"]) ``` ```ts const page = await ( await fetch(`${BASE}/v1/fetch`, { method: "POST", headers, body: JSON.stringify({ url: "https://www.holcim.com/investors", include_html: false }), }) ).json(); if (page.error === null) console.log(page.title, page.tier, page.text.length); else console.log("could not fetch:", page.error, page.request_id); ``` Always check `error`: `/v1/fetch` answers HTTP `200` for unsuccessful fetches too (a `404` on the site is `error: "http_404"`). On success you get: - `text` — the main text, ready for an LLM or an index; - `title`, `content_kind` (`html`, `pdf`, `json`, …), `final_url`, `status`; - `tier` — which tier got it, and `attempts` — everything that was tried; - `body_url` — the exact bytes, if you need them later. ## 4. Many URLs? Use a batch ```sh curl -sS https://api.cormorant.ch/v1/batch/fetch \ -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"idempotency_key": "quickstart-1", "items": [{"url": "https://example.com"}, {"url": "https://example.org"}]}' # → {"job_id": "…", "created": true} curl -sS "https://api.cormorant.ch/v1/jobs/$JOB_ID?include=items" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` See [Sync vs batch](/docs/sync-vs-batch.md) for polling, paging and callbacks. ## Next steps - [Fetching in depth](/docs/fetching.md) — tiers, cost controls, PDFs, exact bytes. - [Errors & retries](/docs/errors.md) — what to retry and how. - [Using Cormorant from AI agents](/docs/agents.md). # Authentication & keys ## Project keys Cormorant authenticates **projects**, not people. Each API key (`cmr_…` or `cma_…`) belongs to exactly one project; the project owns its usage, its monthly budget for third-party providers, its rate limit and its options (such as allowed User-Agents). Keys are issued and revoked by the operator; ask for one key per application or environment so you can rotate them independently. Cormorant stores only an HMAC of each key. A lost key cannot be recovered — ask for a new one and have the old one revoked. ## Account admin key and additional keys Every key has a **role**, fixed when it is issued and visible in its prefix: | Key | Prefix | What it can see | |---|---|---| | **Account admin key** | `cma_…` | the whole customer account, where an endpoint offers it: all keys, projects, usage and invoices | | **Additional key** | `cmr_…` | its own usage only | Keep the account admin key with whoever manages the account (billing, key overview) and give applications additional keys. To change a key's role, ask for a new key with the other role and have the old one revoked. Keys issued before roles existed are additional (`cmr_…`) keys. Every authenticated response tells you which kind of key you used: ```text X-Cormorant-Key-Role: account_admin ``` ```sh curl -sSI https://api.cormorant.ch/v1/usage \ -H "Authorization: Bearer $CORMORANT_API_KEY" | grep -i x-cormorant-key-role ``` ## Sending the key Every `/v1` request needs the key as a Bearer token: ```text Authorization: Bearer cmr_... ``` ```sh curl -sS https://api.cormorant.ch/v1/usage \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` ```python import os import httpx client = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, timeout=180, ) print(client.get("/v1/usage").json()["project"]) ``` ```ts const res = await fetch("https://api.cormorant.ch/v1/usage", { headers: { Authorization: `Bearer ${process.env.CORMORANT_API_KEY}` }, }); console.log((await res.json()).project); ``` `/health` and `/ready` need no key. Everything under `/v1` does, including `GET /v1/bodies/{sha256}` — a `body_url` is not a public link. ## When authentication fails | Status | Body | Meaning | |---|---|---| | `401` | `{"detail": "missing bearer token"}` | No `Authorization: Bearer …` header. | | `401` | `{"detail": "invalid API key"}` | Unknown or revoked key, or a disabled project. | | `403` | `{"detail": {"error": "customer_suspended", "message": "…"}}` | The customer account that owns the project is suspended. | | `403` | `{"detail": {"error": "user_agent_not_allowed", "hint": "…"}}` | The request used a `user_agent` that is not on the project's allowlist. | `401` responses carry `WWW-Authenticate: Bearer`. Do not retry them: fix the key. ## Rate limits Each project has a request-rate limit (by default 200 requests per second, in a fixed one-second window, shared by all of the project's keys). Excess requests get `429 {"detail": "rate limit exceeded"}` with `Retry-After: 1`. A batch call counts as **one** request however many items it carries — another reason to use batches for bulk work. See [Limits & politeness](/docs/limits.md). ## Request ids Every response carries a `request_id` (`req_…`; batch items use `:`). Log it. When something looks wrong, give the operator the request id: the full attempt history of every request is kept. ## Keeping keys safe - Read the key from the environment or a secret manager; never commit it, never embed it in browser or mobile code, never put it in URLs. - Use separate keys (projects) for production, staging and experiments so budgets and usage stay separate. - For AI agents, give the agent a dedicated project with a modest budget, and pass the key to the [MCP server](/docs/mcp.md) through its environment — not through the prompt. # Sync vs batch Cormorant offers the same work two ways: **synchronous** calls that wait for the answer, and **batch jobs** that queue many items and let you collect the results later. Both run exactly the same search and fetch logic and return the same result objects. ## Which one? | | Sync: `POST /v1/search`, `POST /v1/fetch` | Batch: `POST /v1/batch/search`, `POST /v1/batch/fetch` | |---|---|---| | Items per call | 1 | 1 – 50,000 | | You get | the result, in the response | a `job_id` at once; results via `GET /v1/jobs/{job_id}` | | Waits | up to 150 s (or `timeout_s` + 15 s), then `504` | nothing — poll or take a callback | | Rate limit | 1 request per item | 1 request per job | | HTML in results | yes, unless `include_html: false` | no, unless an item sets `include_html: true` | | Retries | yours (see [Errors & retries](/docs/errors.md)) | Cormorant retries transient failures with backoff | | Politeness | waits up to 15 s for a slot on a busy domain, else `503 domain_busy` with `Retry-After` | items for a busy domain wait in the queue while other domains proceed | | Best for | interactive use, agents answering a question, a handful of URLs | crawls, enrichment runs, anything over ~20 items | Rule of thumb: **more than about 20 items, or anything that does not need an answer right now, goes into a batch.** Synchronous calls hold a connection open for the whole wait; a batch of 5,000 fetches is one HTTP request and drains at the fastest polite rate for each domain. ## Synchronous calls A sync call returns when the result is ready. Cache hits return in milliseconds. When the result is not ready in time you get ```json {"detail": {"error": "timeout", "request_id": "req_6b1f2c0e9a7d4b5c8e3f1a2b3c4d5e6f"}} ``` with status `504`. The work usually still completes in the background and lands in the shared cache, so retrying the same request a little later is often instant. Set `timeout_s` on fetches to bound the wait yourself. ## Batch jobs ### 1. Create the job ```sh curl -sS https://api.cormorant.ch/v1/batch/fetch \ -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "idempotency_key": "chipiron-run-2026-10-20-fetch-0001", "callback_url": "https://chipiron.example/hooks/cormorant", "items": [ {"url": "https://www.holcim.com/investors"}, {"url": "https://www.example.com/annual-report-2025.pdf", "allow_premium": false} ] }' ``` ```python import os import httpx client = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, timeout=60, ) urls = ["https://www.holcim.com/investors", "https://www.example.com/annual-report-2025.pdf"] created = client.post( "/v1/batch/fetch", json={ "idempotency_key": "chipiron-run-2026-10-20-fetch-0001", "items": [{"url": url} for url in urls], }, ) created.raise_for_status() # 202 new job, 200 existing job with the same key job_id = created.json()["job_id"] ``` ```ts const BASE = "https://api.cormorant.ch"; const headers = { Authorization: `Bearer ${process.env.CORMORANT_API_KEY}`, "Content-Type": "application/json", }; const urls = ["https://www.holcim.com/investors", "https://www.example.com/annual-report-2025.pdf"]; const created = await fetch(`${BASE}/v1/batch/fetch`, { method: "POST", headers, body: JSON.stringify({ idempotency_key: "chipiron-run-2026-10-20-fetch-0001", items: urls.map((url) => ({ url })), }), }); if (!created.ok) throw new Error(`${created.status}: ${await created.text()}`); const { job_id } = await created.json(); ``` Each item takes the same fields as the single endpoint (`POST /v1/search` or `POST /v1/fetch`), so you can mix options per item. Limits: 50,000 items per job (`413` above that) and a request body of at most 64 MB — split larger runs into several jobs. ### 2. Wait: poll or callback Poll `GET /v1/jobs/{job_id}` with a gentle interval (every 5–30 s, longer for big jobs). The job moves `queued → running → completed` (or `cancelled`), and `counts` tells you how far it is: ```json { "job_id": "1d0c5b8e-6f7a-4c1b-9e2d-3a4b5c6d7e8f", "kind": "fetch", "status": "running", "counts": {"total": 3000, "pending": 1200, "queued": 380, "done": 1390, "failed": 30, "cancelled": 0}, "created_at": "2026-10-19T08:00:00+00:00", "updated_at": "2026-10-19T08:05:12+00:00", "finished_at": null, "callback_url": null, "callback_status": null, "items_url": "/v1/jobs/1d0c5b8e-6f7a-4c1b-9e2d-3a4b5c6d7e8f?include=items&offset=0&limit=100" } ``` Item states: `pending` (waiting for admission), `queued` (admitted, running soon or now), `done`, `failed` (finished with an `error`; the result is kept), `cancelled`. Or pass `callback_url` when creating the job. When the job finishes, Cormorant sends one `POST` to it: ```json {"job_id": "1d0c5b8e-6f7a-4c1b-9e2d-3a4b5c6d7e8f", "kind": "fetch", "status": "completed", "total": 3000, "succeeded": 2970, "failed": 30, "finished_at": "2026-10-19T09:41:02+00:00"} ``` Any `2xx` answer counts as delivered; Cormorant tries 3 times and records the outcome in the job's `callback_status`. The callback is **not signed**: treat it as a hint, then read the job through the API with your key. Keep polling (slowly) as a fallback in case your endpoint was down. Callback URLs must be public http(s) addresses. ### 3. Page through the results Add `include=items` and page with `offset`/`limit` (up to 1,000 per page). Filter with `status=failed` to look at failures only. ```python import time def wait(job_id: str) -> dict: delay = 5.0 while True: job = client.get(f"/v1/jobs/{job_id}").json() if job["status"] in ("completed", "cancelled"): return job time.sleep(delay) delay = min(delay * 1.5, 60) def items(job_id: str, status: str | None = None): offset = 0 while True: params = {"include": "items", "offset": offset, "limit": 1000} if status: params["status"] = status page = client.get(f"/v1/jobs/{job_id}", params=params).json() yield from page["items"] if len(page["items"]) < 1000: return offset += 1000 job = wait(job_id) for item in items(job_id): result = item["result"] or {} print(item["index"], item["status"], item["error"], len(result.get("text") or "")) ``` ```ts async function wait(jobId: string) { let delay = 5_000; for (;;) { const job = await (await fetch(`${BASE}/v1/jobs/${jobId}`, { headers })).json(); if (job.status === "completed" || job.status === "cancelled") return job; await new Promise((r) => setTimeout(r, delay)); delay = Math.min(delay * 1.5, 60_000); } } async function* items(jobId: string, status?: string) { for (let offset = 0; ; offset += 1000) { const params = new URLSearchParams({ include: "items", offset: String(offset), limit: "1000" }); if (status) params.set("status", status); const page = await (await fetch(`${BASE}/v1/jobs/${jobId}?${params}`, { headers })).json(); yield* page.items; if (page.items.length < 1000) return; } } await wait(job_id); for await (const item of items(job_id)) { console.log(item.index, item.status, item.error, item.result?.text?.length ?? 0); } ``` Each item has its `index` in your original list, its `input`, the `result` (the same object `POST /v1/fetch` or `POST /v1/search` returns, with `request_id` `:`) and an `error`. `result` is `null` with `error: "result_unavailable"` when the stored result cannot be read right now — read that page again later. ## Idempotency `idempotency_key` is required and scoped to your project. Posting the same key again returns the **existing** job (`200`, `created: false`) instead of creating a second one, so a client that crashed or timed out while creating a job can simply retry. Cormorant recognises a replay by key, kind and item count: the same key with a different kind or number of items is `409`. A changed list with the same length is *not* detected — derive keys from your run's identity (for example `--`, or a hash of the item list) and never reuse one for different content. ## Cancelling `DELETE /v1/jobs/{job_id}` stops admitting new items within a few seconds. Pending items become `cancelled`, items already queued skip themselves, and finished results stay readable. The response is the job status. ## Retention Jobs and their item results are kept for **30 days after the job finishes**, then deleted. Copy the results you need before that. The exact bytes behind fetch results (`body_url`) follow their own retention (see [Fetching in depth](/docs/fetching.md)). # Fetching in depth `POST /v1/fetch` turns a URL into text, metadata and provenance. This guide explains what happens in between and how to steer it. ## The order of operations 1. **Normalise** the URL (lower-case host, default port and fragment dropped, tracking parameters such as `utm_*`, `gclid`, `fbclid` removed, remaining parameters sorted). This is the cache key. 2. **Cache.** A fresh entry that satisfies your request answers at once, with `cached: true` and `cost_usd: 0`. The cache is shared by all projects: HTML pages stay fresh for 7 days, PDFs for 30. A stale entry is revalidated with a conditional request: if the origin answers `304 Not Modified`, you get the cached copy with `cached: true` and `revalidated: true`, billed as a cache hit. 3. **Safety.** Private, loopback, link-local and metadata addresses are refused (`blocked_destination`), also after redirects. A host whose name does not exist (NXDOMAIN twice within 6 h) answers `error: "domain_dead"` at once for 6 h — or its archived copy, see below. 4. **robots.txt** is honoured for `CormorantBot` and `*` (`robots_disallowed`). A few hosts are fetched **only through the premium routes**, never from Cormorant's own addresses. For them, `allow_premium: false` answers `premium_route_required`, robots.txt is read through the proxy, and a site-wide throttle applies. 5. **Domain memory.** A domain where every tier failed in 3 consecutive fetches is skipped for a while (10 min, then 30 min, 2 h, 6 h) and answers `error: "domain_unavailable"` at once, with `retry_after_s`. Any success ends the skip. Cormorant also remembers which tier (and rendering difficulty) worked for a domain and starts there next time, and which domains refuse even the premium routes: those skip `proxy`/`proxy_render` for 6 h (then 24 h, 3 days) and go to the archive and alternative sources instead, so a blocked domain stops costing premium bytes (`premium_skipped_domain_refuses` attempts). 6. **Tiers** run until one produces a good page. ## Tiers and escalation | Tier | What | Route | |---|---|---| | `direct` | HTTP with Chrome's TLS/HTTP2 fingerprint and headers, redirects, compression | standard | | `browser` | Chromium (new headless) with per-site cookies; waits out JS interstitials | standard | | `archive` | recent Wayback Machine snapshot (≤ 365 days), only when blocked | standard | | `proxy` | residential network (several networks, failover between them) | premium (also per GB) | | `proxy_render` | rendering service behind residential addresses | premium | The ladder starts at `direct` (or at a tier learned for the domain, see architecture). It escalates on network errors, HTTP 403/429/451/5xx and bot-challenge pages (Cloudflare, Akamai, DataDome, PerimeterX, Imperva, Kasada, generic interstitials). Thin main text escalates `direct → browser` only. `archive` and the premium routes are used only when the other standard routes were *blocked* — never for a page that is simply short. Definitive answers (404, 410, 401, …) are returned without escalation — a 404/410 on `direct` never reaches a premium route (at most the archived copy, see below). No CAPTCHA is solved unless the operator enabled it for the project. **Domains that refuse the premium routes too.** When a fetch fails on every tier and every premium route it tried was refused by the site (403/429/451, a challenge, a rendering-tier captcha or "cannot render", the provider's content policy — not provider errors or timeouts) in 2 fetches in a row, the domain is *premium-blocked* for 6 hours (then 24 h, then 3 days on repeats): its fetches still try `direct`, `browser` and the archive, but skip `proxy` and `proxy_render` (attempt reason `premium_skipped_domain_refuses`, no premium bytes) and go straight to the alternative sources, or fail with `all_tiers_failed`. 5 % of these fetches re-probe the premium routes anyway; one premium success ends the block. **Timing.** Each attempt's `ms` splits into `wait_ms` (waiting for a polite per-domain slot) and `upstream_ms` (the site, browser, archive or premium route). `timing` sums them for the request and adds `queue_ms`, the time in Cormorant's queues before a worker ran it (synchronous: from the API to a worker; batch: since admission, politeness requeues included; plus the render queue after a browser handoff): `{"queue_ms": 35, "politeness_wait_ms": 0, "upstream_ms": 412}`. The same three components are stored per request in the request log and summarised in `GET /v1/usage`. Every attempt carries `premium_bytes` (what the residential network billed for it; `0` for standard routes and `proxy_render`, which bills per call). The `proxy` route can show more than one attempt: when the site refuses a residential exit, or the network fails, the request is retried at once through another residential network, each attempt with its own status, reason and premium bytes. Prices are per route and do not depend on which network served the page. A failed fetch says which premium routes were tried in `premium_routes_tried` and summarises every attempt in `error_detail`, e.g. `"all_tiers_failed: direct (http_403), browser (challenge_akamai), proxy (http_403, 16211 premium bytes), proxy_render (http_403); premium routes tried: proxy, proxy_render (16211 premium bytes)"`. **Dead domains and missing pages.** A host whose name does not exist (NXDOMAIN — not a temporary DNS failure such as SERVFAIL or a timeout, which stay `blocked_destination` with reason `dns_error`) twice within 6 hours is remembered as dead for 6 hours: fetches answer `error: "domain_dead"` at once, with a single `{"tier": "dns", "reason": "domain_dead"}` attempt and no tier work. When a fetch ends in `domain_dead` or a 404/410 and `allow_archive` is true (and `tiers`, if given, includes `archive`), Cormorant looks for a Wayback snapshot (≤ 365 days) and, if there is one, returns it instead: `tier: "archive"`, `archived_at`, and `served_from_archive_because: "domain_dead"` or `"not_found"`; `attempts` still shows the 404 or the dead host. These stand-in copies are not cached. The archive's paywall rules apply (no copies for known paywalled domains or of articles marked subscriber-only, `archive_blocked_paywall`). Without a snapshot the answer stays `domain_dead` or `http_404`/`http_410` (with an `archive_not_found` attempt), and only then are alternative sources looked for. Other definitive statuses (401, …) never use the archive. `render=never` uses `direct → archive → proxy`; `render=always` uses `browser → proxy_render`. **Known hard domains.** For sites on Cormorant's list of hard domains (`nestle.com`, `nestleusa.com`, `beyondmeat.com`, `apnews.com`, `nytimes.com`, `researchgate.net`, … — see [Fetching → Known hard domains](/docs/fetching.md#known-hard-domains)) tiers that do not work for the site are skipped (`hard_domain_skip` attempts), the residential exit country is pinned (unless the request sets `country`) and `proxy_render` starts at the site's difficulty. Errors — and archive or alternative copies standing in — carry `hint` (the recommended route) and, on errors, `alternative_urls`. An explicit `tiers` list overrides the route. The `proxy` tier sends GET requests with Chrome's TLS/HTTP2 fingerprint, like `direct`. `attempts[].reason` values include `ok`, `thin_content`, `not_modified`, `challenge_`, `http_`, `timeout`, `tls_error`, `dns_error`, `dns_nxdomain`, `domain_dead`, `network_*`, `not_configured`, `budget_exhausted`, `hard_domain_skip`, `archive_not_found`, premium-route reasons (`…_captcha_rejected`, `…_no_credits`, `…_render_503`), `paywall` (teaser of a hard paywall, returned as is), `paywall_skip_premium`, `premium_skipped_domain_refuses`, `archive_blocked_paywall`, and for the alternative-source steps (`tier: source`) `alternate_` (with the copy's URL in `detail`), `syndication_candidates_`, `open_access_none`, `open_access_miss`, `guardian_api_miss`, `publisher_feed_miss`, `publisher_feed_teaser`, `common_crawl_none`, `common_crawl_not_free`, `common_crawl_paywalled_domain`, `common_crawl_index_error`, `common_crawl_record_error`, `source_timeout`. `proxy_render` attempts carry `detail` with the difficulty used (`low`, `medium`, `high`); other attempts omit `detail`. Busy rendering-tier answers (`…_render_503`) are retried once and appear as two attempts. ## What you get back | Field | What it is | |---|---| | `text` | The retained text: main content for HTML (trafilatura, readability fallback, then visible body text), pages joined by blank lines for PDFs, the decoded document for JSON/XML/plain text, empty for binary. Deterministic — the same bytes always give the same text, and cache hits return identical text, so you can anchor quotes against it. | | `pdf_text_pages`, `pdf_page_count`, `text_truncated` | For PDFs: `[{"page": 1, "text": "…"}, …]`, the number of pages, and whether the text stops early (very long documents). | | `meta` | For HTML pages: author, published/modified dates, language, canonical URL, publisher, site name, description, section and word count ([details](/docs/api.md#article-metadata-meta)). | | `html` | The decoded HTML of HTML pages (sync only by default; see `include_html`). | | `title` | Document title, if found. | | `content_kind` | `html`, `pdf`, `text`, `json`, `xml` or `binary` — from the content type, sniffed when it is missing or generic. | | `status`, `final_url`, `redirects` | The HTTP status, the URL after redirects (`null` when unknown, for `proxy_render`) and each redirect hop. | | `content_type`, `origin_content_type`, `headers` | What the body is, what the origin said it was, and an allowlist of the origin's response headers (never cookies). | | `tier`, `attempts` | Which tier produced the page, and every attempt with its reason, duration (`ms` = `wait_ms` for a polite slot + `upstream_ms`) and `premium_bytes`. | | `timing` | Where the time went: `queue_ms` (Cormorant's queues), `politeness_wait_ms` (busy domain), `upstream_ms` (the site or provider). | | `error_detail`, `premium_routes_tried` | On failures: a one-line account of every attempt, and the premium routes that were actually tried. | | `sha256`, `bytes`, `body_url`, `body_kind` | The exact retrieved bytes and where to download them. | | `cached`, `retrieved_at`, `etag`, `last_modified`, `archived_at` | Freshness and provenance. | | `cost_usd`, `error`, `request_id` | Spend, outcome and the id to quote. | ### Non-HTML content `content_kind` tells you how to read `text`: - `json` — `text` is the decoded document (still valid JSON); - `xml` — feeds, sitemaps, XBRL…: one text node per line, `title` from the first ``; external entities are never resolved; - `text` — plain text, CSV, Markdown, decoded; - `binary` — spreadsheets, archives, images…: `text` is empty, download the bytes from `body_url`. All of these are successful fetches (`error: null`). ### PDFs PDFs are accepted by default (`accept_pdf: false` turns them into `error: "pdf_not_accepted"`). Text is extracted per page; use `pdf_text_pages` to cite page numbers. Large reports may need a higher `max_bytes` (default 15 MB, at most 50 MB); a bigger body is `error: "too_large"`. ## Exact bytes and provenance Every fetch stores the exact bytes it retrieved, content-addressed by `sha256`, and returns `body_url` (null when object storage is unavailable). `body_kind` says what the hash covers: - `origin`: the HTTP entity body the origin (or the archive, for `tier = archive`) sent, after content-encoding (gzip/br) is removed — the bytes a plain HTTP client would hand you; - `rendered_dom`: the document serialized by a browser (`browser`, `proxy_render`) after scripts ran — use it for text, not as proof of the origin's bytes. Non-HTML documents a browser downloads (PDFs) are `origin`. `GET /v1/bodies/{sha256}` (same Bearer key) streams the bytes with the origin `Content-Type`, `Content-Length`, `ETag: "<sha256>"`, `X-Body-Kind` and `Cache-Control: immutable`; `If-None-Match` gives `304`; `HEAD` returns the headers only. A project can download only bodies it has retrieved itself (fresh or from the shared cache); anything else is `404`. Bodies are kept while a cache entry points to them or a project retrieved them within the last 30 days. Downloading a body (Python and TypeScript continue the [Quickstart](/docs/quickstart.md) client): ```sh curl -sS "$BODY_URL" -H "Authorization: Bearer $CORMORANT_API_KEY" -o report.pdf ``` ```python page = client.post( "/v1/fetch", json={"url": "https://www.example.com/annual-report-2025.pdf"} ).json() if page["body_url"]: with client.stream("GET", page["body_url"]) as response: response.raise_for_status() with open("report.pdf", "wb") as out: for chunk in response.iter_bytes(): out.write(chunk) ``` ```ts const bytes = await fetch(page.body_url, { headers }).then((r) => r.arrayBuffer()); ``` For citations, keep `final_url`, `retrieved_at`, `sha256` and `body_kind` with the quote. For `tier = archive`, `final_url` is the Wayback Machine copy and `archived_at` the snapshot time. ## Cache semantics - `cache_ttl_s` is the **maximum age you accept**. `0` bypasses the cache and forces a fresh fetch. - A cache hit is used only if it satisfies your request: `render: "always"` needs a rendered copy and `render: "never"` an origin copy; HTML text must reach your `min_text_chars`; a cached body larger than your `max_bytes` answers `too_large`, exactly as a fresh fetch would; with `tiers` set, hits from other tiers are not used. - Pages that are simply short (direct and the browser both gave less text than `min_text_chars`, nothing blocked) are cached for a day as such and answer later requests; thin pages of blocked fetches are not cached. - A redirected page is cached under the URL the redirects ended at too. - Hits are billed as cache hits (`fetch_cached`), even if the page originally came from a premium route. - Cache classes, TTLs and how to get hits across runs: [Caching and reuse across runs](/docs/caching.md). ## Per-request controls | Field | Use it to | |---|---| | `render` | `auto` (default) climbs the ladder; `never` avoids browsers (`direct → archive → proxy`); `always` goes straight to a browser (`browser → proxy_render`) for JavaScript-only pages. | | `allow_premium` | `false` never uses `proxy` or `proxy_render` — the fetch stays on the standard routes. | | `tiers` | Restrict the ladder, e.g. `["direct", "browser"]`. If nothing remains: `error: "no_tier_allowed"`. | | `allow_archive` | `false` never answers from the Wayback Machine — neither when the live site blocks us nor in place of a dead domain or a 404/410 (`served_from_archive_because`). | | `timeout_s` | A deadline (5–150 s) for the whole fetch across tiers. When it runs out you get `error: "timeout"` — or the best page found so far. | | `min_text_chars` | Main text shorter than this (default 500) counts as thin and escalates `direct → browser` only. Lower it for short pages (product pages, abstracts). | | `max_bytes`, `accept_pdf` | Size cap and PDF acceptance. | | `country` | Geotargeting for the premium routes (ISO 3166-1 alpha-2). | | `accept_language` | The `Accept-Language` header / browser locale, e.g. `de-CH,de;q=0.9`. | | `user_agent` | Your own User-Agent for `direct`, `browser` and `proxy` — some services (SEC EDGAR, Wikimedia) require a contact address. It must be on your project's allowlist, which the operator maintains; otherwise `403 user_agent_not_allowed`. You do not need one for Wikimedia: see below. | | `include_html` | Return `html` (default `true` for `/v1/fetch`, `false` in batches). Turn it off when you only need text — responses get much smaller. | | `project_ref` | Your own text stored with your usage, e.g. the entity the fetch was for. | A cost-capped, latency-capped fetch for an agent: ```json { "url": "https://www.holcim.com/investors", "allow_premium": false, "timeout_s": 45, "include_html": false } ``` A filing from a service that requires a contact User-Agent: ```json { "url": "https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000320193", "user_agent": "Example Research contact@example.org", "tiers": ["direct"], "render": "never" } ``` ### Wikipedia, Wikidata and other Wikimedia sites Fetches of `*.wikipedia.org`, `*.wikidata.org`, `*.wikimedia.org` and the sister projects follow Wikimedia's [User-Agent policy](https://meta.wikimedia.org/wiki/User-Agent_policy) for you: Cormorant identifies itself with its own descriptive User-Agent and the operator's contact (or your allowlisted `user_agent`, when you send one), uses the `direct` tier only (no browser or premium routes, so these fetches are never charged as premium) and keeps the number of concurrent requests to all Wikimedia hosts low across all clients. Under load, expect requests to wait briefly for a slot rather than fail; if Wikimedia answers 429 the fetch fails (`all_tiers_failed`, `attempts` shows `http_429`) and every worker slows down for a few minutes, so retry later rather than at once. Prefer the APIs (`https://en.wikipedia.org/api/rest_v1/page/summary/Zurich`, `https://www.wikidata.org/wiki/Special:EntityData/Q72.json`) to HTML pages: they are smaller and cached the same way. ### Known hard domains Some sites block automated access from datacenter addresses on every route Cormorant has, or answer only behind a paywall. Cormorant ships a list of them with the route that works best per site (operators can extend or override it with `HARD_DOMAINS_FILE`): | Domain | Protection | Route | |---|---|---| | `nestle.com` | Cloudflare managed challenge | skip the browser; residential proxy with a Chrome fingerprint from a Swiss exit, then the rendering tier (`proxy_render`, `medium`) | | `nestleusa.com` | Cloudflare | as above, US exit | | `beyondmeat.com` | Cloudflare | skip the browser; residential proxy (US exit), then the rendering tier | | `apnews.com` | Cloudflare | skip the browser; archive, residential proxy, then the wire copy elsewhere (`syndication`) or a Common Crawl capture — also on synchronous calls | | `nytimes.com` | DataDome + hard paywall | no browser or premium routes: only open copies (archive, publisher feed, syndication) | | `researchgate.net` | Cloudflare on every route | no browser or premium routes; Common Crawl; look for the DOI / open-access copy (`alternative_urls`: OpenAlex, Semantic Scholar) | | `medium.com`, `indeed.com` (all country sites) | Cloudflare | skip the browser (Indeed: and the rendering tier) | | `reuters.com`, `barrons.com` | robots.txt + DataDome | no browser or premium routes; wire copies via syndication | Skipped tiers show as `hard_domain_skip` attempts; a request's own `tiers` list overrides the route, and its `country` wins over the route's exit country. When such a fetch fails — or an archived or alternative copy stands in — the result carries `hint` (the route in one sentence) and, on errors, `alternative_urls` (e.g. the page's Wayback Machine URL, or an OpenAlex search built from a ResearchGate URL). The residential proxy tier always presents Chrome's TLS/HTTP2 fingerprint (like `direct`), and the browser tier presents a headful Chrome identity (no `HeadlessChrome` in the User-Agent or client hints). For a company's own site (10–15 pages per company), use a batch fetch with an `idempotency_key`: batch items wait out busy and blocked domains instead of failing, and may use Common Crawl. ## Errors The fetch-level `error` codes, what they mean and whether to retry them are in [Errors & retries](/docs/errors.md). <!-- source: https://www.cormorant.ch/docs/caching.md --> # Caching and reuse across runs Cormorant keeps a **shared cache** of search answers and fetched pages. A hit answers in milliseconds, is billed at the cache price (`cached: true`, billing category `search_cached` / `fetch_cached`) and is the same for every project. This guide lists what is cached for how long, and how to get hits when you run the same job again. ## Cache classes | What | Fresh for (default) | Kept after that | Key | |---|---|---|---| | Search answers (`/v1/search`, batch search) | 7 days | not deleted: older answers are reused when your `cache_ttl_s` accepts them | normalised query (whitespace collapsed, lower-case) + `language` + `country` + `max_results` + `freshness` | | HTML pages | 7 days | 30 days, for revalidation and longer `cache_ttl_s` | normalised URL (see below) | | PDFs | 30 days | 30 days | normalised URL | | Short pages: `direct` and the browser both gave less text than `min_text_chars`, and nothing blocked | 1 day | 30 days | normalised URL | | SEC EDGAR | archives (filings, documents) 365 days; metadata 12 h; full-text search 1 h | 30 days | normalised URL | The same defaults apply to every customer. The operator can change them (`SEARCH_CACHE_TTL_S`, `HTML_CACHE_TTL_S`, `PDF_CACHE_TTL_S`, `SHORT_PAGE_CACHE_TTL_S`). **Not cached:** - errors - hard-paywall teasers (so open copies are looked for again next time) - thin pages from a fetch that was blocked - archived copies standing in for a 404 or a dead domain - alternative-source copies (`tier: "source"`) - search answers where Brave failed after SearXNG returned too little An **expired** HTML or PDF entry with an `ETag` or `Last-Modified` is revalidated with a conditional request. If the origin answers `304 Not Modified`, you get the cached copy (`cached: true`, `revalidated: true`), billed as a cache hit, and it is fresh again. ## How URLs are normalised The cache key of a fetch is the normalised URL: - scheme and host are lower-cased, and the default port and the fragment are dropped - tracking parameters are removed (`utm_*`, `gclid`, `fbclid`, `mc_*`, `pk_*`, …) - the remaining query parameters are sorted, and percent-encoding is normalised `http://Example.com:80/a?b=2&a=1&utm_source=x#top` and `http://example.com/a?a=1&b=2` are the same entry. `http` and `https`, `www.` and the bare host, and a trailing `/` are **different** URLs. When a fetch is redirected, the page is also cached under the URL the redirects ended at. A later request for `https://www.site.com/` then hits the entry that a request for `http://site.com` created. For the most hits, send the URLs you got back as `final_url`. ## `cache_ttl_s`: how old an answer you accept - Unset: the class's freshness above. - `cache_ttl_s: 0`: bypass the cache and fetch or search again. The new answer replaces the entry. - Any other value is the **maximum age you accept**, measured from when the answer was retrieved. It can be shorter than the default (`3600` for "at most an hour old") or **longer**. With `cache_ttl_s: 2592000` (30 days), a re-run a month later still reuses last month's searches and pages that are still kept. Cache hits must still satisfy the request: `render`, `tiers`, `accept_pdf`, `max_bytes` and `min_text_chars` all apply. A page with less text than your `min_text_chars` is fetched again, unless it was cached as a short page. ## Reusing the cache across runs For a recurring enrichment run (the same companies every week or month): 1. **Send the same request strings.** Use the same query text and the same `language`, `country`, `max_results` and `freshness` for searches. Use the same URLs (preferably the `final_url` of the last run) for fetches. Avoid volatile parts in queries, such as today's date or a run id. Put those in `project_ref` instead, which is not part of the key. 2. **Accept the age you can live with.** Set `cache_ttl_s` to the interval between runs (e.g. `604800` for weekly runs, `2592000` for monthly). Every item that is still in the cache then answers instantly, as a cache hit. 3. **Force freshness only where needed.** Use `cache_ttl_s: 0` on the few URLs that must be live (e.g. a news page), not on the whole job. 4. **Check the hit rate.** Results and `GET /v1/requests` carry `cached`, and `GET /v1/usage` shows `cache_hit_rate` per kind and the `search_cached` / `fetch_cached` categories. A first run naturally has few or no hits. Every query and URL is new, unless another project asked the same thing within the TTL, because the cache is shared. <!-- source: https://www.cormorant.ch/docs/search.md --> # Search in depth `POST /v1/search` answers a query with merged, deduplicated results from several search engines, falling back to a search API only when it has to. ## Request ```json { "query": "Nestlé sustainability report 2025", "language": "en", "country": "CH", "max_results": 20, "freshness": "month", "cache_ttl_s": 86400, "project_ref": "company:4711" } ``` Only `query` is required. `language` is ISO 639-1 (optionally with a region, `de-CH`), `country` ISO 3166-1 alpha-2, `max_results` 1–50 (default 20), `freshness` one of `day`, `week`, `month`, `year`. ## Providers and fallback 1. **Shared cache** — keyed by the normalised query (whitespace collapsed, lower case) plus `language`, `country`, `max_results` and `freshness`; entries live 7 days. `cache_ttl_s` caps the age you accept; `0` bypasses the cache. 2. **SearXNG** — Cormorant's own metasearch instances (Bing, Yahoo, Yep, Yandex, Seznam, Brave, Wikipedia, Bing News). A standard route. 3. **Brave Search API** — the fallback search service, used when SearXNG - fails or times out (`fallback_reason`: `searxng_error`, `searxng_timeout`, `no_searxng_instance`), - reports unresponsive engines and fewer than two engines answered (`unresponsive_engines`, or `captcha_engines` when an engine was CAPTCHA-blocked), or - returns fewer than 3 usable results (`too_few_results`). `provider` says who answered (`searxng` or `brave`), `fallback_reason` why the fallback was considered and `unresponsive_engines` which SearXNG engines failed for this query. When Brave is used, its results come first, followed by results only SearXNG found. If Brave is unavailable or your project's budget is exhausted, whatever SearXNG returned is served (and not cached) with `fallback_reason` set. A search that the engines answered but that found nothing returns `results: []` with `error: null`. Only when the providers could not answer is `error` set — for example `budget_exhausted`, `brave_http_429` or `searxng_timeout`. ## Results: nothing is lost Results are deduplicated by normalised URL (scheme, `www.`, trailing slash and tracking parameters ignored). For each URL, **every distinct snippet and every engine** that returned it is kept, in order of first appearance: ```json { "url": "https://www.nestle.com/sustainability/reporting", "title": "Creating Shared Value and Sustainability Report 2025", "snippets": ["Our 2025 report covers ...", "Download the full report ..."], "engines": ["bing", "yahoo", "brave_api"], "published_at": "2026-03-02T00:00:00", "rank": 1 } ``` More engines agreeing on a URL is a useful relevance signal; more snippets give an LLM more context before it decides to fetch the page. ## Costs SearXNG answers and cache hits are standard routes with no provider cost (billed as `search` and `search_cached`). A Brave call (`search_fallback`) costs a small fixed amount per request (operator-configured, on the order of half a cent), shown in `cost_usd` and charged to your project's budget. Brave's own rate limits are retried by Cormorant and never billed. ## Examples ```sh curl -sS https://api.cormorant.ch/v1/search \ -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query": "Holcim annual report 2025", "language": "en", "freshness": "year", "max_results": 10}' ``` ```python answer = client.post( "/v1/search", json={ "query": "Holcim annual report 2025", "language": "en", "freshness": "year", "max_results": 10, }, ).json() if answer["error"]: print("search failed:", answer["error"]) for hit in answer["results"]: print(f"{hit['rank']:>2} {hit['title']} ({', '.join(hit['engines'])})\n {hit['url']}") ``` ```ts const answer = await ( await fetch(`${BASE}/v1/search`, { method: "POST", headers, body: JSON.stringify({ query: "Holcim annual report 2025", language: "en", freshness: "year", max_results: 10 }), }) ).json(); for (const hit of answer.results) console.log(hit.rank, hit.title, hit.url, hit.engines.join(",")); ``` ## Tips - Searching is cheap; fetching is where the time goes. Use the snippets to decide which results deserve a fetch. - Keep `max_results` as small as you need: it is part of the cache key, so consistent values also mean more cache hits. Keep volatile parts (dates, run ids) out of the query; see [Caching and reuse across runs](/docs/caching.md). - For many queries use `POST /v1/batch/search` — see [Sync vs batch](/docs/sync-vs-batch.md). <!-- source: https://www.cormorant.ch/docs/errors.md --> # Errors & retries Cormorant reports problems on two levels: 1. **HTTP status codes** — something is wrong with the *request*: the key, a limit, validation, or a wait that ran out. 2. **The `error` field** in search and fetch results — the request was fine, but the *web* said no: a 404, a robots.txt rule, a site that blocks every tier. These come with HTTP `200` (or as a batch item's `error`). ## HTTP status codes | Status | Body | Meaning | Retry? | |---|---|---|---| | `422` | `{"detail": [{"loc": […], "msg": "…", "type": "…"}]}` or `{"detail": "…"}` | Invalid request (a field out of range, a bad date, a non-http `callback_url`). | No — fix the request. | | `401` | `{"detail": "missing bearer token"}` / `{"detail": "invalid API key"}` | No key, or an unknown/revoked key. | No. | | `403` | `{"detail": {"error": "customer_suspended", "message": "…"}}` | The account that owns the key's project is suspended (any `/v1` endpoint). | No — contact the operator. | | `403` | `{"detail": {"error": "user_agent_not_allowed", "hint": "…"}}` | A `user_agent` that is not on the project allowlist. | No — ask the operator or drop the field. | | `404` | `{"detail": "job not found"}` / `{"detail": "body not found"}` | Unknown job or body, or one that belongs to another project (or has expired). | No. | | `409` | `{"detail": "idempotency_key already used for a different job"}` | Same `idempotency_key` with a different kind or item count. | No — use a new key. | | `413` | `{"detail": "at most 50000 items per job"}` | Too many items (or a request body over 64 MB). | Split the job. | | `403` | `{"detail": {"error": "key_category_not_allowed", "message": "…"}}` | The operator limited this key to other categories (e.g. search only). | No — use another key. | | `429` | `{"detail": "rate limit exceeded"}` + `Retry-After: 1` | Project rate limit. | Yes, after `Retry-After`. | | `429` | `{"detail": {"error": "key_rate_limited", …}}` + `Retry-After: 1` | The key's own requests-per-second limit. | Yes, after `Retry-After`. | | `429` | `{"detail": {"error": "key_quota_exhausted", "category": "fetch", "limit": 100000, …}}` | The key used its monthly request quota for that category (`search`, `fetch`). `Retry-After` = seconds until the month resets. | Not before the reset — see `GET /v1/quota`. | | `429` | `{"detail": {"error": "key_budget_exhausted", "limit": "50.00", "currency": "USD", …}}` | The key reached its monthly spend cap. `Retry-After` = seconds until the month resets. | Not before the reset. | | `503` | `{"detail": "…"}` | A dependency is briefly unavailable (e.g. object storage while downloading a body). | Yes, with backoff. | | `504` | `{"detail": {"error": "timeout", "request_id": "req_…"}}` | No result within the sync wait (150 s, or `timeout_s` + 15 s). | Yes, later — the work often completes in the background and the retry is a cache hit. Better: use a batch. | ## Fetch errors (`error` in the result) | `error` | Meaning | |---|---| | `invalid_url` | not an http(s) URL | | `blocked_destination` | private, loopback, link-local or metadata address (also after redirects) | | `robots_disallowed` | robots.txt disallows the path for `CormorantBot`/`*` | | `too_large` | body exceeds `max_bytes` (batch PDFs with `max_bytes` unset: 200 MB) | | `pdf_not_accepted` | PDF with `accept_pdf=false` | | `http_<status>` | definitive upstream status (e.g. `http_404`) | | `all_tiers_failed` | every allowed tier failed or was blocked (`status` = last HTTP status; `error_detail` and `premium_routes_tried` say what was tried) | | `domain_unavailable` | every tier failed for this domain in 3 consecutive fetches: it is skipped for a while (10 min, then 30 min, 2 h, 6 h); `retry_after_s` says when to try again. Any success ends the skip | | `domain_dead` | the host name does not exist (NXDOMAIN twice within 6 h); answered at once for 6 h, without trying. With `allow_archive`, a snapshot is returned instead when one exists (`served_from_archive_because: "domain_dead"`) | | `timeout` | `timeout_s` ran out (see per-request controls), or a large PDF took longer than 5 minutes to download (attempt reason `pdf_download_timeout`) | | `no_tier_allowed` | `tiers`/`allow_premium` left no tier to try | | `politeness_timeout` | could not get a per-domain slot in time (transient; batch items are requeued) | | `domain_busy` | synchronous calls only, with **HTTP 503** and `Retry-After`: the expected wait for one of the domain's polite slots exceeds the 15 s a synchronous call waits; `retry_after_s` is that expected wait. Lower the parallelism per domain or use a batch (whose items wait instead) | | `sec_not_configured` | SEC EDGAR URL, but the operator has not configured Cormorant's SEC identity: SEC is disabled | | `rate_limited` | SEC EDGAR URL: the cluster-wide SEC rate limit (at most 10 requests/s for all clients) had no slot within 10 s; retry after `retry_after_s` (transient) | | `sec_unavailable` | SEC EDGAR throttled (403/429/503: all SEC requests back off), failed (5xx) or could not be reached; retry after `retry_after_s` (transient) | | `redirect_not_allowed` | SEC EDGAR URL that redirected to a host outside SEC (not followed) | | `premium_route_required` | a host fetched only through premium routes with `allow_premium: false` or a `tiers` list without `proxy`/`proxy_render` | | `budget_exhausted` | such a host, and the project's budget leaves nothing for the premium routes | When no tier reaches `min_text_chars`, the attempt with the most text is returned (no error). | `error` | Retry? | |---|---| | `invalid_url`, `blocked_destination` | No. | | `robots_disallowed` | No — the site asked crawlers not to fetch this path. | | `too_large`, `pdf_not_accepted`, `no_tier_allowed` | Not as is — change `max_bytes`, `accept_pdf`, `tiers` or `allow_premium`. | | `http_404`, `http_410` and other definitive `http_<status>` | No (perhaps much later, if the page may reappear). | | `all_tiers_failed` | Later (hours), or with premium routes allowed if you had `allow_premium: false`. Look at `attempts` (and `error_detail`, `premium_routes_tried`) to see what blocked and what it cost. A domain that refused the premium routes on recent fetches skips them for a while (`premium_skipped_domain_refuses`). | | `timeout` | Yes, with a larger `timeout_s` — or in a batch. | | `domain_unavailable` | Not before `retry_after_s` seconds — earlier requests get the same answer at once, without trying. Try another source meanwhile. | | `domain_dead` | No (hours at the earliest) — the host name does not exist. Look for the page elsewhere; with `allow_archive` (the default) you already got the archived copy when there is one. | | `politeness_timeout` | Yes, after a short pause: the domain was busy. Batches handle this for you. | | `domain_busy` (HTTP 503, sync only) | Yes, after `retry_after_s` (also in `Retry-After`): the domain's polite slots are taken for longer than a synchronous call waits. Send fewer parallel calls per domain, or use a batch. | | `rate_limited`, `sec_unavailable` | Yes, after `retry_after_s` (SEC EDGAR's shared rate limit or backoff). Batches handle this for you. | | `sec_not_configured` | No — SEC EDGAR is not enabled on this deployment; ask the operator. | | `redirect_not_allowed` | No — an SEC EDGAR URL redirected outside SEC. | Batch items are retried automatically for transient problems (connection errors, politeness timeouts, provider rate limits); what ends up in an item's `error` is final for that job. ## Search errors A search whose providers could not answer returns `results: []` and `error` — for example `budget_exhausted` (no budget left for the fallback search service), `brave_http_429` (the fallback was rate-limited) or a SearXNG failure such as `searxng_timeout`. These are transient: retry later with backoff. A search that the engines answered but that found nothing is **not** an error: `results: []` with `error: null` — rephrase the query instead of retrying. ## Backoff Retry only what is retryable, cap the number of attempts, and back off exponentially with jitter. Respect `Retry-After` when present. ```python import random import time import httpx RETRY_STATUS = {429, 502, 503, 504} # but not key_quota/budget_exhausted: see below def post_with_retries(client: httpx.Client, path: str, body: dict, attempts: int = 5) -> dict: for attempt in range(attempts): try: response = client.post(path, json=body) except httpx.TransportError: response = None if response is not None and response.status_code not in RETRY_STATUS: response.raise_for_status() # 4xx other than 429: do not retry return response.json() if attempt == attempts - 1: break retry_after = response.headers.get("retry-after") if response is not None else None delay = ( float(retry_after) if retry_after else min(60, 2**attempt) * random.uniform(0.5, 1.5) ) time.sleep(delay) raise RuntimeError(f"{path} failed after {attempts} attempts") ``` ```ts const RETRY_STATUS = new Set([429, 502, 503, 504]); async function postWithRetries(path: string, body: unknown, attempts = 5): Promise<any> { for (let attempt = 0; attempt < attempts; attempt++) { let res: Response | undefined; try { res = await fetch(`${BASE}${path}`, { method: "POST", headers, body: JSON.stringify(body) }); } catch { res = undefined; // network error: retry } if (res && !RETRY_STATUS.has(res.status)) { if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`); return res.json(); } if (attempt === attempts - 1) break; const retryAfter = Number(res?.headers.get("retry-after")); const delay = retryAfter > 0 ? retryAfter * 1000 : Math.min(60_000, 2 ** attempt * 1000) * (0.5 + Math.random()); await new Promise((r) => setTimeout(r, delay)); } throw new Error(`${path} failed after ${attempts} attempts`); } ``` A fetch that returns `200` with an `error` is a *result*. Decide per code with the table above whether to retry it later; do not loop on it immediately. ## Debugging Every response and every batch item has a `request_id`. `attempts` shows each tier with its HTTP status, reason (`challenge_cloudflare`, `http_403`, `thin_content`, `timeout`, …) and duration. When you ask the operator about a request, send the `request_id`. <!-- source: https://www.cormorant.ch/docs/limits.md --> # Limits & politeness Cormorant is a good citizen of the web on your behalf. That shapes how fast you can go — mostly per *domain*, not per project. ## Hard limits | Limit | Value | What happens | |---|---|---| | Requests per project | 200/s by default (operator can set per project) | `429` with `Retry-After: 1`. A batch call counts as one request. | | Per-key limits | set by the operator per key (see below) | `429 key_quota_exhausted`, `key_budget_exhausted`, `key_rate_limited`; `403 key_category_not_allowed` | | Synchronous wait | 150 s (or `timeout_s` + 15 s) | `504 {"detail": {"error": "timeout", …}}` | | `timeout_s` | 5–150 s | validation error outside the range | | Items per batch job | 50,000 | `413` | | Request body | 64 MB | `413` — split big batches | | Fetch body size (`max_bytes`) | default 15 MB, at most 50 MB | `error: "too_large"` | | PDFs in batch fetches | up to 200 MB when the item leaves `max_bytes` unset (an explicit `max_bytes` applies to PDFs too) | `error: "too_large"` | | Large PDF download time | 5 minutes in total (or `timeout_s`, when shorter) | `error: "timeout"` | | PDF text | first 5,000 pages / 5,000,000 characters | `text_truncated: true`; the full PDF is at `body_url` | | Redirects | at most 10 followed per fetch | the fetch fails | | Search `max_results` | 1–50 | validation error | | Job page size (`limit`) | 1–1,000 | validation error | | Retention | jobs and results: 30 days after the job finishes; bodies: at least 30 days after your last retrieval | `404` afterwards | ## Per-key limits The operator can give each API key its own limits, enforced on every request (ask for them, e.g. to give a client or a notebook a bounded key): | Limit | What happens | |---|---| | Monthly spend cap (your invoice currency) | `429 key_budget_exhausted` until the month resets | | Monthly requests per category: `search`, `fetch` (SEC EDGAR requests count here), `fetch_premium`, `company` | `search`/`fetch`/`company`: `429 key_quota_exhausted`; `fetch_premium` used up: fetches still run, without the premium routes | | Requests per second | `429 key_rate_limited` with `Retry-After: 1` | | Allowed categories (e.g. search only) | `403 key_category_not_allowed`; without `fetch_premium`, fetches run without the premium routes | Batch jobs are checked when created and again for each item when it runs: items beyond a quota fail with the same error code and are not billed. Request counts are exact; the spend cap compares the charges of the aggregates (refreshed every 5 minutes) plus an estimate of the requests since then, so a key can overshoot its cap by a few minutes of traffic. Every authenticated response tells you where you stand: ``` RateLimit-Policy: "project";q=200;w=1, "key";q=10;w=1 RateLimit: "project";r=187;t=1, "key";r=9;t=1 X-Cormorant-Key-Budget-Remaining: 12.40 USD X-Cormorant-Quota-Remaining: fetch=18766, search=900 ``` `RateLimit-Policy`/`RateLimit` follow the IETF draft *RateLimit header fields for HTTP* (`q` limit, `w` window in seconds, `r` remaining, `t` seconds to reset). The two `X-Cormorant-…` headers appear only when the key has a spend cap or quotas. `GET /v1/quota` has the full picture, and `POST /v1/alerts` registers a signed webhook for 50 / 80 / 100 % of a limit. ## robots.txt Cormorant checks robots.txt for the user-agent token `CormorantBot` (then `*`) before fetching and answers `error: "robots_disallowed"` when a path is off limits. robots.txt files are cached for a day; a missing robots.txt (4xx) allows everything. ## Per-domain politeness To avoid hammering any single site, Cormorant limits concurrency **per domain and egress IP, across all of its hosts**: - at most 2 requests in flight to the same domain from the same egress, - at least 250 ms between request starts, - after a `429` or a bot challenge the interval doubles (up to 30 s) and decays again after about 10 minutes. What this means for you: - **Many URLs on many domains go fast.** A batch of 10,000 URLs spread over 2,000 sites runs in parallel across all workers. - **Many URLs on one domain go at the domain's polite rate** — a few requests per second at best, slower if the site pushes back. 5,000 pages from one site take tens of minutes however you send them. - **A domain that blocks every tier is skipped for a while.** After three consecutive fetches where every tier failed, fetches for that domain answer `domain_unavailable` with `retry_after_s` (10 minutes at first, up to 6 hours) instead of spending time and money on it. - **Batches handle this for you.** Admission is domain-aware: a domain that dominates your job drains at its polite rate while the other domains proceed, and items waiting for a busy domain do not block workers. - **Synchronous calls wait** up to 15 s for a slot on a busy domain. When the expected wait is longer (the requests ahead of yours, divided over the domain's slots, times how long a request to that site takes), the call answers at once with HTTP **503**, `error: "domain_busy"`, `retry_after_s` and a `Retry-After` header instead of holding your connection for a minute. Hitting one domain with many parallel sync calls is the slowest way to use Cormorant. - **A polite burst for a site you need several pages of.** Send `domain_concurrency` (up to 4) with a fetch to let up to that many requests to the domain run at once — e.g. the 10–15 pages of a company's own site. It is ignored while the site pushes back (429s, challenges) and for hosts with their own site-wide limits (e.g. Wikimedia); the 250 ms between request starts still applies. ### Recommended pattern: many pages per company For an enrichment run (say 10–15 pages of each company's own site), create **one batch job per slice of companies** with an `idempotency_key` you can re-post safely, and set `domain_concurrency: 4` on the items: ```json { "idempotency_key": "chipiron-run-2026-10-20-sites-0001", "items": [ {"url": "https://www.nestle.com/investors", "domain_concurrency": 4, "project_ref": "company:nestle"}, {"url": "https://www.nestle.com/sustainability", "domain_concurrency": 4, "project_ref": "company:nestle"} ] } ``` Batch items never fail with `domain_busy`: a busy domain's items go back to the queue for a few seconds while other companies proceed, so a job of 10,000 companies × 12 pages finishes at the polite rate of every site without holding a connection or a worker. If the job creation request itself fails (timeout, 5xx), re-post it with the same key — you get the existing job, not a duplicate. ## Search throughput Each SearXNG instance takes about 15 queries per second (a query takes ~4 s because engines are queried upstream); bursts queue instead of failing. The Brave fallback is capped cluster-wide. For hundreds of queries, use `POST /v1/batch/search`. ## Practical advice - Prefer one batch over many parallel sync calls. - Do not run your own retry loop on `politeness_timeout` or `all_tiers_failed` in tight succession — see [Errors & retries](/docs/errors.md). - Re-fetching the same URL within its cache lifetime is instant and billed as a cache hit; `cache_ttl_s: 0` forces a real request — use it only when you need it. - If a site needs a contact User-Agent (SEC EDGAR, Wikimedia), ask the operator to add yours to the project allowlist and send it with `user_agent`. <!-- source: https://www.cormorant.ch/docs/monitoring.md --> # Monitoring Cormorant's status Cormorant publishes the health of its subsystems — API, search, fetch, browser rendering, archive, premium routes, SEC EDGAR, batch jobs, storage and each company data source — so you can watch it from your own monitoring and tell "our pipeline is broken" from "Cormorant is having a problem". People can read the same information at [`/status`](/status); machines read `GET /v1/status`. No API key is needed, and the answer is shared by everyone and refreshed at most every 15 seconds, so polling it costs you nothing against your project's rate limit. ## Quick start ```sh # Tiny body for uptime monitors curl -s https://api.cormorant.ch/v1/status/summary # {"status":"operational","description":"All systems operational","updated_at":"…"} # Everything: components, reasons, metrics, incidents, maintenance curl -s https://api.cormorant.ch/v1/status ``` ## What to poll, and how often - **Uptime monitors** (Uptime Kuma, Better Stack, Pingdom, a cron job): `GET /v1/status/summary` and a keyword check for `"status":"operational"`. - **Your own dashboards and alerting**: `GET /v1/status`, then look at the components your integration depends on. The component ids are `api`, `search`, `fetch`, `browser_rendering`, `archive`, `premium_routes`, `sec` (SEC EDGAR: `/v1/sec/…` and fetches of SEC hosts; listed while SEC is configured), `company_profiles` (extended company profiles: profiles, discovery and company networks; listed while a premium route is configured), `batch_jobs`, `storage`, and one per active company data source in the group `company_data` (currently `company_gleif`, `company_uid_register`, `company_zefix_rest`, `company_lindas_zefix`, `company_sogc`, `company_ch_cantonal_excerpt`). A degraded company source only means fresher data from it may be delayed — the company register still answers. - **Interval**: every **30–60 seconds**. Polling more often only returns the same snapshot (`Cache-Control: public, max-age=15`); send `If-None-Match` with the last `ETag` to get a `304` when nothing changed. More than 120 requests per minute from one IP address get `429`. ## Why the status endpoints always answer 200 The HTTP code says whether *Cormorant answered*; the body says *how it is doing*. A status endpoint that returned `503` during an outage could not be told apart from a broken proxy, a DNS problem or your own network. So: | You see | It means | Alert | |---|---|---| | `200` and `status: operational` | all good | no | | `200` and `status: degraded` | slower or more failures than usual; requests mostly work | warning | | `200` and `status: partial_outage` / `major_outage` | part or all of the service is failing | page someone if you depend on it | | `200` and `status: maintenance` | announced maintenance in progress (see `maintenance`) | no, or informational | | timeout, connection error, any non-200 | Cormorant unreachable (or your network) | yes, after 2–3 consecutive checks | ## Recommended alert rules 1. **Unreachable**: 3 consecutive failed checks (timeout 10 s) → critical. 2. **Outage of what you use**: a component you depend on is `partial_outage` or `major_outage` in 2 consecutive checks → critical. 3. **Degradation**: a component you use is `degraded` for 10 minutes → warning. Short blips are normal: status is computed from the last 10 minutes of traffic (or the last hour, see below). 4. **Incidents**: a new entry in `incidents` with `status` other than `resolved` → notify your team with its `title` and latest `updates`. 5. Do **not** alert on a component that is missing: components not part of the deployment are not listed. Use the stable `id`, never `name`. `reasons` says why a component is not operational (`no_workers`, `queue_lag`, `low_success_rate`, …) and `metrics` gives the numbers behind it, e.g. the fetch success rate or the age of the oldest waiting request. ## Windows and minimum samples Rates need enough traffic to mean anything. Each component's `metrics` carries the counts behind its rate and the window they cover: | Component | Counts | Minimum sample | |---|---|---| | `search`, `fetch`, `sec` | `requests`, `failures` (uncached; the site's or caller's definitive answers excluded) | 20 requests | | `browser_rendering`, `archive` | `attempts`, `failures` (infrastructure failures only) | 10 attempts | | `premium_routes` | `proxy_attempts`/`proxy_failures`, `rendering_attempts`/`rendering_failures` | 10 attempts per tier | | `company_profiles` | `requests` (all operations; the rates cover the uncached ones), always the last 600 s | 10 uncached operations | - The default window is the last **600 s** (`window_s` at the top level and in each component's `metrics`). - When a component has fewer than its minimum sample there — moderate or bursty load — it reports over the last **3600 s** instead, with `metrics.window_s: 3600` (`proxy_window_s`/`rendering_window_s` for the premium routes). Its status is judged on that window too. - `success_rate: null` means even the hour is short of the minimum: no evidence either way, and the rate never changes the status then. - At most the 20,000 most recent requests are read per window. ## Example: check from Python ```python import httpx r = httpx.get("https://api.cormorant.ch/v1/status", timeout=10) r.raise_for_status() # non-200: Cormorant unreachable status = r.json() mine = {c["id"]: c for c in status["components"] if c["id"] in ("search", "fetch")} for c in mine.values(): if c["status"] in ("partial_outage", "major_outage"): print(f"ALERT {c['name']}: {c['status']} ({', '.join(c['reasons'])})") for incident in status["incidents"]: if incident["status"] in ("investigating", "identified", "monitoring"): print(f"Incident: {incident['title']} — {incident['updates'][0]['message']}") ``` ## Incidents, maintenance and history The operator posts incidents (with an impact and a timeline of updates) and announces maintenance windows; both appear in `GET /v1/status` and on the status page. During an announced window the affected components show `maintenance` rather than an outage. `GET /v1/status/history?hours=24` gives the worst status per 5 minutes for the last 24 hours (hourly up to 7 days) and the uptime per component — useful for SLA reports. ## Reference Public status of the service for your monitoring, and the page at [`/status`](/status). **No authentication** (an `Authorization` header is accepted and ignored), CORS-enabled, refreshed at most every 15 s and shared by all callers (`Cache-Control: public, max-age=15`, `ETag` / `If-None-Match` → `304`). Limited to 120 requests per minute per client IP (`429` with `Retry-After`). The status never contains host names, addresses, instance URLs, project names, keys or internal error strings — only component ids, statuses, reason codes and aggregate numbers. The status endpoints answer **`200` whatever the service status**: the status is in the body. Alert on the body (`status` is not `operational`), and treat a timeout, a connection error or any other HTTP code as "Cormorant is unreachable" — a status endpoint that answered `503` during an outage could not be told apart from a broken proxy or network in between. | Endpoint | Returns | |---|---| | `GET /v1/status` | overall `status`, every component with `reasons` and `metrics`, incidents and maintenance | | `GET /v1/status/summary` | only `status`, `description`, `updated_at` (for uptime monitors) | | `GET /v1/status/components/{id}` | one component plus the incidents and maintenance naming it (`404` for an id not in this deployment) | | `GET /v1/status/history?hours=24` | worst status per 5 minutes (per hour beyond 24 h, up to 168 h) and uptime per component; overall as `_overall` | ```json { "status": "degraded", "description": "Degraded performance", "updated_at": "2026-10-19T09:41:15+00:00", "window_s": 600, "components": [ {"id": "api", "name": "API", "description": "…", "status": "operational", "reasons": [], "metrics": {}, "group": null}, {"id": "fetch", "name": "Fetch", "description": "…", "status": "degraded", "reasons": ["low_success_rate"], "metrics": {}, "group": null}, {"id": "company_gleif", "name": "Company data: GLEIF", "description": "…", "status": "operational", "reasons": [], "metrics": {}, "group": "company_data"} ], "incidents": [ {"id": "inc_3f9a1c0b7e", "title": "Elevated fetch failures", "status": "identified", "impact": "degraded", "components": ["fetch"], "created_at": "…", "updated_at": "…", "resolved_at": null, "updates": [{"at": "…", "status": "identified", "message": "…"}]} ], "maintenance": [] } ``` **Statuses**, best to worst: `operational`, `maintenance`, `degraded`, `partial_outage`, `major_outage`. The overall `status` is the worst component status, except that a major outage of a component other than `api`, `search`, `fetch` or `storage` counts as a partial outage, and it is `maintenance` when the only components not operational are under announced maintenance. An unresolved incident shows its components at least at its `impact`. Components not part of the deployment are not listed (`archive` when archived copies are disabled, `premium_routes` when no premium route is configured, `sec` while no SEC identity is configured, `company_profiles` while no premium route is configured) — do not alert on a missing component. Counts and rates cover the last `window_s` = 600 s. A component with fewer than its minimum sample there (20 uncached requests; 10 attempts for a tier) reports `requests`/`attempts`, `failures` and `success_rate` over the **last hour** instead; `metrics.window_s` (`<tier>_window_s` in `premium_routes`) says which window was used, and the status is judged on that window. `null` rates mean even the hour has too few samples. | Component | Metrics (rates 0–1; `failures` and `window_s` with every rate) | Not operational when | |---|---|---| | `api` | — | database or cache unreachable (major) | | `search` | `success_rate`, `requests`, `engines_healthy_share`, `engines`, `instances_healthy_share`, `fallback_available`, `fallback_share`, `fallback_success_rate`, `workers_alive`, `queue_lag_s` | success rate of uncached searches < 0.95 / 0.80 / 0.50 (degraded / partial / major); healthy search instances < 50 % (degraded) or none (partial with the fallback search service, major without); healthy engines < 34 % (degraded); no worker; queue lag. `engines_healthy_share`: share of search engines (per instance) whose latest answer in the last 15 minutes returned results (the instance share until searches were observed); `engines`: `[{engine, status}]` with `ok` (healthy everywhere), `degraded` (on some instances) or `down` (rate limited, CAPTCHA or suspended everywhere) | | `fetch` | `success_rate`, `requests`, `latency_p95_ms`, `workers_alive`, `sync_queue_backlog`, `sync_queue_lag_s`, `batch_queue_backlog` | success rate of uncached fetches < 0.85 / 0.60 / 0.30 (4xx answers of the site, robots.txt, invalid or blocked URLs, too large, PDF not accepted and deferred are not failures); p95 > 60 s (degraded); no worker (major); queue lag | | `browser_rendering` | `success_rate`, `attempts`, `workers_alive`, `sync_queue_backlog`, `sync_queue_lag_s`, `batch_queue_backlog` | browser attempts failing for infrastructure reasons: success < 0.80 / 0.50 / 0.10; no browser worker (major) | | `archive` | `success_rate`, `attempts` | archive unreachable or erroring: success < 0.80 / 0.50 / 0.10 | | `premium_routes` | `proxy_success_rate`, `proxy_attempts`, `rendering_success_rate`, `rendering_attempts` (configured tiers only) | provider errors: success < 0.80 / 0.50 / 0.10 | | `sec` | `success_rate`, `requests`, `throttle_wait_p95_ms` (p95 wait for a slot in the cluster-wide SEC rate limiter), `backoff_active`, `rate_limit_rps` | SEC backing off after 403/429/503 (degraded, `sec_backoff`); success rate of uncached SEC requests < 0.85 / 0.60 / 0.30 (4xx, invalid identifiers and not found are not failures); p95 wait > 5 s (degraded, `throttled`) | | `providers` | — | an outside service Cormorant relies on (search fallback, premium routes, alternative sources, notifications) currently refuses our access (degraded, `provider_access`); which one is not shown — the operator is alerted | | `company_profiles` | `success_rate`, `requests`, `latency_p95_ms` (uncached extended company profile operations), `cache_hit_rate`, `refused_share` (operations the source refused: `blocked`, `rate_limited`), `rendering_share` (share served by the rendering service rather than the residential proxy), `throttle_wait_p95_ms` (p95 wait for a slot in the shared rate limit), `backoff_active`, `proxy_skipped` (the residential proxy is being skipped after a recent refusal); always over the last 600 s | success rate of uncached operations < 0.85 / 0.60 / 0.30 (not found, invalid identifiers, robots.txt, premium routes not allowed and exhausted budgets are not failures; `null` below 10); refused share ≥ 0.20 (degraded, `upstream_refusing`); the site throttle backing off after 429 (degraded, `site_backoff`); p95 wait > 10 s (degraded, `throttled`) | | `batch_jobs` | `scheduler_last_run_s`, `workers_alive`, `running_jobs`, `oldest_waiting_job_s`, `queue_backlog` | scheduler last ran > 30 s (degraded) / > 120 s ago (partial); no batch worker (major); a job waiting to start > 15 min (degraded) / > 60 min (partial) | | `storage` | `database_ok`, `cache_ok`, `object_storage_ok` (`null`: not configured) | database or cache unreachable (major); object storage unreachable (degraded) | | `company_<source>` (group `company_data`, one per active company data source, e.g. `company_gleif`) | `success_rate`, `requests` (last hour of calls to the source; `null` below 10), `latency_p95_ms`, `parse_failure_rate`, `canary_ok`, `backoff_active`, `schema_drift` (fields that used to be filled and now come back empty), `change_feed_lag_s` | success < 0.85 / 0.50 / 0.20; canaries (known companies checked every 15 minutes) failing (partial); backoff after repeated failures, schema drift, parse failures > 10 % (degraded). Counts at most as `degraded` for the overall status | Queue lag (`queue_lag_s`, `sync_queue_lag_s`: how long the oldest synchronous request has waited for a worker) above 15 / 60 / 120 s is degraded / partial / major. Rates computed on fewer than 20 requests (10 tier attempts) are reported but do not change a status. New metric keys may be added. **Reason codes**: `no_workers`, `queue_lag`, `low_success_rate`, `high_latency`, `engines_reduced`, `engines_unavailable`, `tier_errors`, `scheduler_stalled`, `jobs_waiting`, `database_unreachable`, `cache_unreachable`, `object_storage_unreachable`, `sec_backoff`, `throttled`, `provider_access`, `upstream_refusing`, `site_backoff`, `incident`, `maintenance`; company data sources also `canary_failed`, `source_backoff`, `schema_drift`, `parse_failures`. **Incidents and maintenance** are posted by the operator. `incidents` lists unresolved incidents (`investigating`, `identified`, `monitoring`) and those `resolved` within 7 days, newest first, each with its timeline (`updates`, newest first). `maintenance` lists `scheduled` and `in_progress` windows and those `completed` within 7 days, soonest first; an empty `components` list means the whole service. **History**: `GET /v1/status/history` returns, per component, `statuses` (oldest first; `null` where nothing was recorded) and `uptime` over the period, where a `partial_outage` bucket counts as 30 % downtime and a `major_outage` bucket as 100 %. Kept 7 days. <!-- source: https://www.cormorant.ch/docs/sec-edgar.md --> # SEC EDGAR via Cormorant Cormorant gives you company and filing data from SEC EDGAR — tickers and CIKs, filing histories, annual reports, filing documents, XBRL facts and full-text search — without running your own SEC client. It follows SEC's [fair-access rules](https://www.sec.gov/os/accessing-edgar-data) for you: - **Cormorant declares its own identity.** SEC requires every client to send a `User-Agent` naming the organisation and a contact address. Cormorant sends the operator's identity on every SEC request. **Do not send your own User-Agent**: for SEC hosts a `user_agent` or `accept_language` in `POST /v1/fetch` is ignored (and does not need to be on your project's allowlist). - **One shared rate limit.** SEC allows at most 10 requests per second per IP address. All Cormorant clients share one limit (8 requests/s by default) per egress IP; a request waits up to 10 s for a slot, then answers `error: "rate_limited"` with `retry_after_s`. - **Shared backoff.** When SEC throttles (HTTP 403, 429 or 503), every SEC request of that egress pauses — 10 s, doubling up to 10 minutes — and answers `error: "sec_unavailable"` with `retry_after_s` without contacting SEC. The `sec` component of [`GET /v1/status`](/docs/monitoring.md) shows `backoff_active` and `throttle_wait_p95_ms`. - **Direct only.** SEC is never reached through the browser, archived copies, the residential network or alternative sources. robots.txt is honoured and nothing is crawled: each call reads exactly the SEC documents it needs. - **Cached.** Filings never change, so filing documents and indexes are cached for a year; `company_tickers.json`, submissions and XBRL facts for 12 hours (then revalidated with conditional requests); full-text search for an hour. Cached answers are fast and do not count towards SEC's limit. SEC requests are metered as their own usage kind `sec` (categories `sec` and `sec_cached` in [`GET /v1/usage`](/docs/usage.md)); they cost Cormorant nothing upstream. Your project's rate limit still applies, and a key's `fetch` quota counts them. All examples use `$CORMORANT` for the base URL and `$KEY` for your key. ## Resolve a company ```sh curl -s "$CORMORANT/v1/sec/companies/resolve?ticker=TSN" -H "Authorization: Bearer $KEY" ``` ```json {"cik": 100493, "cik_padded": "0000100493", "title": "TYSON FOODS, INC.", "tickers": ["TSN"], "exchanges": ["NYSE"], "ambiguous": false, "candidates": [...], "error": null, "cached": true, "source_url": "https://www.sec.gov/files/company_tickers.json"} ``` Also `?cik=100493` (or `0000100493`), `?name=…` and `?isin=…`. `BRK.B` and `BRK-B` are the same ticker. A name that matches several companies answers `ambiguous: true` with the matches in `candidates` (and `cik: null`); pick one and continue with its CIK. ### Companies listed outside the US SEC EDGAR knows companies registered with the SEC, under their **US** tickers. A Swiss, Danish or Japanese home-market ticker (`NESN`, `NOVO-B`, `7203`) is not an SEC identifier: - **Foreign private issuers that file with the SEC** (20-F, 40-F, 6-K) are found by the ticker of their US listing or ADR: Novo Nordisk is `NVO`, Toyota `TM`, Canadian Natural `CNQ`. `latest` returns their 20-F / 40-F by default. Use the ADR ticker, or give the **ISIN**: `?isin=DK0062498333` finds the issuer (company register, GLEIF) and the SEC filer with its name (`NVO`). - **Companies that do not file with the SEC** (Nestlé: its ADR `NSRGY` trades over the counter without SEC registration) answer `error: "not_sec_filer"` when Cormorant knows the company: a listing on a non-US exchange (`NESN.SW` or `XSWX:NESN`), a home-market ticker or ISIN the company register knows. `hint` says what is known: `company` (`id`, `name`, `country`, `listings`, `lei`, `isins`), `listing` (the home listing) and `sec_candidates` — SEC filers with the company's name; when there is one, the company files through it: retry with that ticker. - Otherwise the answer is `error: "unknown_identifier"`: neither SEC nor Cormorant knows the ticker, CIK, name or ISIN. ```json {"error": "not_sec_filer", "message": "XCSE:NOVO-B is not registered with the SEC (Novo Nordisk A/S); the company files with the SEC as NVO: use that ticker", "hint": {"listing": "XCSE:NOVO-B", "company": {"id": "cmp_…", "name": "Novo Nordisk A/S", "country": "DK", "listings": ["XCSE:NOVO-B"], "lei": "…", "isins": ["DK0062498333"]}, "sec_candidates": [{"cik": 353278, "title": "NOVO NORDISK A S", "tickers": ["NVO"], …}]}} ``` Batch items (`POST /v1/batch/sec`) answer the same codes and `hint`, and `resolve`, `submissions`, `latest` and `facts` items take `isin` instead of `cik` or `ticker`. ## List filings ```sh curl -s "$CORMORANT/v1/sec/companies/100493/submissions?forms=10-K,10-K/A&since=2015-01-01&limit=20" \ -H "Authorization: Bearer $KEY" ``` `filings` are newest first, each with `accession`, `form`, `filing_date`, `report_date`, `primary_document`, `primary_document_url`, `index_url` and `is_amendment` (plus SEC's other columns). Cormorant also reads the pages of older filings that SEC lists under `files` — only those that can contain filings after `since` (`history=false` skips them). `filings_total` counts the matches before `limit`; the raw SEC JSON is at `body_url`. ## The latest annual report ```sh curl -s "$CORMORANT/v1/sec/companies/1094517/filings/latest" -H "Authorization: Bearer $KEY" ``` By default the newest `10-K`, `20-F` (foreign private issuers) or `40-F` (Canadian issuers). Amendments are included only when you ask for them: `?forms=10-K,10-K/A` returns a newer `10-K/A` with `is_amendment: true`. ## The documents of a filing ```sh curl -s "$CORMORANT/v1/sec/filings/100493/0000100493-25-000070" -H "Authorization: Bearer $KEY" ``` Accession numbers work with or without dashes. `documents` lists every file with `type` (`10-K`, `EX-21`, `EX-97`, `EX-101.SCH`, …), `description`, `size` and `url`. ## Fetch a document ```sh curl -s "$CORMORANT/v1/sec/documents/fetch" -H "Authorization: Bearer $KEY" \ -H "Content-Type: application/json" \ -d '{"cik": 100493, "accession": "0000100493-25-000070", "document": "tsn-20250927.htm", "include_html": false}' ``` The answer is exactly a [`POST /v1/fetch`](/docs/fetching.md) result — `text`, `pdf_text_pages` for PDFs, `content_kind`, `sha256` and `body_url` for the exact bytes — so you can keep your artifact handling. Only documents of the given filing can be fetched. You can also pass an SEC URL to `POST /v1/fetch` or a batch job directly; the same SEC rules apply. ```python import httpx client = httpx.Client(base_url=CORMORANT, headers={"Authorization": f"Bearer {KEY}"}, timeout=180) cik = client.get("/v1/sec/companies/resolve", params={"ticker": "ZTS"}).json()["cik"] latest = client.get(f"/v1/sec/companies/{cik}/filings/latest").json() if latest["error"] is None: doc = client.post( "/v1/sec/documents/fetch", json={ "cik": cik, "accession": latest["accession"], "document": latest["primary_document"], "include_html": False, }, ).json() print(latest["form"], latest["report_date"], len(doc["text"]), doc["body_url"]) ``` ## XBRL facts and full-text search ```sh curl -s "$CORMORANT/v1/sec/companies/100493/facts" -H "Authorization: Bearer $KEY" curl -s "$CORMORANT/v1/sec/search?q=%22avian%20influenza%22&forms=10-K&dateRange=5y" \ -H "Authorization: Bearer $KEY" ``` `facts` summarises the company's XBRL facts (concepts per taxonomy); the complete JSON — often several megabytes — is at `body_url`. `search` is EDGAR full-text search (filings since 2001): `total` and up to 100 `hits` per page (`from` for the next page), each with `accession`, `ciks`, `form`, the matching document and its `url`. Dates: `dateRange` = `all`, `10y`, `5y`, `1y`, `30d`, or `startdt`/`enddt`. ## Errors Every SEC answer is HTTP 200; check `error`: | `error` | What to do | |---|---| | `unknown_identifier` | Neither SEC nor Cormorant knows this ticker, CIK, name or ISIN. For a company listed outside the US use its ADR ticker or ISIN. Do not retry. | | `not_sec_filer` | The company is known but not registered with the SEC under this identifier; `hint.sec_candidates` names its SEC filing (e.g. the ADR) when there is one. Do not retry as is. | | `not_found` | SEC has no such filing or document, or no filing of the requested forms. Do not retry. | | `invalid_identifier` | Fix the CIK, accession number, document name, form type or date (`message` says which). | | `rate_limited` | Retry after `retry_after_s`. Batches retry automatically. | | `sec_unavailable` | SEC is throttling or failing: retry after `retry_after_s`, not earlier — the answer would be the same. | | `sec_not_configured` | SEC access is not enabled on this deployment; ask the operator. | Spread large jobs over time and prefer batch jobs: SEC's limit is shared by everyone, and cached answers are instant. ## Many companies: POST /v1/batch/sec For thousands of companies, send one batch job instead of thousands of calls: resolve, latest filing and the documents themselves, mixed in one job. Tickers work instead of CIKs for `submissions`, `latest` and `facts`. ```sh curl -s -X POST https://api.cormorant.ch/v1/batch/sec \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \ -d '{"items": [{"op": "latest", "ticker": "TSN"}, {"op": "latest", "ticker": "ZTS"}], "idempotency_key": "annual-reports-2026-10"}' # then: GET /v1/jobs/{job_id}?include=items ``` A typical 10,000-company run is two jobs: `latest` per ticker, then `document` for each `primary_document` you need. SEC allows 10 requests per second per IP for everyone together, so plan on roughly 10 operations per second at best (an operation reads one or more SEC documents; cached ones do not count against SEC's limit): a 10,000-item `latest` job takes about 20–40 minutes when nothing is cached. Items wait for the rate limit in the queue, never fail because of it; `rate_limited` and `sec_unavailable` are retried automatically. Company and filing data from SEC EDGAR. Cormorant sends **its own** declared identity to SEC (`"<Org name> <contact email>"`, set by the operator) and keeps all clients together under SEC's fair-access limit: one cluster-wide rate limit per egress IP (default 8 requests/s, never more than 10), queued for up to 10 s, and a shared backoff when SEC throttles (403/429/503). Only the direct tier is used — never the browser, archived copies, the residential network or alternative sources. **Do not send your own User-Agent** for SEC: a `user_agent`/`accept_language` in `POST /v1/fetch` is ignored for SEC hosts. The same policy applies to `POST /v1/fetch` and batch items for URLs on `www.sec.gov`, `data.sec.gov` and `efts.sec.gov`. All endpoints need an API key, count towards the project's rate limit and are metered as kind `sec` (no provider cost). Answers are HTTP 200 also when they failed; check `error`: | `error` | Meaning | |---|---| | `unknown_identifier` | neither SEC nor Cormorant knows this ticker, CIK, name or ISIN (a home-market ticker of a non-US company: use its ADR ticker or the ISIN) | | `not_sec_filer` | the company is known (a non-US listing such as `NESN.SW` / `XSWX:NESN`, the company register, GLEIF for an ISIN) but not registered with the SEC under this identifier; `hint`: `company`, `listing`, `sec_candidates` (SEC filers with its name, e.g. the ADR) | | `not_found` | SEC has no such filing or document (also: no filing of the requested forms) | | `invalid_identifier` | a CIK, accession number, document name, form type, date or query is malformed (`message` says which); nothing was requested from SEC | | `rate_limited` | no slot in the shared SEC rate limit within 10 s; retry after `retry_after_s` | | `sec_unavailable` | SEC throttles (all SEC requests back off) or failed; retry after `retry_after_s` | | `sec_not_configured` | SEC is not enabled on this deployment | | `timeout`, `too_large`, `robots_disallowed`, … | the fetch errors | Identifiers: CIKs as a number or 10 digits (`100493`, `0000100493`); accession numbers with or without dashes (`0000100493-25-000070`, `000010049325000070`); form types comma-separated (`10-K,20-F,40-F,10-K/A`). Only the SEC URL patterns behind each endpoint can be requested. Every answer has `error`, `message`, `retry_after_s`, `cached` (`true` when all SEC documents came from Cormorant's cache), `retrieved_at` (oldest document), `source_url` (the main SEC document), `body_url` and `sha256` (its exact bytes, e.g. the raw submissions JSON) and `request_id`. **Caching.** Filing documents and indexes (`/Archives/edgar/data/{cik}/{accession}/…`) are immutable and cached for 365 days (revalidated only when you ask with `cache_ttl_s`); `company_tickers.json`, submissions and XBRL facts for 12 h, then revalidated with conditional requests; full-text search for 1 h. ### GET /v1/sec/companies/resolve?ticker=TSN Also `?cik=100493` or `?name=tyson foods` (exactly one). From `company_tickers.json` and `company_tickers_exchange.json`: ```json {"cik": 100493, "cik_padded": "0000100493", "title": "TYSON FOODS, INC.", "tickers": ["TSN"], "exchanges": ["NYSE"], "ambiguous": false, "candidates": [{"cik": 100493, "cik_padded": "0000100493", "title": "TYSON FOODS, INC.", "tickers": ["TSN"], "exchanges": ["NYSE"]}], "error": null, "cached": true, "source_url": "https://www.sec.gov/files/company_tickers.json", "…": "…"} ``` `BRK.B` and `BRK-B` are the same ticker. A name matches exact titles first, then titles starting with or containing it; when several companies match, `ambiguous` is `true`, the company fields are `null` and `candidates` lists up to 10. A CIK without a listed ticker is resolved from its submissions. ### GET /v1/sec/companies/{cik}/submissions Query: `forms`, `since`, `until` (filing dates, `YYYY-MM-DD`), `limit` (default 100, max 10,000), `history` (default `true`: also read the pages of older filings SEC lists under `files`, only those that can hold filings after `since`). Company fields `cik`, `cik_padded`, `name`, `title`, `tickers`, `exchanges`, `entity_type`, `sic`, `sic_description`, `category`, `fiscal_year_end`, `state_of_incorporation`, `ein`, `lei`, `website`, `phone`, `business_address`, `former_names`; `filings` (newest first, at most `limit`), `filings_total`, `truncated`, and `files` (`name`, `filing_count`, `filing_from`, `filing_to`, `url`, `fetched`, `body_url`). Each filing: | Field | From SEC's column | |---|---| | `accession` | `accessionNumber` | | `form`, `filing_date`, `report_date`, `acceptance_datetime` | `form`, `filingDate`, `reportDate`, `acceptanceDateTime` | | `act`, `file_number`, `film_number`, `items`, `core_type`, `size` | `act`, `fileNumber`, `filmNumber`, `items`, `core_type`, `size` | | `is_xbrl`, `is_inline_xbrl` | `isXBRL`, `isInlineXBRL` (as booleans) | | `primary_document`, `primary_document_description` | `primaryDocument`, `primaryDocDescription` | | `primary_document_url`, `index_url` | built: the document and the `-index.htm` page (`null` URL when the primary document is in a sub-folder, e.g. ownership forms) | | `is_amendment` | built: the form ends in `/A` | ### GET /v1/sec/companies/{cik}/filings/latest?forms=10-K,20-F,40-F The newest filing (by filing date) of `forms` — default `10-K,20-F,40-F` (annual reports; amendments only when you list them, e.g. `10-K,10-K/A`): the filing fields above at the top level (`accession`, `form`, `filing_date`, `report_date`, `primary_document`, `primary_document_url`, `index_url`, `is_amendment`, …) plus `cik`, `cik_padded`, `name`, `tickers`, `exchanges`. Older filing pages are searched when the recent list has none; none at all: `not_found`. ### GET /v1/sec/filings/{cik}/{accession} The filing index: `form`, `filing_date`, `report_date`, `accepted`, `company`, `is_amendment`, `index_url`, `folder_url`, `accession`, `accession_nodash`, `index_source` (`index_htm`, or `index_json` when only the folder listing has the documents) and `documents`: `sequence`, `description`, `document` (file name), `type` (`10-K`, `EX-21`, `EX-101.SCH`, …), `size`, `url`. Sub-folders are not listed. ### POST /v1/sec/documents/fetch ```json {"cik": 100493, "accession": "0000100493-25-000070", "document": "tsn-20250927.htm", "max_bytes": 52428800} ``` Fetches one document of a filing (HTML/iXBRL, PDF, exhibit, XML) and answers **exactly like `POST /v1/fetch`** (`text`, `pdf_text_pages`, `content_kind`, `body_url`, `sha256`, `body_kind: origin`, …). Optional: `include_html` (default `true`), `accept_pdf`, `cache_ttl_s`, `timeout_s`, `project_ref`. Invalid identifiers answer `error: invalid_identifier` in that shape. ### GET /v1/sec/companies/{cik}/facts A summary of `data.sec.gov/api/xbrl/companyfacts/CIK##########.json`: `cik`, `cik_padded`, `name`, `taxonomies` (concepts per taxonomy), `concepts`, `bytes`; the complete JSON is at `body_url`. ### GET /v1/sec/search?q=…&forms=…&dateRange=… EDGAR full-text search (filings since 2001). Query: `q` (EDGAR syntax), `forms`, `dateRange` (`all`, `10y`, `5y`, `1y`, `30d`, or `custom` with `startdt`/`enddt`), `from` (offset, pages of 100). Answer: `query`, `total`, `total_relation`, `hits` (`accession`, `ciks`, `display_names`, `form`, `root_forms`, `file_type`, `file_description`, `file_date`, `period_ending`, `document`, `url`, `index_url`, `score`). ### Batch: POST /v1/batch/sec Up to 50,000 SEC operations as one job of kind `sec`: ```json { "items": [ {"op": "resolve", "ticker": "TSN"}, {"op": "latest", "ticker": "TSN", "forms": "10-K"}, {"op": "latest", "cik": 100493}, {"op": "document", "cik": 100493, "accession": "0000100493-25-000070", "document": "tsn-20250927.htm"} ], "idempotency_key": "chipiron-run-2026-10-20-sec-0001", "callback_url": "https://chipiron.example/hooks/cormorant" } ``` `op` is `resolve` (`ticker`, `cik` or `name`), `submissions` (`cik` or `ticker`, `forms`, `since`, `until`, `limit`, `history`), `latest` (`cik` or `ticker`, `forms`), `facts` (`cik` or `ticker`), `filing` (`cik`, `accession`) or `document` (`cik`, `accession`, `document`, `max_bytes`, `accept_pdf`, `include_html` — default `false` in a batch); `project_ref` on any item. Identifiers are checked when the job is created (`422 invalid_item` naming the item). Item results have the shape of the single endpoint (plus `op`); `document` results that of `POST /v1/sec/documents/fetch`. Items share the cluster-wide SEC rate limit with every other SEC request (at most 10 requests/s per egress IP) and are admitted a few at a time: a full limiter or a SEC backoff puts an item back in the queue instead of failing it. Each item is metered once as kind `sec` (a ticker resolved inside an item is not extra) and counts as `fetch` for key limits. <!-- source: https://www.cormorant.ch/docs/billing.md --> # Costs & billing Every call is billed at your plan's price, and every response tells you exactly what it is charged. What a call costs depends on the route that answered it: **standard routes** (the shared cache, Cormorant's own search engines, a direct request, a headless browser, archives) or **premium routes** (the search fallback service, the residential network and the rendering service), which are priced higher. ## What a call costs | Action | Route | Billing category | Control it with | |---|---|---|---| | Search answered by SearXNG | standard | `search` | — | | Search that falls back to Brave | premium (fallback search service, billed on its cost) | `search_fallback` | fewer, better queries; the cache | | Search answered from the cache | standard | `search_cached` | — | | Fetch with `direct`, `browser` or `archive` | standard | `fetch_own` | — | | Fetch with `proxy` | premium (residential network, also per GB transferred) | `fetch_premium` | `allow_premium: false` | | Fetch with `proxy_render` | premium (rendering service, per successful call × the site's difficulty) | `fetch_premium` | `allow_premium: false`, `tiers` | | Fetch answered from the cache | standard, even if the page first came from a premium route | `fetch_cached` | — | The premium routes are used **only when the standard routes were blocked** — never for a page that is merely short, never for a definitive answer such as a 404. In batches, an item blocked on the standard routes is retried once after a cool-down before a premium route is tried. Your plan sets the price of each category (and any included units). ## Seeing the cost - Every billable answer — search, fetch, batch items, `/v1/sec/…`, `/v1/companies/…` — has a `billing` block: what *this* request is charged under your plan (see below). `billing_category` says which category of your plan it counts toward. - `cost_usd` is the provider spend caused by the request (cache hits and standard routes report `0`) — `null` for keys of a customer account: customers see what they are charged, never what Cormorant pays. - `tier` and `attempts` show which tier answered and what was tried before. - `GET /v1/usage` sums it up per project. ### The `billing` block ```json "billing": { "category": "fetch_premium", "billable": true, "currency": "CHF", "charge": "0.014000", "within_allowance": false, "units": [ {"meter": "fetch_premium", "unit": "fetch", "quantity": "1", "included": "0", "charge": "0.004000"}, {"meter": "premium_gb", "unit": null, "quantity": null, "included": null, "charge": "0.010000"} ] } ``` - `charge` — this request at your plan price (decimal string, 6 places, in your invoice currency): the plan's volume tiers at your account's position in the month, its included units, contract rates and discount. `"0.000000"` inside the allowance. - `within_allowance` — `true` when every unit fell inside your plan's included units; `null` when nothing was priced (a failed request your plan does not bill, `billable: false`, or a category your plan does not charge). - `units` — per plan meter: units added, how many were included, their charge. Meters billed on provider cost show only their charge. - Not included: the monthly base fee, the minimum top-up and VAT (account lines of the invoice). The invoice rates the whole month at once and rounds each line to the cent, so it is authoritative; summing `charge` over a month matches it up to rounding. - Projects without a customer plan (internal use) get `currency` and `charge` `null`, and see `cost_usd`. ## Keeping it bounded ```json {"url": "https://www.example.com/page", "allow_premium": false} ``` `allow_premium: false` removes the premium routes from the ladder, so the fetch is always billed at a standard-route price; a site that blocks every standard route then answers `all_tiers_failed`. `tiers` gives finer control, for example `["direct", "browser", "proxy"]` to allow the proxy but not the rendering tier (`proxy_render`). ### Capping the cost of a run A run (a research job, an agent session) can enforce its own cost cap from the answers alone: 1. Before the run, read `GET /v1/quota` → `spend_budget`: your key's `cap`, `spent` and `remaining` this month (`cap: null` means no cap is set on the key — ask the operator for one, see below). 2. Keep a running total of `billing.charge` over the run's answers (batch items carry it too). Stop, or switch to `allow_premium: false`, when the total reaches the run's cap. 3. Budget for the worst case before expensive steps: a premium-route fetch is at least the plan's `fetch_premium` price, a search fallback is billed on provider cost. Answers with `within_allowance: true` are inside your plan's included units. ```python RUN_CAP = Decimal("5.00") # in your invoice currency spent = Decimal("0") for url in urls: page = cormorant.post("/v1/fetch", json={"url": url, "allow_premium": spent < RUN_CAP}).json() spent += Decimal(page["billing"]["charge"] or "0") ``` A client-side cap is a convenience, not a guarantee (several runs share the key, a crash loses the total). **Ask the operator to set a monthly spend cap on the key** (Mission Control → Customers → Keys): Cormorant enforces it on every request and batch item (`429 key_budget_exhausted`), shows it in `X-Cormorant-Key-Budget-Remaining` and `spend_budget`, and can alert you at 50 / 80 / 100 % (`POST /v1/alerts`). Use one key per workload so each gets its own cap. ## Budgets Each project has a **monthly budget** in USD for third-party providers (calendar month, UTC), set by the operator. Before every provider call Cormorant reserves the estimated cost atomically, so concurrent requests can never overspend; after the call the reservation is settled to the actual cost. When the budget is exhausted, premium routes and the Brave fallback are skipped (`budget_exhausted` in `attempts` or in a search's `error`) while the standard routes keep working. ## Usage: GET /v1/usage Usage of the caller's scope and what it costs. Inclusive dates (UTC), default: current month to date, at most 400 days. Read from aggregates refreshed every 5 minutes, never from the request log, so it answers in the same time at any volume. **Who sees what.** A standard key (`cmr_…`) sees only its own requests. An account admin key (`cma_…`) sees every key and project of its customer — revoked keys included, and usage recorded before per-key attribution as `"key": null`. A project without a customer is its own account. Nothing ever crosses a customer boundary; a key or project you cannot see is `404`, like one that does not exist. **Parameters** | Parameter | Meaning | |---|---| | `from`, `to` | Range of UTC days (inclusive). | | `group_by` | Comma-separated, combinable: `key`, `project`, `category`, `day`, `hour` (hourly data: last 35 days). | | `key` | Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). | | `project` | Only these projects (comma-separated names). | | `category` | Only these billing categories: `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. | | `format` | `csv` (same as `Accept: text/csv`). | ```json { "project": "acme-crawler", "key": "cma_XyZaBc", "role": "account_admin", "scope": {"customer": "Acme", "projects": ["acme-crawler", "acme-search"], "keys": "all"}, "from": "2026-10-01", "to": "2026-10-19", "as_of": "2026-10-19T11:05:02+00:00", "final": false, "currency": "USD", "totals": {"requests": 9120, "succeeded": 8800, "failed": 320, "success_rate": 0.9649, "cached": 1200, "cache_hit_rate": 0.1316, "errors": {"http_404": 210, "all_tiers_failed": 110}, "latency_ms_p50": 900, "latency_ms_p95": 7400, "bytes": 812345678, "premium_requests": 90, "premium_share": 0.0113, "charge": "15.96", "billable": {"fetch_own": {"quantity": "8710", "included": "1000", "billable": "7710", "amount": "15.42"}, "fetch_premium": {"quantity": "90", "included": "0", "billable": "90", "amount": "0.36"}, "premium_gb": {"quantity": null, "included": null, "billable": null, "amount": "0.18"}}}, "search": {"requests": 1200, "cached": 300, "cache_hit_rate": 0.25, "succeeded": 1198, "success_rate": 0.998, "bytes": 0, "premium_bytes": 0, "latency_ms_p50": 1100, "latency_ms_p95": 3900, "fallbacks": 60, "fallback_rate": 0.0667, "providers": {"searxng": {"requests": 840, ...}, "brave": {...}}}, "fetch": {"requests": 7920, "cached": 900, "tiers": {"direct": {...}, "proxy": {...}}, "errors": {"http_404": 210}, "by_outcome": {"succeeded": {"requests": 7600, "bytes": 800000000, "premium_bytes": 8800000, "premium_requests": 80}, "failed": {"requests": 320, "bytes": 0, "premium_bytes": 323456, "premium_requests": 10}}, ...}, "company_profile": {}, // extended company profiles (kind company_profile), same fields; {} when none "group_by": ["key"], "groups": [{"key": "cmr_AbCdEf", "key_label": "crawler", "key_role": "standard", "key_status": "active", "project": "acme-crawler", "requests": 9000, ..., "charge": "15.70", "billable": {...}}], "keys": [{"prefix": "cmr_AbCdEf", "label": "crawler", "role": "standard", "project": "acme-crawler", "active": true, "requests": 9000, "cached": 1150, "succeeded": 8690, "search_requests": 1100, "fetch_requests": 7900, "company_requests": 0, "company_profile_requests": 0, "premium_bytes": 9123456, "charge": "15.70"}], "account": [{"period": "2026-10", "from": "2026-10-01", "to": "2026-10-19", "currency": "USD", "complimentary": true, "usage_charges": "15.96", "account_level_amount": "-15.96", "account_lines": [{"kind": "complimentary", "description": "Complimentary plan: 100 % discount", "amount": "-15.96"}], "list_total": "15.96", "net_total": "0.00"}] } ``` - **Metrics** (in `totals` and every group): `requests` (cache hits included), `succeeded`, `failed`, `cached`, `errors` (failures by error code, top 50), `latency_ms_p50`/`latency_ms_p95` of uncached requests (estimated from a latency histogram with bounds 100, 250, 500 ms, 1, 2, 5, 10, 20, 30, 60 and 120 s — exact to the bucket), `bytes`, `premium_requests` and `premium_share` (fetches that needed the premium routes, of all uncached fetches). - **Charges** (projects with a customer): `charge` is what the group costs at your plan's prices in your invoice currency, as a decimal string; `billable` lists each price-plan meter with the measured `quantity`, the `included` units it used (its pro-rata share of the plan's included units of the month), `billable` and `amount`. Meters billed on provider cost show only the amount. The period is rated exactly like the invoice and each usage line is split to the cent, first per key, then per day and category: **any grouping adds up to the same total**, and the keys add up to the usage lines of the estimate (`GET /v1/account`) or invoice. Base fee, minimum, contract discount and the complimentary credit are account-level lines: account admin keys see them per period in `account`. Complimentary accounts see list-price charges and the 100 % `complimentary` line. - **Provider cost** — `cost_usd` and the project `budget` (Cormorant's spend with its search and proxy providers) appear only for projects without an external customer (internal use). Customers see what they pay, never what Cormorant pays. - **Freshness**: `as_of` is the time up to which every request is included; `final: true` (top level, and per group with `group_by=day`) means the days are closed and will not change (a day becomes final two hours after it ends). - `search` and `fetch` keep the per-kind blocks of earlier versions (now for the caller's scope), with `providers`/`tiers` breakdowns. With `key` in `group_by`, `keys` keeps the earlier per-key list as well. - **Latency components**: every per-kind block has `latency_components` (`queue_ms`, `politeness_wait_ms`, `upstream_ms`, each `{p50, p95}` of uncached requests), and each entry of `tiers`/`providers` adds `latency_ms_p50/p95` and `<component>_p50/_p95`, so a slow tier can be told apart from a busy domain or a queue backlog. Estimated from histograms like the other percentiles; requests from before these were recorded are not included. - `by_outcome` (in every per-kind block) splits the same requests into `succeeded` and `failed`, each with `requests`, `bytes`, `premium_bytes`, `premium_requests` (fetch) and `cost_usd` (provider cost, internal projects only): `fetch.by_outcome.failed.premium_bytes` is what blocked pages cost before the fetch gave up. Whether failed requests are billed depends on your plan (`billable`). **CSV.** `Accept: text/csv` (or `format=csv`) returns one row per group — or one row of totals without `group_by` — with the dimensions, the metrics, `charge`, the meters as `<meter>_quantity`, `<meter>_included`, `<meter>_billable`, `<meter>_amount` and `errors` as `code=n;code=n`. ```sh curl -sS "$CORMORANT_URL/v1/usage?from=2026-10-01&to=2026-10-31&group_by=key,day" \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Accept: text/csv" -o usage.csv ``` ```sh curl -sS "https://api.cormorant.ch/v1/usage?from=2026-10-01&to=2026-10-19" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` ```python usage = client.get("/v1/usage", params={"from": "2026-10-01", "to": "2026-10-19"}).json() if usage["currency"]: # a customer account: charges at your plan price print(f"charges {usage['totals']['charge']} {usage['currency']} (as of {usage['as_of']})") else: # a project without a customer: provider spend and budget print(f"spent ${usage['cost_usd']:.2f}, budget left ${usage['budget']['remaining_usd']:.2f}") print("fetch cache hit rate:", usage["fetch"].get("cache_hit_rate")) ``` ```ts const usage = await ( await fetch(`${BASE}/v1/usage?from=2026-10-01&to=2026-10-19`, { headers }) ).json(); console.log(usage.currency ? `${usage.totals.charge} ${usage.currency}` : `$${usage.cost_usd}`); ``` ## Your account, plan and invoices Two different things are measured: - **Provider spend** (`cost_usd` on results, budgets) — what Cormorant spent on its search and proxy providers because of your requests. `GET /v1/usage` shows it only for projects without a customer account. - **What you are billed** — a project can belong to a **customer** account with a price plan. Every served request falls into a billing category (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`), is metered hourly and priced monthly by the plan (included units, volume tiers, pass-through of provider cost). Complimentary accounts get the same invoice at list price with a 100 % discount line, so you can see what your usage is worth. Keys belong to a project; a project may belong to a **customer** (the billed account). These endpoints show the key's own project usage plus the plan, current-period estimate and issued invoices of that customer — never another customer's data, never drafts. Money is a decimal **string** in the invoice currency (USD, CHF or EUR). Usage is aggregated every 5 minutes (`usage_refreshed_at`). See [billing.md](/docs/billing.md) for pricing rules. ### GET /v1/account ```json { "project": {"id": 7, "name": "acme-crawler"}, "period": {"period": "2026-10", "from": "2026-10-01", "to": "2026-10-31"}, "usage_refreshed_at": "2026-10-19T11:00:04+00:00", "project_usage": {"fetch_own": {"requests": 9120, "succeeded": 8800, "bytes": 812345678, "bytes_ok": 800000000, "premium_bytes": 0, "premium_bytes_ok": 0, "cost_usd": "0", "cost_ok_usd": "0"}, "search": {...}}, // cost_usd / cost_ok_usd: null for projects of an external customer "customer": {"name": "Acme", "company_name": "Acme Research AG", "currency": "USD", "status": "active", "complimentary": true}, "customer_usage": {...}, "plan": {"id": 1, "code": "standard-usd", "version": 1, "name": "Standard (USD)", "currency": "USD", "complimentary": false}, "estimate": {"period": "2026-10", "currency": "USD", "lines": [{"kind": "usage", "component": "fetch_own", "description": "Page fetches (1,000 included)", "quantity": "8800", "unit": "fetch", "unit_price": null, "amount": "15.60", "detail": {"included": "1000", "billable": "7800", "tiers": [{"from": "0", "to": "100000", "quantity": "7800", "unit_price": "0.002", "amount": "15.60"}]}}, {"kind": "complimentary", "description": "Complimentary plan: 100 % discount", "amount": "-15.60", ...}], "subtotal": "15.60", "discount_total": "15.60", "net_total": "0.00", "vat_rate": "8.1", "vat_amount": "0.00", "total": "0.00", "complimentary": true, "list_total": "15.60"} } ``` `keys` breaks the period down per API key. An account admin key (`cma_…`) sees every key of the customer's projects (or of this project without a customer); an additional key (`cmr_…`) sees only itself. Keys appear by `prefix`, `label` and `role`, never ids or secrets, with its usage by category and `amount`, its share of the usage charges at list price. Each usage line of the estimate is split between keys in proportion to their share of that meter (to the cent), so the amounts add up to the usage lines; base fee, minimum, discounts and the complimentary credit stay on the account. A row with `"prefix": null` is usage recorded before per-key attribution. ```json "keys": [{"prefix": "cma_XyZaBc", "label": "billing", "role": "account_admin", "project": "acme-crawler", "active": true, "requests": 9120, "succeeded": 8800, "usage": {"fetch_own": {"requests": 9120, ...}}, "amount": "15.60"}] ``` Projects without a customer get `"customer": null, "plan": null, "estimate": null` (and `amount: null` per key). Categories: `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. ### GET /v1/account/invoices `{"invoices": [{"number": "CORM-2026-0001", "status": "issued|paid|void", "period": "2026-09", "period_start", "period_end", "currency", "plan_label", "complimentary", "subtotal", "discount_total", "net_total", "vat_rate", "vat_amount", "total", "vat_note", "issued_at", "due_at", "paid_at", "voided_at", "pdf_url": "/v1/account/invoices/CORM-2026-0001/pdf"}]}` ### GET /v1/account/invoices/{number} and /pdf The invoice with its lines (and tier breakdown), or the PDF. Another customer's number (or a draft) is `404`. ```sh curl -sS https://api.cormorant.ch/v1/account \ -H "Authorization: Bearer $CORMORANT_API_KEY" curl -sS https://api.cormorant.ch/v1/account/invoices/CORM-2026-0001/pdf \ -H "Authorization: Bearer $CORMORANT_API_KEY" -o CORM-2026-0001.pdf ``` ```python account = client.get("/v1/account").json() estimate = account["estimate"] if estimate: print(f"{estimate['period']}: {estimate['total']} {estimate['currency']}") for invoice in client.get("/v1/account/invoices").json()["invoices"]: print(invoice["number"], invoice["status"], invoice["total"], invoice["currency"]) ``` ```ts const account = await (await fetch(`${BASE}/v1/account`, { headers })).json(); if (account.estimate) console.log(account.estimate.period, account.estimate.total); const { invoices } = await (await fetch(`${BASE}/v1/account/invoices`, { headers })).json(); for (const invoice of invoices) console.log(invoice.number, invoice.status, invoice.total); ``` Amounts are decimal **strings** (`"15.60"`): parse them with a decimal type, not a float. If the account is suspended, every `/v1` request answers `403 customer_suspended`. ## Attributing spend Provider spend is attributed only to the project whose request made the provider call; the caches are shared, so a page another project fetched through a premium route is a cache hit for you. Within an account, every request is also attributed to the **API key** that made it (batch items to the key that created the job): `GET /v1/usage?group_by=key` splits usage and charges per key, and `keys` in `GET /v1/account` shows each key's usage and share of the month's charges. With the account admin key (`cma_…`) you see every key of the account; an additional key (`cmr_…`) sees only itself. Give each client or environment its own key (ask the operator for a labelled key) to see who uses what. ```python for group in client.get("/v1/usage", params={"group_by": "key"}).json()["groups"]: print(group["key"], group["key_role"], group["requests"], group.get("charge")) ``` Set `project_ref` on requests to tag usage with your own identifiers (a customer, a company id, a run). <!-- source: https://www.cormorant.ch/docs/usage.md --> # Usage, spending and quotas Everything your API keys did, what it costs you and how much of each limit is left — per key, project, category, day or hour, in JSON or CSV. The figures come from the same metering and rating as your invoice, so they add up to it to the cent. ## Who sees what Your key's prefix tells you what it can see: | Key | Sees | |---|---| | `cma_…` account admin key | every key (revoked ones included) and project of your account, usage recorded before per-key attribution (`key: null`), account-level lines, plan allowances | | `cmr_…` standard key | only its own usage, requests, limits and alerts | Asking about a key or project you cannot see answers `404`, exactly like one that does not exist. Every response carries `X-Cormorant-Key-Role`, and usage responses say what they cover in `scope`. ## The endpoints | Endpoint | Use it for | |---|---| | `GET /v1/usage` | Totals and groups (`group_by=key,project,category,day,hour`) with charges | | `GET /v1/usage/timeseries` | The same metrics per hour or day, zero-filled, for charts | | `GET /v1/usage/domains` | Your busiest fetched domains (90 days) | | `GET /v1/requests` | Every request, newest first, with cursor paging (90 days) | | `GET /v1/quota` | Every limit and allowance: used, left, projected, status | | `GET /v1/keys`, `GET /v1/keys/self` | Your keys, their roles and limits | | `POST /v1/alerts` | A signed webhook at 50 / 80 / 100 % of a limit | Details of every field: [API contract](/docs/api.md). ## Charges, not costs `charge` is what you pay: your plan's prices (with your contract rates) in your invoice currency, as a decimal string. `billable` shows, per price-plan meter, the measured `quantity`, the `included` units it used and the `amount`. Cormorant's own provider costs are not part of customer figures. Base fee, minimum, contract discount and the complimentary credit belong to the account, not to a key: an account admin key finds them per month under `account`. On a complimentary plan you still see list-price charges, followed by the 100 % `complimentary` line, so you know what your usage is worth. ## Fresh and final Usage is aggregated every 5 minutes. `as_of` is the moment up to which every request is included; a day is `final` two hours after it ended, and its figures will not change afterwards. The request ledger is real time — a request's `charge` appears once it has been aggregated. ## Examples Charges per key this month, then per key and day as CSV: ```sh curl -sS "https://api.cormorant.ch/v1/usage?group_by=key" \ -H "Authorization: Bearer $CORMORANT_API_KEY" curl -sS "https://api.cormorant.ch/v1/usage?from=2026-10-01&to=2026-10-31&group_by=key,day" \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Accept: text/csv" -o usage-october.csv ``` ```python import csv import io import os import httpx client = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, ) usage = client.get("/v1/usage", params={"group_by": "key"}).json() print(f"{usage['totals'].get('charge')} {usage['currency']} as of {usage['as_of']}") for group in usage["groups"]: print(group["key"], group["key_role"], group["requests"], group.get("charge")) # The same as CSV, e.g. for a spreadsheet or pandas.read_csv. text = client.get( "/v1/usage", params={"group_by": "project,category", "from": "2026-10-01", "to": "2026-10-31"}, headers={"Accept": "text/csv"}, ).text rows = list(csv.DictReader(io.StringIO(text))) ``` Every failed request of a key yesterday, page by page: ```python params = {"key": "cmr_AbCdEf", "outcome": "failed", "from": "2026-10-18", "to": "2026-10-18"} cursor = None while True: page = client.get("/v1/requests", params={**params, "cursor": cursor} if cursor else params) body = page.json() for row in body["requests"]: print(row["time"], row["domain"], row["error"], row["request_id"]) cursor = body["next_cursor"] if not cursor: break ``` Where do I stand? Each limit has `limit`, `used`, `remaining`, `projected_end_of_period` and `status` (`ok`, `warning` from 80 %, `exhausted`): ```python quota = client.get("/v1/quota").json() for key in quota["keys"]: for limit in key["limits"]: print(key["key"], limit["name"], limit["used"], "/", limit["limit"], limit["status"]) for allowance in (quota.get("customer") or {}).get("allowances", []): print(allowance["name"], allowance["used"], "of", allowance["limit"], "included") ``` ## Limits and what happens at them The operator can give a key a monthly spend cap, monthly request quotas (`search`, `fetch`, `fetch_premium`, `company`), a rate limit and allowed categories — see [Limits & politeness](/docs/limits.md#per-key-limits). Every response tells you how much is left: ``` RateLimit-Policy: "project";q=200;w=1, "key";q=10;w=1 RateLimit: "project";r=187;t=1, "key";r=9;t=1 X-Cormorant-Key-Budget-Remaining: 12.40 USD X-Cormorant-Quota-Remaining: fetch=18766, search=900 ``` At a limit, requests answer `429 key_quota_exhausted` or `429 key_budget_exhausted` with `Retry-After` until the month resets — do not retry those in a loop. ### Spend against the caps: `spend_budget` `GET /v1/quota` and `GET /v1/usage` both carry `spend_budget`: this month's spend per key (the calling key, or every active key for an account admin key) and, for account admin keys, for the whole customer: ```json "spend_budget": { "period": "2026-10", "currency": "CHF", "basis": "charges", "resets_at": "2026-11-01T00:00:00+00:00", "keys": [{ "key": "cmr_AbCdEf", "key_label": "research", "project": "chipiron", "is_self": true, "currency": "CHF", "cap": null, "spent": "12.34", "pending": "0.04", "remaining": null, "status": "no_cap", "by_kind": {"search": "1.20", "fetch": "11.10", "sec": "0.00", "company_profile": "0.00", "company": "0.00"} }], "customer": {"customer": "Chipiron", "currency": "CHF", "cap": null, "spent": "12.30", "pending": "0.00", "remaining": null, "status": "no_cap", "by_kind": {"…": "…"}, "account_charges": "0.00", "net_estimate": "12.30"} } ``` `cap` is the key's monthly spend cap; **`null` means no cap is set** (status `no_cap`), it is never left out. Customers have no account-wide cap: caps are per key. `spent` is your charges at plan price as of the last refresh plus the per-request estimate since (`pending`); `by_kind` splits the refreshed part by request kind. `basis` is `provider_cost` (USD) only for projects without a customer. How to cap a run on the client side: [Costs & billing](/docs/billing.md#capping-the-cost-of-a-run). ## Alerts Register a webhook and get an event when a key, project or your account crosses 50, 80 or 100 % of a limit or allowance (once per threshold and month): ```sh curl -sS https://api.cormorant.ch/v1/alerts \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H 'Content-Type: application/json' \ -d '{"url": "https://hooks.example.com/cormorant", "thresholds": [80, 100]}' ``` Store the `secret` from the answer: it is shown once. Verify each delivery's `X-Cormorant-Signature` (`t=<unix>,v1=<hex>`, the HMAC-SHA256 of `"<t>.<raw body>"`) before trusting it — the [API contract](/docs/api.md) has a ready-made function. The same events are listed in `GET /v1/quota` (`events`). ## From an agent The MCP server's `cormorant_usage` tool answers "what did I use and what is left?" in a compact form — see [MCP server](/docs/mcp.md). <!-- source: https://www.cormorant.ch/docs/companies.md --> # Company data Cormorant keeps a **company register**: one structured record per company anyone has asked about, with every value attributed to the source that published it, when it was seen and the raw document it came from. Records are versioned, never thrown away, refreshed when they are asked for again, and kept current by official change feeds. Data comes from official registers, GLEIF and other open, legitimate sources (see [Sources](/docs/companies-sources.md)). ## How a request is answered 1. You name a company by its Cormorant id (`cmp_…`) or an identifier (`ch_uid:CHE-101.329.561`, `lei:…`, `us_cik:320193`, `uk_crn:…`, `isin:CH0038863350`). 2. Cormorant answers from the register. For each source that knows the company, data older than the source's **freshness class** (registry core 24 h, people 7 days, …) or your `max_age` is re-fetched first. 3. A company that is not yet in the register is fetched from the sources that know the identifier's scheme and added. 4. The answer says per source what happened (`refresh`): `refreshed`, `fresh`, `failed` (the register still answers), `not_found`, `pending` (slower than 2 s, or the source's canaries fail: it finishes in the background and the next request has the data) or `skipped` (the source is backing off after failures). Stale sources are refreshed at the same time, and the answer never waits more than about 2 s for them: you get the stored record with its `freshness` at once. An **ISIN** names the company that issued the security: Cormorant asks GLEIF's ISIN-to-LEI mapping, fetches the issuer by its LEI and keeps the ISIN on the record, so the next request by that ISIN is answered by the register. An identifier no source knows is remembered as unknown for a while (ISINs 24 hours, other schemes 1 hour; `refresh=true` asks again), so repeating it in bulk runs costs nothing at the sources. Every call is metered as one `company` request (see [Costs & billing](/docs/billing.md)); answers from the register without any refresh count as `cached`. ## Examples Find a company, then read its record: ```sh curl -s "https://api.cormorant.ch/v1/companies/search?q=UBS%20Group&country=CH" \ -H "Authorization: Bearer $CORMORANT_API_KEY" curl -s "https://api.cormorant.ch/v1/companies/lei:549300SZJ9VS8SGXAN81?include=ownership" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` ```python import os, httpx api = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, ) hits = api.get("/v1/companies/search", params={"q": "UBS Group", "country": "CH"}).json() best = hits["results"][0] record = api.get(f"/v1/companies/{best['ref']}", params={"include": "people,ownership"}).json() print(record["names"]["legal"], record["status"]["value"]) for field, p in record["provenance"].items(): print(field, "from", p["source"], "seen", p["observed_at"]) ``` Match a name to a company (entity resolution): ```sh curl -s -X POST https://api.cormorant.ch/v1/companies/resolve \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \ -d '{"name": "UBS AG", "country": "CH", "city": "Zürich"}' ``` `match` is set only when one candidate is a confident match; otherwise read `candidates` and their `reasons`. A match is put into the register right away, so it comes back with its `cmp_` id and `in_register: true`: read the record with `GET /v1/companies/{match.id}`, no second lookup by `ref` needed. Pass `"persist": false` to leave the register untouched (the match may then be a live hit with `id: null` and a `scheme:value` `ref`). **How fast.** A company already in the register is answered from it: `resolve` with a confident register match and `search` with a near-exact register hit do not ask any source (milliseconds). Otherwise the live sources are asked at the same time and waited for at most 2.5 s; a source that is slower, backing off after failures, or failing its canaries does not hold the answer up. `live_sources` says per source what happened (`ok`, `failed`, `timeout`, `skipped`) and how long it took, so you can tell a partial answer from a complete one. After `resolve` stored a new company, the other sources' data about it is fetched in the background. Need fresher data than the freshness class? Pass `max_age` (seconds) or `refresh=true`: ```sh curl -s "https://api.cormorant.ch/v1/companies/ch_uid:CHE-101.329.561?max_age=3600" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` What changed and when — every value each source ever asserted: ```sh curl -s "https://api.cormorant.ch/v1/companies/ch_uid:CHE-101.329.561/history?field=names.legal" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` Many companies at once: a batch job (JSON or CSV in, JSON or CSV out): ```sh curl -s -X POST https://api.cormorant.ch/v1/companies/batch \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \ -d '{"action": "get", "items": [{"ref": "lei:BFM8T61CT2L1QCEMIK50"}, {"ref": "ch_uid:CHE-116.281.710"}], "idempotency_key": "crm-sync-2026-10-20"}' # then: GET /v1/jobs/{job_id}?include=items or GET /v1/companies/batch/{job_id}.csv ``` ## Bulk: thousands of companies For a list of 10,000 companies, use batch jobs, not 10,000 synchronous calls: 1. **Resolve once, with what you know best.** `POST /v1/companies/batch` with `action: "resolve"` (or `/batch/csv` with `name,country,city,isin,identifiers` columns). An identifier (`isin`, `lei`, `ch_uid`, …) is matched exactly and costs at most one source call; a name is scored over the register and the sources of its `country` (always give `country`). Items wait up to 12 s for slow sources (completeness before speed) and confident matches are stored, so every result has `company_id`. 2. **Keep the `cmp_` ids.** Later runs use `action: "get"` with `ref` = the `cmp_` id: answered from the register, refreshed only when a source's data is older than its freshness class (or your `max_age`). 3. **Follow changes** with `GET /v1/companies/changes` or a watch instead of re-reading every record. The sources' own limits set the pace of the first run for companies not yet in the register (GLEIF: 60 requests per minute; the Swiss UID register about 20), so a first run of 10,000 new companies takes hours; repeated runs are answered from the register. Identifiers no source knows are remembered as unknown for a while and answered at once when repeated. ## Read more - [The company/v1 record](/docs/companies-model.md): every field, claims and provenance, identifier schemes, legal forms, activity codes. - [Sources](/docs/companies-sources.md): which sources, which countries, how fresh, precedence when they disagree, licences, health. - [Change feeds and watches](/docs/companies-changes.md): follow changes without polling records. - The full contract: [Company data in the API reference](/docs/api.md#company-data). <!-- source: https://www.cormorant.ch/docs/companies-model.md --> # The company/v1 record `GET /v1/schemas/company` serves the JSON Schema (2020-12) of the record; this page explains it. The OpenAPI document describes every field too. ## Claims and provenance Every value is a **claim** of one source: the value, the source, `observed_at` (first seen), `confirmed_at` (last time the source still said so), `valid_from` / `valid_to`, `confidence` and the raw document (`document_ref` with `sha256` and `body_url`). Claims are never overwritten: when a source reports a different value, the old claim is closed (`valid_to`) and a new one opened. So the record can show what changed and when (`GET /v1/companies/{id}/history`). For each field the record shows the resolved value: the claim of the source with the highest precedence (official register > official gazette > GLEIF > regulatory filings > crosswalks > commercial sources), then the field's preferred sources within that class, then the more recent one. The legal name is stricter: it is the official national register's primary entry in its official form and language (in Switzerland Zefix, then the UID register), ties go by source name rather than by time, so refreshing GLEIF after the register (or the other way round) never changes it. Other sources' forms are in `names.alternates` (`{name, source}`), language versions in `names.translations` (with `source`). `provenance` names the claim behind each field and flags `conflict` when another source currently says something else; `quality.conflicts` lists those values. `include=claims` returns every current claim; `sources=` restricts the record to some sources. ## Fields | Field | Content | |---|---| | `id` | `cmp_<ulid>`, stable; merged records redirect (`redirected_from`) | | `identifiers[]` | `{scheme, value, issuer, status, parts, sources}` in canonical form | | `names` | `legal`, `previous[]`, `trading[]`, `translations[]` (each `{name, lang, valid_from, valid_to}`; translations also `source`), `alternates[]` (other sources' forms of the legal name: `{name, source}`) | | `legal_form` | `{local_code, local_name, elf_code, category}`: ISO 20275 code and one of `corporation`, `partnership`, `sole_trader`, `cooperative`, `foundation`, `association`, `public_body`, `branch`, `other` | | `status` | `{value, local_value, since, reason}`: `active`, `inactive`, `in_liquidation`, `dissolved`, `merged`, `struck_off`, `unknown` | | `jurisdiction` | `{country, subdivision, register: {authority, court, entity_id}}` | | `addresses[]` | `{type: registered/business/postal/branch, lines, street, house_number, postal_code, city, municipality_code, region, country, geo}` | | `dates` | `incorporated`, `registered`, `deregistered`, `fiscal_year_end` | | `purpose` | Text per language | | `activities[]` | `{scheme: nace/noga/wz/sic/naics/isic, code, label, primary, nace}` | | `capital[]` | `{type: nominal/paid_in/authorised, amount, currency, since}` | | `size` | Employees and revenue, only from official or filing data | | `people[]` | Officers as the register publishes them: `{name, role, local_role, function, signing_authority, since, until, residence_city, residence_country, nationality, origin, person_id}` (`include=people`) | | `ownership[]` | `{type: shareholder/direct_parent/ultimate_parent/beneficial_owner, holder, share_pct, since}` (`include=ownership`) | | `relationships[]` | `{type: subsidiary/branch_of/head_office/merged_into/predecessor/successor/auditor/…, company, since, until}` | | `filings[]`, `financials[]` | Filed documents and official figures (`include=filings,financials`) | | `events[]` | Timeline from gazettes and registers (`include=events`) | | `securities[]`, `web` | Listings; website and domains | | `sources[]` | Raw documents (newest first) with `licence` and `parser_version` | | `freshness[]` | Per source and field group: `last_checked`, `last_changed`, `next_refresh_due`, `error` | | `quality` | `completeness` (share of core fields filled) and `conflicts` | | `provenance`, `claims` | See above | Companies referenced in `ownership` and `relationships` carry the identifiers the source gave; fetch them with `GET /v1/companies/scheme:value`. ## Identifier schemes Write identifiers as `scheme:value`; spacing, dots and case do not matter, check digits are verified, and the record shows the canonical form. | Scheme | Example | Check | |---|---|---| | `ch_uid` | `CHE-101.329.561` | eCH-0097 mod 11 | | `lei` | `BFM8T61CT2L1QCEMIK50` | ISO 17442 (ISO 7064 mod 97-10) | | `us_cik` | `320193` | leading zeros dropped | | `uk_crn` | `00445790`, `SC123456` | format | | `de_register` | `München:HRB:6684` (also `Amtsgericht München HRB 6684`) | court, type (HRA, HRB, GnR, PR, VR, GsR), number | | `fr_siren`, `fr_siret` | `552100554`, `55210055400013` | Luhn | | `no_orgnr`, `dk_cvr`, `fi_ytunnus`, `ee_regcode`, `cz_ico` | `923609016`, `24256790`, `0112038-9`, `10137319`, `45274649` | national check digits | | `pl_krs`, `be_kbo` | `0000028860`, `0403.170.701` | format, mod 97 | | `au_abn`, `au_acn`, `nz_nzbn` | `51824753556`, `004085616`, `9429040402307` | national check digits, GS1 | | `jp_corporate_number`, `br_cnpj`, `sg_uen`, `ca_corp` | `7000012050002`, `33000167000101` | national check digits (CNPJ also alphanumeric) | | `vat` | `DE129273398`, `CHE101329561` | country format; Swiss VAT numbers are UIDs | | `isin` | `CH0038863350` | ISO 6166 (Luhn over the letters-as-numbers form); the issuer is found through GLEIF's ISIN-to-LEI mapping | | `ticker`, `wikidata`, `ch_chid`, `ch_ehraid` | `XNAS:AAPL`, `Q312` | format | ## Legal forms and activity codes Legal forms map to ISO 20275 Entity Legal Form codes (GLEIF's list, every jurisdiction) with a normalised `category`. Activity codes keep the scheme the source uses; NOGA 2008 (Switzerland) and WZ 2008 (Germany) codes also name their NACE Rev. 2 class (`nace`). Labels are in English. <!-- source: https://www.cormorant.ch/docs/companies-sources.md --> # Company data sources `GET /v1/companies/sources` lists the sources with their countries, identifier schemes, capabilities, precedence, licence/terms URL, freshness classes and live health. Sources are open and legitimate: official open APIs, open data and public register pages within their terms and robots.txt. Two exceptions, decided by the operator: the SOGC's official public API and the cantonal commercial-register excerpts (with their canton lookup on LINDAS) are read although their hosts' robots.txt disallows everything (see [Switzerland](/docs/companies-switzerland.md#access)). ## Precedence When sources disagree, the higher class wins and the other values are kept as conflicts: official register > official gazette > GLEIF > regulatory filings (SEC) > crosswalks (Wikidata) > commercial sources. Each field declares its rule: within a class, preferred sources first, then for most fields the newer observation. The legal name (`names.legal`) is the national register's primary entry and never depends on refresh order: in Switzerland `zefix_rest` > `lindas_zefix` > `uid_register` > `ch_cantonal_excerpt`, then `sogc`, then `gleif`; remaining ties go by source name. Other sources' forms of the name are listed in `names.alternates` with their `source`. ## Freshness Each source declares how long its data counts as fresh per field group (`core`, `people`, `ownership`, `relationships`, `filings`, `financials`, `events`). A request for older data re-fetches it from that source first; `max_age` and `refresh=true` override it per request. Official change feeds update companies already in the register without your request. ## Sources | Source | Countries | Gives | Change feed | |---|---|---|---| | `lindas_zefix` (Zefix on LINDAS) | CH | name and translations, UID, CH-ID, EHRA-ID, legal form, seat municipality, address, purpose (active entities) | daily data | | `uid_register` (UID register, BFS) | CH | master data of every UID (also outside the commercial register), status, addresses, VAT status and number, LEI | – | | `zefix_rest` (Zefix Public REST) | CH | full register record: status and deletion, capital, former names, SOGC references, mergers, branches, audit firms; needs credentials | – | | `sogc` (Swiss Official Gazette of Commerce) | CH | events timeline (incl. officer changes, mergers, deletions), latest published state | daily publications | | `ch_cantonal_excerpt` (cantonal register excerpts) | CH | officers and signatories with role, signing authority, since/until | – | | `gleif` (GLEIF, LEI) | global | legal and other names, ELF legal form, status, jurisdiction and register id, legal and headquarters addresses, creation date, successor; direct and ultimate parents; the LEI ↔ national register id crosswalk; the issuer of an ISIN (ISIN-to-LEI mapping) | daily updates | Details per country: [Switzerland](/docs/companies-switzerland.md). More sources follow (SEC EDGAR, UK Companies House, France, Norway, …); the live list is always `GET /v1/companies/sources`. ## Health Every source is measured continuously (success rate, latency, parse failures, fields that suddenly come back empty) and checked every 15 minutes with known companies. The [status page](/status) shows one line per active source in the group "Company data"; a degraded source only means fresher data from it may be delayed — the register still answers. <!-- source: https://www.cormorant.ch/docs/companies-switzerland.md --> # Switzerland Swiss companies come from the federal commercial register (Zefix), the federal UID register, the Swiss Official Gazette of Commerce (SOGC) and the cantonal register excerpts. All five sources are official and open. Four of them need no account; the Zefix REST API needs credentials that the operator enters once (see [Activating Zefix REST](#activating-zefix-rest)). ## Sources | Source | Gives | Access | Default | |---|---|---|---| | `lindas_zefix` (Zefix on LINDAS) | legal name and its de/fr/it/en versions, UID, CH-ID, EHRA-ID, legal form, seat municipality (BFS number), registered address, purpose | SPARQL, no login; active entities only, updated daily | on | | `uid_register` (UID register, BFS) | master data of every UID, also outside the commercial register (administrations, sole traders, associations, VAT-only entities): names, legal form, UID and register status, legal and postal addresses, register entry and deletion dates, VAT status and number, LEI | SOAP, no login; about 20 requests per minute | on | | `zefix_rest` (Zefix Public REST API) | the full register record, deleted companies included: status and deletion date, nominal capital, former names, SOGC publications, head and branch offices, mergers, audit firms, link to the cantonal excerpt | HTTP basic credentials from the Federal Office of the Commercial Register | **off** until credentials are set | | `sogc` (Swiss Official Gazette of Commerce) | the events timeline: incorporation, name, seat, address, purpose and capital changes, officer changes (who joined or left, with dates), mergers, liquidation, deletion; the company as its latest publication states it; daily change feed | Amtsblattportal public API, no login; robots.txt not applied (official API, see [Access](#access)) | on | | `ch_cantonal_excerpt` (cantonal register excerpts) | officers and signatories: name, role, signing authority, since/until, residence, origin or nationality as published | public excerpt pages through the residential proxy tier; robots.txt not applied (official register, see [Access](#access)) | on | Where sources disagree, the register (Zefix, UID register, cantonal excerpts) ranks above the gazette; every value keeps its source, and conflicts are shown (see [Company data sources](/docs/companies-sources.md)). ## Access How Cormorant reaches the two sources that need special handling. Both are shown on the source card in Mission Control (**Company data → Sources**, row *Access*). ### SOGC: robots.txt not applied (official API) amtsblattportal.ch serves `User-agent: *` / `Disallow: /` for the whole host, its public API (`/api/v1/publications`) included, so every request was refused as `blocked`. The API is documented for public use and the operator decided to use it regardless. The source therefore has the robots policy `official_api` ("robots.txt: not applied (official API)"): - only this connector's requests skip robots.txt, on both transport paths (plain GETs through the Fetcher and requests with headers or a body); - its rate bucket (`amtsblattportal`, 1 request/s, burst 2), the per-domain politeness, the SSRF checks and the storage of the exact response bodies still apply; - every other fetch, including `/v1/fetch` of the same host, still obeys robots.txt (`RESPECT_ROBOTS`). The policy is reserved for documented official APIs the operator has decided to use; web pages never get it. The API negotiates its representation: a browser's `Accept` header gets XML (`application/xhtml+xml`), so the source asks for `Accept: application/json`. A response of another type fails as `parse` with what came back ("expected application/json, got …"), and a cached copy of another type is never used. ### Cantonal excerpts: through the premium routes The cantonal register portals (e.g. prestations.vd.ch, zh.chregister.ch) answer HTTP 403 to datacenter addresses, robots.txt included. Excerpt pages are therefore fetched only through the premium routes: - the residential proxy first, in a sticky session per portal host (renewed every 25 requests or 10 minutes, requests on one exit at least 1.5 s apart), with the headers of a Chrome navigation; - when the proxy is refused (403, 429, a challenge), the request is retried once on a new exit, then the rendering tier answers; refusals from 3 different exits within 5 minutes skip the proxy for that host only, for 5 minutes, doubling while refusals continue (at most an hour); - never Cormorant's own egress for excerpt pages. The chregister.ch portals (ZH, BE, LU, ZG, BS, TI) show the excerpt in a panel that the page loads after itself (a JSF request). The source does the same: it requests the page (never from the cache), then loads the panel with a POST in the same residential-proxy session, and reads the panel. Both requests are metered. The portals are asked in German, whatever the canton's language: the German panel is the one verified in production (the English one comes back empty). The rendering tier cannot send that POST, so these cantons need the residential proxy; without it the source answers `not_configured`. VD and GE have their own portals and are asked in French. The canton (which portal to ask) comes from what the register already knows about the company (its seat canton from Zefix or the UID register), else from the source option `cantons`. Only for a company the register does not know yet is it looked up on LINDAS (one SPARQL query from our own egress). The canary (ABB Ltd, canton of Zurich) carries its canton and checks the excerpt route itself. **robots.txt: not applied (official register).** Every `*.chregister.ch` portal and lindas.admin.ch serve `User-agent: *` / `Disallow: /`. These are the official public commercial registers, and the operator decided that this source reads them regardless: it has the robots policy `official_register` ("robots.txt: not applied (official register)" on the source card). Only this source's own requests (excerpt pages and the LINDAS canton lookup) skip robots.txt; its rate bucket (0.2 requests/s), the per-domain politeness, the SSRF checks, the stored exact bodies, the premium route and the billing below are unchanged, and every other fetch of these hosts, `/v1/fetch` included, still obeys robots.txt. **Cost.** Premium-route usage of the source is company-data pipeline cost, not a customer's fetch: its premium attempts are reserved against the internal project `company-data` (`COMPANY_DATA_PROJECT`, created on first use under the internal customer with a monthly budget of `COMPANY_DATA_MONTHLY_BUDGET_USD`, 20 USD; change it in Mission Control). Each excerpt fetch that reached the premium routes is one `request_log` row on that project with kind `company` (billing category `company`, `reason = source:ch_cantonal_excerpt`), its premium bytes and cost, so it shows with the company traffic and on the Costs page and never counts towards a customer's premium fetches. Your own company requests are metered as before. Errors: `not_configured` when the premium routes or the `company-data` project is not available, `budget_exhausted` when that project's budget is used up, `blocked` for a refused destination (SSRF checks). The message says how; a refused page lists every attempt ("all_tiers_failed [proxy 403 http_403, proxy_render 403 http_403]"). The same text is shown with failing canary checks and **Test connection** in Mission Control, and logged by the worker. ## Identifiers | Scheme | Example | Notes | |---|---|---| | `ch_uid` | `CHE-101.049.653` | Enterprise identification number; the check digit is verified. `CHE101049653`, `CHE-101.049.653 MWST` and `che 101 049 653` are accepted. | | `ch_chid` | `CH02030216152` | The former 11-digit commercial-register number (CH-ID); `CH-020.3.021.615-2` is accepted. | | `ch_ehraid` | `460530` | Internal id of the federal register office, one per register entry. | | `vat` | `CHE101049653` | Swiss VAT numbers are UIDs. A company in a VAT group is taxed under the group's number; that number is not added to the members, so they stay separate records. | A company with two head offices (Nestlé AG in Cham and Nestlé S.A. in Vevey) has one UID and two register entries: the record carries both EHRA-IDs and CH-IDs, the entry with the lower EHRA-ID gives the name, and the second seat appears as an address of type `other`. ## Fields - **Legal name** (`names.legal`): the commercial register's primary entry, as Zefix REST, Zefix on LINDAS, the UID register or the cantonal excerpt (in this order) publish it; the gazette, GLEIF and other sources never replace it, whatever order the sources are refreshed in. Swiss firms often have equal-ranking versions in several languages (Nestlé AG / Nestlé S.A. / Nestlé Ltd.): the others are in `names.translations`, and other sources' forms (GLEIF's `NESTLÉ S.A.`) in `names.alternates`, each with its `source`. - **Legal form**: the eCH-0097 code as `local_code` (`0106`), the name in the canton's language as `local_name` (Aktiengesellschaft, société anonyme, società anonima), the ISO 20275 ELF code and the category. Examples: AG / SA `MVII`, GmbH / Sàrl / Sagl `3EKS`, Genossenschaft `QSI2`, Verein `H781`, Stiftung `2JZ4`, Einzelunternehmen `AZA0`, Kollektivgesellschaft `MRSY`, Kommanditgesellschaft `CQMY`, Swiss branch `UNA9`, branch of a foreign company `JB25`, federal administration `2XJA`. - **Status**: `active`; `in_liquidation` (status `BEING_CANCELLED`, or "in Liquidation / en liquidation / in liquidazione" in the name); `dissolved` after deletion; `merged` when the company was taken over. `local_value` keeps the source's wording and `since` the deletion date. - **Addresses**: the registered address with the BFS municipality number, and the postal address (PO box) from the UID register. - **Purpose**: in the register's language (`de`, `fr` or `it`). - **Capital**: nominal capital, and paid-in capital from the SOGC. - **Relationships**: `auditor`, `merged_into`, `absorbed`, `branch`, `head_office` (Zefix REST). - **Activities**: NOGA codes, with their NACE class, when the UID register publishes them. - **People** (`include=people`): every officer and signatory the cantonal excerpt lists, current and former, with role (`chair`, `director`, `ceo`, `managing_director`, `partner`, `auditor`, `liquidator`, `secretary`, `signatory`, `member`) and the register's own wording, signing authority (`individual`, `joint_two`, `joint`, `procuration_individual`, `procuration_joint_two`, `none`), since/until from the register journal, residence and origin or nationality. Officer changes published in the SOGC are also in `events` with the people who joined or left. The register publishes no birth dates and Cormorant stores none. - **Events** (`include=events`): one entry per change and publication, with the publication link on shab.ch, the publication and journal numbers and dates, and details such as the old and new name or capital, the other company of a merger, or the people of an officer change. ```sh curl -s "https://api.cormorant.ch/v1/companies/ch_uid:CHE-105.909.036?include=people,events" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` ## Freshness | Field group | Refreshed after | Kept current by | |---|---|---| | core (name, legal form, status, address, purpose, capital) | 24 hours | the SOGC daily feed | | people | 7 days | officer changes in the SOGC feed | | events | 24 hours | the SOGC daily feed | | relationships | 24 hours | Zefix REST | The gazette publishes every register change on the day it takes effect for third parties. Cormorant reads its daily feed and applies the publications to companies already in the register, without your request. LINDAS and the UID register update daily. A company that is not in LINDAS has been deleted or is not in the commercial register: the UID register and Zefix REST still know it. ## Activating Zefix REST Zefix REST adds deleted companies, capital, former names, mergers, branches and audit firms. It stays inactive until credentials are set: 1. The operator requests credentials from the Federal Office of the Commercial Register (zefix@bj.admin.ch). The office issues them without charge. 2. In Mission Control, open **Company data → Sources → zefix_rest** and enter the **username** and **password**. Credentials are encrypted at rest, never shown again and never logged. 3. Switch the source **on**. The source card shows its health and canaries (Nestlé, UBS Group). A wrong password shows as error class `auth`. `GET /v1/companies/sources` then lists `zefix_rest` as active. ## Limits - The VD and GE portals have not yet been checked against production responses; ZH (chregister.ch) has. - Cantonal excerpts exist for the cantons in the source's URL table (ZH, BE, LU, ZG, BS, TI, VD, GE) and those the operator adds in its options; elsewhere officers come from the SOGC events only. - Cantonal excerpts need the premium routes (residential proxy or rendering). Without it the source answers `not_configured`. - Some French-speaking register offices write officer changes as prose, not as labelled lists. Cormorant reads the usual sentences ("n'est plus administrateur", "Nouvel organe de révision: …", "Signature individuelle est conférée à …"). Unusual wording yields an officer-change event without the people. - The UID register allows about 20 requests per minute. Bulk lookups take their time, and the register answers in the meantime. <!-- source: https://www.cormorant.ch/docs/companies-changes.md --> # Change feeds and watches ## The change feed `GET /v1/companies/changes` lists every change of the register, oldest first: which company and which top-level fields changed (`names`, `status`, `addresses`, …). Start with `since`, then keep the `next_cursor`: ```python cursor = None while True: params = {"cursor": cursor} if cursor else {"since": "2026-10-19T00:00:00Z"} page = api.get("/v1/companies/changes", params=params).json() for change in page["items"]: print(change["seq"], change["company_id"], change["kind"], change["fields"]) cursor = page["next_cursor"] if not page["items"]: break ``` `kind` is `created`, `updated`, `merged` (the record was merged into another one; its id redirects) or `split`. Read the record for the new values and `GET /v1/companies/{id}/history` for the old ones. ## Watches (webhooks) Register the companies you care about and an HTTPS endpoint: ```sh curl -s -X POST https://api.cormorant.ch/v1/companies/watches \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \ -d '{"companies": ["lei:BFM8T61CT2L1QCEMIK50", "ch_uid:CHE-116.281.710"], "url": "https://example.org/hooks/companies"}' ``` The answer has the watch `id` and its `secret` (shown only once). Every minute, changes of the watched companies are POSTed as one `company.changed` delivery (`event`, `id`, `watch_id`, `changes`). Verify `X-Cormorant-Signature: t=<unix>,v1=<hex>`, the HMAC-SHA256 of `"<t>.<raw body>"` under the secret, exactly as for [usage alerts](/docs/usage.md): ```python import hashlib, hmac, time def verify(secret: str, body: bytes, header: str, tolerance_s: int = 300) -> bool: parts = dict(p.split("=", 1) for p in header.split(",")) t = int(parts["t"]) expected = hmac.new(secret.encode(), f"{t}.".encode() + body, hashlib.sha256).hexdigest() return abs(time.time() - t) <= tolerance_s and hmac.compare_digest(expected, parts["v1"]) ``` Answer any 2xx to acknowledge. Failed deliveries are retried after 1, 5, 15, 60 and 180 minutes, then hourly, and no change is skipped; use the delivery `id` as idempotency key. A watched company merged into another is followed automatically. `GET /v1/companies/watches` lists your watches, `DELETE /v1/companies/watches/{id}` removes one (at most 20 per project, 1000 companies each). <!-- source: https://www.cormorant.ch/docs/agents.md --> # Using Cormorant from AI agents Cormorant was built to be called by programs that read the web on someone's behalf — including LLM agents. This page collects the patterns that keep an agent fast, cheap, polite and within its context window. ## Three ways in 1. **MCP** — run the [Cormorant MCP server](/docs/mcp.md) and your agent gets `cormorant_search`, `cormorant_fetch`, `cormorant_batch_fetch`, `cormorant_job_status`, `cormorant_usage` (what it used, what it costs, what is left) and the company tools `cormorant_company_search`, `cormorant_company_get` and `cormorant_company_resolve` (register data with sources, see [Company data](/docs/companies.md)) as tools. Best for Claude Desktop, Claude Code and other MCP clients. 2. **HTTP tools** — describe `POST /v1/search` and `POST /v1/fetch` as tools in your framework. The [OpenAPI document](/openapi.json) has every field described and stable `operationId`s (`search`, `fetch`, `batchFetch`, `getJob`, …) for tool generators. 3. **Docs for the model** — [`/llms.txt`](/llms.txt) indexes these guides as Markdown; [`/llms-full.txt`](/llms-full.txt) is everything in one file, including a compact endpoint reference generated from the OpenAPI document. Every guide is also available as Markdown at `/docs/<page>.md`. ## Rules of thumb | Situation | Do | |---|---| | Answering a question | Search, read the snippets, fetch the 1–3 best results. | | More than ~20 URLs or queries | Use a batch job, poll gently, page the results. | | Feeding text to the model | Send `include_html: false`; use `text`, trimmed to what you need. | | PDFs | Use `pdf_text_pages` to pick and cite pages instead of the whole `text`. | | Need the original file | Download `body_url` only when you must (spreadsheets, archives, exact quotes). | | Unattended runs | `allow_premium: false` unless paying is part of the task; set `timeout_s`. | | Errors | Retry only what is retryable; respect `Retry-After`; never hot-loop. | | Creating jobs | Deterministic `idempotency_key`s, so a retried call never duplicates work. | | Citing | Keep `final_url`, `retrieved_at` and `sha256` with every quote. | ## Pattern: research a question ```python import os import httpx client = httpx.Client( base_url="https://api.cormorant.ch", headers={"Authorization": f"Bearer {os.environ['CORMORANT_API_KEY']}"}, timeout=180, ) def research(question: str, pages: int = 3, chars: int = 6000) -> list[dict]: found = client.post("/v1/search", json={"query": question, "max_results": 8}).json() sources = [] for hit in found["results"][:pages]: page = client.post( "/v1/fetch", json={ "url": hit["url"], "include_html": False, "allow_premium": False, "timeout_s": 45, }, ).json() if page["error"] is None and page["text"]: sources.append( { "url": page["final_url"] or page["url"], "title": page["title"] or hit["title"], "retrieved_at": page["retrieved_at"], "text": page["text"][:chars], } ) return sources ``` ```ts const BASE = "https://api.cormorant.ch"; const headers = { Authorization: `Bearer ${process.env.CORMORANT_API_KEY}`, "Content-Type": "application/json", }; const post = async (path: string, body: unknown) => (await fetch(`${BASE}${path}`, { method: "POST", headers, body: JSON.stringify(body) })).json(); async function research(question: string, pages = 3, chars = 6000) { const found = await post("/v1/search", { query: question, max_results: 8 }); const sources = []; for (const hit of found.results.slice(0, pages)) { const page = await post("/v1/fetch", { url: hit.url, include_html: false, allow_premium: false, timeout_s: 45, }); if (page.error === null && page.text) { sources.push({ url: page.final_url ?? page.url, title: page.title ?? hit.title, retrieved_at: page.retrieved_at, text: page.text.slice(0, chars), }); } } return sources; } ``` ## Pattern: bulk enrichment When an agent plans to read many pages, it should submit them all at once and come back later rather than fetching one by one: ```python import hashlib import json import time urls = [...] # hundreds of URLs key = "agent-" + hashlib.sha256(json.dumps(sorted(urls)).encode()).hexdigest()[:32] job = client.post( "/v1/batch/fetch", json={"idempotency_key": key, "items": [{"url": u, "allow_premium": False} for u in urls]}, ).json() job_id = job["job_id"] while client.get(f"/v1/jobs/{job_id}").json()["status"] not in ("completed", "cancelled"): time.sleep(15) page = client.get(f"/v1/jobs/{job_id}", params={"include": "items", "limit": 1000}).json() texts = {item["input"]["url"]: (item["result"] or {}).get("text") for item in page["items"]} ``` Batches are rate-limited as one request, drain each domain at its polite rate, retry transient failures themselves, and keep results for 30 days — a much better fit for agents than hundreds of parallel synchronous calls. See [Sync vs batch](/docs/sync-vs-batch.md) for paging beyond 1,000 items. ## Keeping token use low - `include_html: false` on every fetch. HTML is often 5–20× larger than `text`. - Trim `text` to a budget, or search it locally for the passages you need, before putting it into the prompt. - Do not echo whole result objects into the context: `headers`, `attempts`, `redirects` and `pdf_text_pages` are for debugging and citations. - Prefer search snippets to decide *whether* to fetch. Results carry every snippet each engine returned. - Re-fetching within the cache lifetime is instant and billed as a cache hit — no need for the agent to memorise pages it may need again. ## Being a good citizen - Respect `robots_disallowed`; do not try to work around it. - Handle `429` by waiting `Retry-After` seconds. On `504`, wait before retrying (the work usually finishes in the background and the retry becomes a cache hit). - Treat `politeness_timeout` and `all_tiers_failed` as "not now", not "try again immediately". `domain_unavailable` comes with `retry_after_s`: do not ask for that domain again before then — use another source. - Use a dedicated project (and key) per agent with a modest budget, so a runaway loop cannot spend more than intended. Check `GET /v1/usage`. ## A system prompt fragment > You can search the web and read pages with the Cormorant tools. Search first > and read the snippets; fetch only the most relevant 1–3 pages. Fetch with > `include_html` false and use the returned `text`. If a fetch result has an > `error`, do not retry it immediately; try another source. For more than 20 > URLs, use the batch tool and check the job status later. Cite `final_url` > and `retrieved_at` for every fact you take from a page. <!-- source: https://www.cormorant.ch/docs/mcp.md --> # MCP server `cormorant-mcp` exposes Cormorant to AI agents as [Model Context Protocol](https://modelcontextprotocol.io) tools over stdio. It is a small, separate Python package (in `clients/mcp/` of the Cormorant repository) that calls the public API with your project key — install it where the agent runs, not on the Cormorant servers. ## Tools | Tool | What it does | |---|---| | `cormorant_search` | Web search: ranked results with title, URL and up to three snippets. | | `cormorant_fetch` | Fetch a page, PDF or document: main text (clipped to `max_chars`, continue with `offset`), title, `final_url`, `content_kind`, `retrieved_at`, cost and `error`. Options: `render`, `allow_premium`, `timeout_s`, `pdf_pages`, `include_body_url` (adds `body_url`, `sha256`, `body_kind`). | | `cormorant_batch_fetch` | Queue many URLs as one batch job; returns a `job_id`. The same URL list maps to the same idempotency key, so a retried call never creates a second job. | | `cormorant_job_status` | Job state and counts; with `include_items`, a page of results with clipped text. | | `cormorant_sec_company` | SEC EDGAR: resolve a ticker, CIK or company name to `cik`, `title`, `tickers`, `exchanges` (candidates when a name is ambiguous). | | `cormorant_sec_filings` | SEC EDGAR: a company's filings (accession, form, dates, primary document), filtered by `forms` and `since`; `latest: true` for the newest annual report (10-K/20-F/40-F). | | `cormorant_sec_document` | SEC EDGAR: the documents of a filing, or the text of one of them (clipped, continue with `offset`; `pdf_pages` for PDFs). | | `cormorant_usage` | Usage, charges and limits of the key (or the whole account with an account admin key): `view` = `summary` (totals and the busiest groups with `group_by`), `quota` (limits, allowances, recent alerts), `requests` (latest requests) or `domains`. | | `cormorant_company_search` | Find companies by name or identifier (`scheme:value`), optionally by country; each hit has a `ref`. | | `cormorant_company_get` | One company's register record (identifiers, names, legal form, status, addresses, relationships …) with the source of every field; `include` adds people, ownership, filings, financials, events. | | `cormorant_company_resolve` | Match name/country/city/identifiers to the register: a confident `match` or ranked candidates. | The tools never return HTML and clip text, to keep the agent's context small. Fetches **do not use the premium routes** unless the agent passes `allow_premium: true` or you set `CORMORANT_ALLOW_PREMIUM=1`. ## Configuration | Variable | Default | | |---|---|---| | `CORMORANT_API_KEY` | — | Required. The project key (`cmr_…`). Use a dedicated project with a modest budget for agents. | | `CORMORANT_URL` | `https://api.cormorant.ch` | API base URL. | | `CORMORANT_ALLOW_PREMIUM` | off | `1` lets fetches use the premium routes by default. | ## Install From a checkout of the repository (Python 3.10+): ```sh pip install ./clients/mcp # provides the `cormorant-mcp` command # or, without installing anything permanently: uvx --from ./clients/mcp cormorant-mcp ``` The main project also declares the dependency as an optional group: `pip install -e ".[mcp]"` and `python clients/mcp/cormorant_mcp/server.py`. ## Register it ### Claude Code ```sh claude mcp add --transport stdio cormorant --scope user \ --env CORMORANT_API_KEY=cmr_... \ -- uvx --from /path/to/cormorant/clients/mcp cormorant-mcp ``` Use `--scope project` to share the server through the repository's `.mcp.json` instead (then keep the key out of the file and reference an environment variable). Check it with `claude mcp list`. ### Claude Desktop Add to `claude_desktop_config.json` (Settings → Developer → Edit Config) and restart Claude Desktop: ```json { "mcpServers": { "cormorant": { "command": "uvx", "args": ["--from", "/path/to/cormorant/clients/mcp", "cormorant-mcp"], "env": {"CORMORANT_API_KEY": "cmr_..."} } } } ``` ### Other MCP clients Any client that launches stdio servers works the same way: command `cormorant-mcp` (or `uvx --from /path/to/cormorant/clients/mcp cormorant-mcp`), with `CORMORANT_API_KEY` in its environment. To try the tools interactively, use the MCP Inspector: ```sh CORMORANT_API_KEY=cmr_... npx @modelcontextprotocol/inspector uvx --from ./clients/mcp cormorant-mcp ``` ## How agents should use it The server sends these instructions to the client: search first and read the snippets, fetch only the most relevant pages, do not retry a fetch whose result has an `error`, use the batch tool for more than 20 URLs, and cite `final_url` and `retrieved_at`. See [Using Cormorant from AI agents](/docs/agents.md) for the reasoning and for patterns without MCP. ## Development The tool logic is plain async functions over an `httpx.AsyncClient` (`search`, `fetch`, `batch_fetch`, `job_status` in `cormorant_mcp/server.py`); `tests/test_mcp_server.py` in the main repository tests them, and the MCP registration, with mocked HTTP. <!-- source: https://www.cormorant.ch/docs/api.md --> # Cormorant API v1 Base URL: `https://api.cormorant.ch`. All endpoints below are under `/v1` except health and metrics. The contract is versioned: fields are only ever **added** within v1; nothing is renamed or removed. Guides, the interactive reference and `llms.txt` are served at `/docs` (see the [changelog](/docs/changelog.md) for what changed). ## Authentication Every `/v1` request needs a project API key (except the public [service status](#service-status), `/v1/status*`): ``` Authorization: Bearer cmr_... ``` Keys belong to exactly one project, are stored only as HMAC-SHA256 hashes, and are created/revoked by the operator (Mission Control, or `python -m app.cli create-key <project> [--role account_admin]`, `revoke-key <prefix>`). Missing or invalid keys get `401`. There are two kinds of keys, and the prefix tells them apart: | Prefix | Role (`X-Cormorant-Key-Role`) | Sees | |---|---|---| | `cma_…` | `account_admin` — the **account admin key** | everything of the customer account where an endpoint offers it: all its keys, projects, usage and invoices | | `cmr_…` | `standard` — an **additional key** | its own usage only | Both authenticate every `/v1` endpoint the same way. A key's role is fixed when it is created; to change it, have a new key issued and the old one revoked. Keys issued before roles existed are standard keys. Every authenticated `/v1` response carries the header `X-Cormorant-Key-Role: account_admin` or `X-Cormorant-Key-Role: standard`. Each project has a request-rate limit (default 200 requests/s, fixed one-second window); excess requests get `429` with `Retry-After: 1`. A batch call counts as one request. Every authenticated response carries `RateLimit-Policy` and `RateLimit` (IETF draft; `"project";q=200;w=1` and, for a key with its own limit, `"key";q=10;w=1`). **Debug requests (available on request).** When you report a problem with a specific request, Cormorant support can enable debugging for one of your keys. Send that key's synchronous `POST /v1/search` or `POST /v1/fetch` request with the header `X-Cormorant-Debug: 1`: the request is recorded in full detail for support, and the answer carries `debug_url`, a link to that record (it opens in the operator console, not with an API key). Quote it in your report. Without debugging enabled for the key, the header is ignored and the answer is the usual one. Debug requests are billed like any other request. **Per-key limits.** The operator can set, per key, a monthly spend cap (in the customer's invoice currency; provider cost in USD for projects without a customer), monthly request quotas per category (`search`, `fetch`, `fetch_premium`, `company`; SEC EDGAR requests count as `fetch`), requests per second and the allowed categories. They are enforced on every request and batch item: `403 key_category_not_allowed`, `429 key_quota_exhausted` / `key_budget_exhausted` (`Retry-After` until the month resets, UTC) and `429 key_rate_limited`. A key without `fetch_premium` (not allowed, or its quota used up) still fetches, without the premium routes. Keys with limits get `X-Cormorant-Key-Budget-Remaining: 12.40 USD` and `X-Cormorant-Quota-Remaining: fetch=18766, search=900` on every response. Request counts are exact; spend is the aggregates (5 minutes) plus an estimate since then. Keys of a suspended customer get `403` with `{"detail": {"error": "customer_suspended", "message": "…"}}`. Every response carries a `request_id` (`req_…`; batch items use `<job_id>:<index>`). Quote it when debugging — Mission Control can show the full attempt history for any request id. `cost_usd` is Cormorant's provider spend caused by *this* request in USD (search fallback and premium routes); cache hits and standard routes cause no provider cost. Spend is attributed only to the project whose request made the provider call; the caches are shared. **Keys of an external customer get `cost_usd: null`**: customers see what they are charged — per request in `GET /v1/requests` (`charge`), aggregated in `GET /v1/usage` — never Cormorant's provider cost. Every search, fetch and SEC result (also batch items) carries `billing_category` (`search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `sec`, `sec_cached`; company data requests count as `company` / `company_cached`; new categories may be added): the category, and so the plan meters, the request counts toward. SEC EDGAR requests (`/v1/sec/…` and fetches of SEC hosts) are `sec` / `sec_cached`: no provider cost, billed at your plan's price for them, and counted against a key's `fetch` quota. ## POST /v1/search ```json { "query": "Nestlé sustainability report 2025", "language": "en", // optional, ISO 639-1 (SearXNG language, Brave search_lang) "country": "CH", // optional, ISO 3166-1 alpha-2 (combined with language for SearXNG) "max_results": 20, // 1..50, default 20 "freshness": "month", // optional: day | week | month | year (pd/pw/pm/py accepted) "cache_ttl_s": 604800, // optional: max acceptable cache age; 0 = bypass the cache "project_ref": "company:4711" // optional your own text, stored with usage } ``` Response `200`: ```json { "query": "Nestlé sustainability report 2025", "results": [ { "url": "https://www.nestle.com/sustainability/...", "title": "Creating Shared Value and Sustainability Report 2025", "snippets": ["Our 2025 report covers ...", "Download the full report ..."], "engines": ["google", "bing", "brave_api"], "published_at": "2026-03-02T00:00:00", "rank": 1 } ], "provider": "searxng", // "searxng" | "brave" "fallback_reason": null, // why Brave was used (see below), else null "unresponsive_engines": [], // SearXNG engines that failed for this query "cached": false, "cost_usd": 0.0, // null for keys of an external customer "billing_category": "search", // search | search_fallback | search_cached "error": null, "request_id": "req_6b1f..." } ``` Order of operations: shared search cache (key = normalised query + language, country, max_results, freshness; 7 days) → our SearXNG instances (least-loaded across all hosts, a second instance on transport failure) → Brave Search API. Brave is used when SearXNG errors or times out (`searxng_error`, `searxng_timeout`, `no_searxng_instance`), reports unresponsive engines (`unresponsive_engines`, or `captcha_engines` when any reason mentions a CAPTCHA), or returns fewer than `MIN_RESULTS` (3) usable results (`too_few_results`). By default unresponsive engines trigger the fallback only when fewer than two engines returned results (`SEARCH_FALLBACK_ON_UNRESPONSIVE= quorum`, `SEARCH_QUORUM_ENGINES=2`); `majority` (as many engines failed as answered) and `any` are stricter. A Brave rate limit (429) is retried after the indicated wait; batch items that still fail with it are retried later. Results are deduplicated by normalised URL (scheme, `www.`, trailing slash and tracking parameters ignored). **No snippet is lost**: every distinct snippet and engine seen for a URL is kept, in order of first appearance. When Brave is used, Brave's results come first, followed by any SearXNG-only results. If Brave is unavailable or the project budget is exhausted, whatever SearXNG returned is served (not cached) with `fallback_reason` set. A **genuinely empty** search — the engines (and Brave, when consulted) answered and found nothing — returns `results: []` with `error: null`. Only when the providers could not answer is `error` set (e.g. `budget_exhausted`, `brave_http_429`, `searxng_timeout`). ## POST /v1/fetch ```json { "url": "https://www.example.com/annual-report-2025.pdf", "render": "auto", // auto | never | always "max_bytes": 15728640, // default 15 MB, max 50 MB (batch items left at the default: PDFs up to 200 MB) "accept_pdf": true, "cache_ttl_s": 604800, // optional max acceptable cache age; 0 = bypass "min_text_chars": 500, // below this, HTML main text counts as "thin" "allow_archive": true, // allow an archived copy when the live site blocks us, is gone or 404s "include_html": true, // default true here, false for batch items "allow_premium": true, // false: never use proxy / proxy_render "tiers": ["direct", "browser", "archive"], // optional allowlist of tiers "timeout_s": 60, // optional overall deadline, 5–150 s "country": "CH", // optional geotargeting (premium routes) "user_agent": "Example Research contact@example.org", // must be on the project allowlist "accept_language": "de-CH,de;q=0.9", "project_ref": "company:4711" } ``` Response `200` (also for unsuccessful fetches — check `error`): ```json { "url": "https://www.example.com/annual-report-2025.pdf", "final_url": "https://www.example.com/annual-report-2025.pdf", // null when unknown (proxy_render) "redirects": [ // one entry per redirect before final_url {"url": "http://example.com/ar25", "status": 301, "location": "https://www.example.com/annual-report-2025.pdf"} ], "status": 200, "content_type": "application/pdf", // what the body is (rendered pages: text/html) "origin_content_type": "application/pdf", // what the origin sent; null when unknown "headers": {"content-type": "application/pdf", "last-modified": "Tue, 03 Mar 2026 09:00:00 GMT"}, "tier": "direct", // direct | browser | archive | proxy | proxy_render "attempts": [ {"tier": "direct", "status": 200, "reason": "ok", "ms": 412} ], "html": null, // decoded HTML for HTML pages "text": "Annual Report 2025\n\n...", "pdf_text_pages": [{"page": 1, "text": "Annual Report 2025"}, {"page": 2, "text": "..."}], "pdf_page_count": 148, // PDFs: number of pages; null otherwise "text_truncated": false, // PDF text stopped at the extraction limits "title": "Annual Report 2025", "meta": null, // article metadata for HTML pages (see below) "sha256": "a3f1...", // of the retrieved bytes "bytes": 2483112, "cached": false, "revalidated": false, // true: expired cache entry confirmed by the origin (304) "retrieved_at": "2026-10-19T08:12:44.120Z", "etag": "\"5f2-1a\"", "last_modified": "Tue, 03 Mar 2026 09:00:00 GMT", "archived_at": null, // snapshot time (tier = archive) or common_crawl capture time "served_from_archive_because": null, // domain_dead | not_found when the archive stands in "body_url": "https://api.cormorant.ch/v1/bodies/a3f1...", // exact retrieved bytes "body_kind": "origin", // origin | rendered_dom (what sha256 covers) "content_kind": "pdf", // html | pdf | text | json | xml | binary "paywall": null, // HTML only: none | metered | hard | unknown "paywall_evidence": [], // e.g. ["jsonld_not_free", "vendor_piano"] "text_complete": null, // false when text is only a paywall teaser "alternate_of": null, // set when text comes from a verified copy elsewhere "alternate_source": null, // publisher_feed | guardian_api | open_access | syndication | common_crawl "match_score": null, "primary_sources": [], // [{url, kind, text}] for paywalled/unavailable pages "cost_usd": 0.0, // null for keys of an external customer "billing_category": "fetch_own", // fetch_own | fetch_premium | fetch_cached | sec | sec_cached "error": null, "request_id": "req_91c2..." } ``` `text` is exactly the retained text Cormorant stores and serves again on cache hits (main text via trafilatura, readability fallback, then visible body text; whitespace normalised). Anchor quotes against it. For PDFs, `text` is the pages joined with blank lines and `pdf_text_pages` has the per-page text. **Large PDFs.** `max_bytes` applies to every response of `/v1/fetch`. In batch fetches whose items leave `max_bytes` unset, a PDF (by `Content-Type: application/pdf` or the `%PDF-` signature) may be up to **200 MB**; other content keeps the default 15 MB. An item that sets `max_bytes` gets exactly that limit, for PDFs too. Such a PDF is fetched by the `direct` or `proxy` tier (the browser, archive and rendering tiers keep `max_bytes`); its exact bytes are available from `body_url`. Downloading it may take at most 5 minutes in total (or the item's `timeout_s`, when shorter); a slower download ends with `error: timeout` and an attempt with reason `pdf_download_timeout`. Text extraction stops after 5,000 pages or 5,000,000 characters of page text: then `text_truncated` is `true`, `pdf_text_pages` covers the pages read so far (the last one possibly cut) and `pdf_page_count` still counts every page. To skip PDFs entirely, set `accept_pdf: false`. **Non-HTML content.** `content_kind` classifies the response (content type, sniffed when missing or generic). `json`: `text` is the decoded document (still valid JSON). `xml` (feeds, sitemaps, XBRL…): `text` is the text content, one node per line, `title` from the first `<title>`; external entities are never resolved. `text` (plain, CSV, Markdown): decoded text. `binary` (spreadsheets, archives, images…): `text` is empty — the bytes are in `body_url`. All of these are successful fetches (`error: null`). **Paywalls.** HTML pages carry a `paywall` verdict. Cormorant detects paywalls and never circumvents them (no crawler User-Agents, cookie clearing, script blocking or referer tricks). | `paywall` | Meaning | `text_complete` | |---|---|---| | `hard` | `text` is only the teaser: the publisher declares the article subscriber-only (`isAccessibleForFree: false`) and the text is cut short, or a subscription call to action follows a teaser together with paywall markup, vendor scripts or truncation | `false` | | `metered` | a meter ("2 articles left this month") or a declared paywall, but the full text came through | `true` | | `unknown` | weak signals only (a paywall vendor's script, paywall-like markup) | `true` | | `none` | no paywall signal | `true` | `paywall_evidence` lists the signals: `jsonld_not_free`, `vendor_<name>` (piano, poool, zephr, pelcro, laterpay, steady, blendle, …), `paywall_markup`, `truncation_markup`, `phrase_<lang>`, `metered_<lang>` (en, de, fr, it) and `length_mismatch` (declared word count far above the extracted text). A `hard` teaser is a successful fetch (`error: null`) returned at once — other tiers see the same paywall — and is not cached. Domains whose articles are repeatedly hard-paywalled skip the premium routes (`paywall_skip_premium` attempts). Wayback copies of articles the publisher marks as subscriber-only are not served (`archive_blocked_paywall`); the operator can allow them per project (`archive_paywalled`). **Alternative sources.** When the page is a hard-paywall teaser, disallowed by robots.txt, gone (404/410, `domain_dead`) or blocked on every tier (`all_tiers_failed`, `domain_unavailable`), Cormorant looks for a verified open copy elsewhere (`allow_alternates`, default `true`): | `alternate_source` | How it is found and verified | |---|---| | `publisher_feed` | tried first, for paywall teasers and blocked pages only: the article's item in the publisher's own full-text RSS/Atom feed (feeds Cormorant found linked from the publisher's pages or the operator added, and detected to carry whole articles). The item must link to the requested URL (tracking parameters ignored; `match_score` 1.0) or have the same title and contain ≥ 50 % of the teaser's opening word 5-grams; items that are themselves teasers (short, or ending in a subscribe call-to-action) are never used. `final_url` is the feed | | `guardian_api` | theguardian.com URLs via the Guardian Content API (when the operator configured a key) | | `open_access` | a DOI in the URL or the page's citation metadata → the best open-access location OpenAlex knows (repository, arXiv, Europe PMC, publisher OA), fetched with the standard `direct` route; `match_score` 1.0 | | `syndication` | the headline is searched; copies on other sites (wire services first) are fetched with `direct` and accepted only if ≥ 50 % of the teaser's opening word 5-grams appear in them and they are not paywalled themselves; `match_score` is that share | | `common_crawl` | **batch fetches only** (too slow for synchronous calls), for pages that are gone (404/410, `domain_dead`) or blocked on every tier — never for paywall teasers or robots-disallowed pages: the newest HTTP 200 capture of the exact URL in the last three Common Crawl crawls (CDX index, then the single WARC record by HTTP Range request). Domains known to be paywalled are skipped, and a capture whose markup declares the article subscriber-only (`isAccessibleForFree: false`) or that is itself a paywall teaser is not served. `archived_at` is the capture time, `final_url` the captured URL, `html` the captured page; `match_score` 1.0 | The result then has `tier: "source"`, `alternate_of` (the URL asked for), `final_url` (the copy), `text_complete: true` and `paywall` still describing the original page. Without a verified copy the original answer stands (the flagged teaser or the error) and `attempts` show what was tried. `primary_sources` lists links from the page to court decisions, government/regulator pages, press releases, DOIs and PDFs. Searching for copies may use the Brave fallback (billed like a search). The operator can turn alternates off per project (`set-option <project> alternates off`). **Provenance.** `redirects`, `headers` (an allowlist of the origin's final response headers: content type/length/language/disposition, ETag, Last-Modified, Cache-Control, Expires, Date, Link, X-Robots-Tag — never cookies) and `origin_content_type` are kept with cache entries and returned on hits. For `tier = archive`, `final_url` is the archived copy's URL (`https://web.archive.org/web/<timestamp>id_/<url>`) and `archived_at` the snapshot time. The rendering tier (`proxy_render`) does not report the post-redirect URL or origin headers: `final_url`, `origin_content_type` are `null` and `headers` is empty rather than guessed. **Per-request controls.** `allow_premium=false` removes the premium routes; `tiers` restricts the ladder to the listed tiers (an empty intersection is `error: no_tier_allowed`; cache hits from other tiers are not used). `timeout_s` is a deadline for the whole fetch across tiers — when it runs out the result is `error: timeout` with a `request_timeout` attempt (or the best page so far). `country` geotargets the premium routes (`proxy`, `proxy_render`). `user_agent` replaces our browser User-Agent for `direct`, `browser` and `proxy` (not `proxy_render`); it must be on the project's allowlist, which the operator sets (`set-option <project> allowed_user_agents '["…"]'`) — services such as SEC EDGAR and Wikimedia require a contact User-Agent. Otherwise `403 user_agent_not_allowed`. `accept_language` sets the Accept-Language header / browser locale header. **Cache hits** are used only when they satisfy the request: `render=always` needs a rendered copy, `render=never` an origin copy; HTML text must reach `min_text_chars`; a cached body larger than `max_bytes` answers `too_large` (as a fresh fetch would). Otherwise the page is fetched again. Pages returned as *best effort* (every tier gave thin text) are never cached. **Revalidation.** An expired entry (or one older than `cache_ttl_s`) that would otherwise satisfy the request and has an `ETag` or `Last-Modified` is revalidated: the `direct` tier sends `If-None-Match` / `If-Modified-Since`. `304 Not Modified` returns the cached copy with `cached: true`, `revalidated: true`, a single `direct` attempt with reason `not_modified`, `cost_usd: 0`, a new `retrieved_at`, and renews the entry's freshness (new validators from the 304 are kept). Any other answer takes the normal path (a `200` replaces the entry). `cache_ttl_s: 0` never revalidates. Other tiers never send conditional requests. ### Article metadata: meta For HTML pages `meta` describes the article; it is `null` for other content kinds and on errors. It is kept with cache entries (identical on hits) and included in batch results. ```json "meta": { "author": "Anna Muster; Beat Beispiel", "published_at": "2026-03-19T08:30:00Z", "modified_at": "2026-03-19T10:05:12Z", "language": "de-CH", "canonical_url": "https://www.example.com/news/central-bank-cuts-rates", "publisher": "Example Media AG", "site_name": "Example News", "description": "The central bank cut its policy rate by 0.25 percentage points.", "section": "Economy", "word_count": 1240 } ``` | Field | Meaning | |---|---| | `author` | Author name(s); several are separated by `; ` | | `published_at` | Publication date, ISO 8601 (format below) | | `modified_at` | Last modification date, ISO 8601 | | `language` | Language tag (BCP 47), e.g. `de-CH` | | `canonical_url` | Canonical URL the page declares, absolute | | `publisher` | Publishing organisation | | `site_name` | Name of the site or periodical | | `description` | The page's own summary (at most 2,000 characters) | | `section` | Section or category (the first one) | | `word_count` | Words in `text` — what was retrieved, not what the page declares | Every field is `null` when the page does not say. Sources, in order of preference: JSON-LD (the `NewsArticle`, `Article`, `BlogPosting`, `ScholarlyArticle` or `Report` node, also inside `@graph` or arrays; malformed JSON-LD is skipped or repaired, never an error), OpenGraph and `article:*` meta tags, Highwire `citation_*` meta tags (scholarly pages), `<html lang>` and `<link rel=canonical>` (which wins for `canonical_url`), then trafilatura's metadata detection. Dates are `YYYY-MM-DDTHH:MM:SSZ` (converted to UTC) when the page gives a time zone, `YYYY-MM-DDTHH:MM:SS` when it gives a time without zone, and `YYYY-MM-DD` (or `YYYY-MM`, `YYYY`) when it gives only a date — no time or zone is invented. ### Exact bytes: GET /v1/bodies/{sha256} Every fetch stores the exact bytes it retrieved, content-addressed by `sha256`, and returns `body_url` (null when object storage is unavailable). `body_kind` says what the hash covers: - `origin`: the HTTP entity body the origin (or the archive, for `tier = archive`) sent, after content-encoding (gzip/br) is removed — the bytes a plain HTTP client would hand you; - `rendered_dom`: the document serialized by a browser (`browser`, `proxy_render`) after scripts ran — use it for text, not as proof of the origin's bytes. Non-HTML documents a browser downloads (PDFs) are `origin`. `GET /v1/bodies/{sha256}` (same Bearer key) streams the bytes with the origin `Content-Type`, `Content-Length`, `ETag: "<sha256>"`, `X-Body-Kind` and `Cache-Control: immutable`; `If-None-Match` gives `304`; `HEAD` returns the headers only. A project can download only bodies it has retrieved itself (fresh or from the shared cache); anything else is `404`. Bodies are kept while a cache entry points to them or a project retrieved them within the last 30 days. ### Tiers and escalation | Tier | What | Route | |---|---|---| | `direct` | HTTP with Chrome's TLS/HTTP2 fingerprint and headers, redirects, compression | standard | | `browser` | Chromium (new headless) with per-site cookies; waits out JS interstitials | standard | | `archive` | recent Wayback Machine snapshot (≤ 365 days), only when blocked | standard | | `proxy` | residential network (several networks, failover between them) | premium (also per GB) | | `proxy_render` | rendering service behind residential addresses | premium | The ladder starts at `direct` (or at a tier learned for the domain, see architecture). It escalates on network errors, HTTP 403/429/451/5xx and bot-challenge pages (Cloudflare, Akamai, DataDome, PerimeterX, Imperva, Kasada, generic interstitials). Thin main text escalates `direct → browser` only. `archive` and the premium routes are used only when the other standard routes were *blocked* — never for a page that is simply short. Definitive answers (404, 410, 401, …) are returned without escalation — a 404/410 on `direct` never reaches a premium route (at most the archived copy, see below). No CAPTCHA is solved unless the operator enabled it for the project. **Domains that refuse the premium routes too.** When a fetch fails on every tier and every premium route it tried was refused by the site (403/429/451, a challenge, a rendering-tier captcha or "cannot render", the provider's content policy — not provider errors or timeouts) in 2 fetches in a row, the domain is *premium-blocked* for 6 hours (then 24 h, then 3 days on repeats): its fetches still try `direct`, `browser` and the archive, but skip `proxy` and `proxy_render` (attempt reason `premium_skipped_domain_refuses`, no premium bytes) and go straight to the alternative sources, or fail with `all_tiers_failed`. 5 % of these fetches re-probe the premium routes anyway; one premium success ends the block. **Timing.** Each attempt's `ms` splits into `wait_ms` (waiting for a polite per-domain slot) and `upstream_ms` (the site, browser, archive or premium route). `timing` sums them for the request and adds `queue_ms`, the time in Cormorant's queues before a worker ran it (synchronous: from the API to a worker; batch: since admission, politeness requeues included; plus the render queue after a browser handoff): `{"queue_ms": 35, "politeness_wait_ms": 0, "upstream_ms": 412}`. The same three components are stored per request in the request log and summarised in `GET /v1/usage`. Every attempt carries `premium_bytes` (what the residential network billed for it; `0` for standard routes and `proxy_render`, which bills per call). The `proxy` route can show more than one attempt: when the site refuses a residential exit, or the network fails, the request is retried at once through another residential network, each attempt with its own status, reason and premium bytes. Prices are per route and do not depend on which network served the page. A failed fetch says which premium routes were tried in `premium_routes_tried` and summarises every attempt in `error_detail`, e.g. `"all_tiers_failed: direct (http_403), browser (challenge_akamai), proxy (http_403, 16211 premium bytes), proxy_render (http_403); premium routes tried: proxy, proxy_render (16211 premium bytes)"`. **Dead domains and missing pages.** A host whose name does not exist (NXDOMAIN — not a temporary DNS failure such as SERVFAIL or a timeout, which stay `blocked_destination` with reason `dns_error`) twice within 6 hours is remembered as dead for 6 hours: fetches answer `error: "domain_dead"` at once, with a single `{"tier": "dns", "reason": "domain_dead"}` attempt and no tier work. When a fetch ends in `domain_dead` or a 404/410 and `allow_archive` is true (and `tiers`, if given, includes `archive`), Cormorant looks for a Wayback snapshot (≤ 365 days) and, if there is one, returns it instead: `tier: "archive"`, `archived_at`, and `served_from_archive_because: "domain_dead"` or `"not_found"`; `attempts` still shows the 404 or the dead host. These stand-in copies are not cached. The archive's paywall rules apply (no copies for known paywalled domains or of articles marked subscriber-only, `archive_blocked_paywall`). Without a snapshot the answer stays `domain_dead` or `http_404`/`http_410` (with an `archive_not_found` attempt), and only then are alternative sources looked for. Other definitive statuses (401, …) never use the archive. `render=never` uses `direct → archive → proxy`; `render=always` uses `browser → proxy_render`. **Known hard domains.** For sites on Cormorant's list of hard domains (`nestle.com`, `nestleusa.com`, `beyondmeat.com`, `apnews.com`, `nytimes.com`, `researchgate.net`, … — see [Fetching → Known hard domains](/docs/fetching.md#known-hard-domains)) tiers that do not work for the site are skipped (`hard_domain_skip` attempts), the residential exit country is pinned (unless the request sets `country`) and `proxy_render` starts at the site's difficulty. Errors — and archive or alternative copies standing in — carry `hint` (the recommended route) and, on errors, `alternative_urls`. An explicit `tiers` list overrides the route. The `proxy` tier sends GET requests with Chrome's TLS/HTTP2 fingerprint, like `direct`. `attempts[].reason` values include `ok`, `thin_content`, `not_modified`, `challenge_<vendor>`, `http_<status>`, `timeout`, `tls_error`, `dns_error`, `dns_nxdomain`, `domain_dead`, `network_*`, `not_configured`, `budget_exhausted`, `hard_domain_skip`, `archive_not_found`, premium-route reasons (`…_captcha_rejected`, `…_no_credits`, `…_render_503`), `paywall` (teaser of a hard paywall, returned as is), `paywall_skip_premium`, `premium_skipped_domain_refuses`, `archive_blocked_paywall`, and for the alternative-source steps (`tier: source`) `alternate_<source>` (with the copy's URL in `detail`), `syndication_candidates_<n>`, `open_access_none`, `open_access_miss`, `guardian_api_miss`, `publisher_feed_miss`, `publisher_feed_teaser`, `common_crawl_none`, `common_crawl_not_free`, `common_crawl_paywalled_domain`, `common_crawl_index_error`, `common_crawl_record_error`, `source_timeout`. `proxy_render` attempts carry `detail` with the difficulty used (`low`, `medium`, `high`); other attempts omit `detail`. Busy rendering-tier answers (`…_render_503`) are retried once and appear as two attempts. ### Errors | `error` | Meaning | |---|---| | `invalid_url` | not an http(s) URL | | `blocked_destination` | private, loopback, link-local or metadata address (also after redirects) | | `robots_disallowed` | robots.txt disallows the path for `CormorantBot`/`*` | | `too_large` | body exceeds `max_bytes` (batch PDFs with `max_bytes` unset: 200 MB) | | `pdf_not_accepted` | PDF with `accept_pdf=false` | | `http_<status>` | definitive upstream status (e.g. `http_404`) | | `all_tiers_failed` | every allowed tier failed or was blocked (`status` = last HTTP status; `error_detail` and `premium_routes_tried` say what was tried) | | `domain_unavailable` | every tier failed for this domain in 3 consecutive fetches: it is skipped for a while (10 min, then 30 min, 2 h, 6 h); `retry_after_s` says when to try again. Any success ends the skip | | `domain_dead` | the host name does not exist (NXDOMAIN twice within 6 h); answered at once for 6 h, without trying. With `allow_archive`, a snapshot is returned instead when one exists (`served_from_archive_because: "domain_dead"`) | | `timeout` | `timeout_s` ran out (see per-request controls), or a large PDF took longer than 5 minutes to download (attempt reason `pdf_download_timeout`) | | `no_tier_allowed` | `tiers`/`allow_premium` left no tier to try | | `politeness_timeout` | could not get a per-domain slot in time (transient; batch items are requeued) | | `domain_busy` | synchronous calls only, with **HTTP 503** and `Retry-After`: the expected wait for one of the domain's polite slots exceeds the 15 s a synchronous call waits; `retry_after_s` is that expected wait. Lower the parallelism per domain or use a batch (whose items wait instead) | | `sec_not_configured` | SEC EDGAR URL, but the operator has not configured Cormorant's SEC identity: SEC is disabled | | `rate_limited` | SEC EDGAR URL: the cluster-wide SEC rate limit (at most 10 requests/s for all clients) had no slot within 10 s; retry after `retry_after_s` (transient) | | `sec_unavailable` | SEC EDGAR throttled (403/429/503: all SEC requests back off), failed (5xx) or could not be reached; retry after `retry_after_s` (transient) | | `redirect_not_allowed` | SEC EDGAR URL that redirected to a host outside SEC (not followed) | | `premium_route_required` | a host fetched only through premium routes with `allow_premium: false` or a `tiers` list without `proxy`/`proxy_render` | | `budget_exhausted` | such a host, and the project's budget leaves nothing for the premium routes | When no tier reaches `min_text_chars`, the attempt with the most text is returned (no error). ## Batch: POST /v1/batch/search and POST /v1/batch/fetch ```json { "items": [{"query": "..."}, {"query": "..."}], // same objects as the single endpoints "idempotency_key": "chipiron-run-2026-10-20-search-0001", "callback_url": "https://chipiron.example/hooks/cormorant" // optional } ``` Response `202` `{"job_id": "…", "created": true}`. Re-posting the same `idempotency_key` returns the existing job (`200`, `created: false`); the same key with a different kind or item count is `409`. Up to 50,000 items per job. Fetch items may set `"include_html": true` to keep the HTML in stored results (default: text only). Batch fetch items that leave `max_bytes` unset accept PDFs up to 200 MB (see *Large PDFs* above). Items are admitted to the queues gradually (fair share across running jobs), executed with the same logic as the single endpoints, and retried with backoff only for transient problems. A batch fetch item blocked on our own tiers is retried once after a cool-down before a premium route is used. When the job completes, `callback_url` receives a `POST` with `{"job_id","kind","status","total","succeeded","failed","finished_at"}` (3 attempts; private addresses are refused). The host is resolved once, checked, and the connection goes to that checked address (Host header and TLS name stay the callback's host name), so a DNS answer that changes in between cannot redirect it; redirects are not followed. ### GET /v1/jobs/{job_id} ```json { "job_id": "1d0c...", "kind": "fetch", "status": "running", // queued | running | completed | cancelled "counts": {"total": 3000, "pending": 1200, "queued": 380, "done": 1390, "failed": 30, "cancelled": 0}, "created_at": "...", "updated_at": "...", "finished_at": null, "callback_url": null, "callback_status": null, "items_url": "/v1/jobs/1d0c...?include=items&offset=0&limit=100" } ``` Add `?include=items&offset=0&limit=1000` (max 1000) to page through item results, optionally `&status=failed`: ```json "items": [ {"index": 0, "status": "done", "input": {"url": "..."}, "result": { ...fetch response... }, "error": null} ] ``` **Retention:** jobs and their item results are kept for 30 days after the job finishes, then deleted (`JOB_RETENTION_DAYS`). Download results you need to keep before that. If a stored result cannot be read (object storage unavailable), the item has `"result": null` and `"error": "result_unavailable"`; retry the request later. ### DELETE /v1/jobs/{job_id} Cancels the job: no new items are admitted (the admission pump runs every 3 s, well within 60 s), pending items become `cancelled`, queued items skip themselves. Returns the job status. Jobs are only visible to their project. ## SEC EDGAR: /v1/sec/... Company and filing data from SEC EDGAR. Cormorant sends **its own** declared identity to SEC (`"<Org name> <contact email>"`, set by the operator) and keeps all clients together under SEC's fair-access limit: one cluster-wide rate limit per egress IP (default 8 requests/s, never more than 10), queued for up to 10 s, and a shared backoff when SEC throttles (403/429/503). Only the direct tier is used — never the browser, archived copies, the residential network or alternative sources. **Do not send your own User-Agent** for SEC: a `user_agent`/`accept_language` in `POST /v1/fetch` is ignored for SEC hosts. The same policy applies to `POST /v1/fetch` and batch items for URLs on `www.sec.gov`, `data.sec.gov` and `efts.sec.gov`. All endpoints need an API key, count towards the project's rate limit and are metered as kind `sec` (no provider cost). Answers are HTTP 200 also when they failed; check `error`: | `error` | Meaning | |---|---| | `unknown_identifier` | neither SEC nor Cormorant knows this ticker, CIK, name or ISIN (a home-market ticker of a non-US company: use its ADR ticker or the ISIN) | | `not_sec_filer` | the company is known (a non-US listing such as `NESN.SW` / `XSWX:NESN`, the company register, GLEIF for an ISIN) but not registered with the SEC under this identifier; `hint`: `company`, `listing`, `sec_candidates` (SEC filers with its name, e.g. the ADR) | | `not_found` | SEC has no such filing or document (also: no filing of the requested forms) | | `invalid_identifier` | a CIK, accession number, document name, form type, date or query is malformed (`message` says which); nothing was requested from SEC | | `rate_limited` | no slot in the shared SEC rate limit within 10 s; retry after `retry_after_s` | | `sec_unavailable` | SEC throttles (all SEC requests back off) or failed; retry after `retry_after_s` | | `sec_not_configured` | SEC is not enabled on this deployment | | `timeout`, `too_large`, `robots_disallowed`, … | the fetch errors | Identifiers: CIKs as a number or 10 digits (`100493`, `0000100493`); accession numbers with or without dashes (`0000100493-25-000070`, `000010049325000070`); form types comma-separated (`10-K,20-F,40-F,10-K/A`). Only the SEC URL patterns behind each endpoint can be requested. Every answer has `error`, `message`, `retry_after_s`, `cached` (`true` when all SEC documents came from Cormorant's cache), `retrieved_at` (oldest document), `source_url` (the main SEC document), `body_url` and `sha256` (its exact bytes, e.g. the raw submissions JSON) and `request_id`. **Caching.** Filing documents and indexes (`/Archives/edgar/data/{cik}/{accession}/…`) are immutable and cached for 365 days (revalidated only when you ask with `cache_ttl_s`); `company_tickers.json`, submissions and XBRL facts for 12 h, then revalidated with conditional requests; full-text search for 1 h. ### GET /v1/sec/companies/resolve?ticker=TSN Also `?cik=100493` or `?name=tyson foods` (exactly one). From `company_tickers.json` and `company_tickers_exchange.json`: ```json {"cik": 100493, "cik_padded": "0000100493", "title": "TYSON FOODS, INC.", "tickers": ["TSN"], "exchanges": ["NYSE"], "ambiguous": false, "candidates": [{"cik": 100493, "cik_padded": "0000100493", "title": "TYSON FOODS, INC.", "tickers": ["TSN"], "exchanges": ["NYSE"]}], "error": null, "cached": true, "source_url": "https://www.sec.gov/files/company_tickers.json", "…": "…"} ``` `BRK.B` and `BRK-B` are the same ticker. A name matches exact titles first, then titles starting with or containing it; when several companies match, `ambiguous` is `true`, the company fields are `null` and `candidates` lists up to 10. A CIK without a listed ticker is resolved from its submissions. ### GET /v1/sec/companies/{cik}/submissions Query: `forms`, `since`, `until` (filing dates, `YYYY-MM-DD`), `limit` (default 100, max 10,000), `history` (default `true`: also read the pages of older filings SEC lists under `files`, only those that can hold filings after `since`). Company fields `cik`, `cik_padded`, `name`, `title`, `tickers`, `exchanges`, `entity_type`, `sic`, `sic_description`, `category`, `fiscal_year_end`, `state_of_incorporation`, `ein`, `lei`, `website`, `phone`, `business_address`, `former_names`; `filings` (newest first, at most `limit`), `filings_total`, `truncated`, and `files` (`name`, `filing_count`, `filing_from`, `filing_to`, `url`, `fetched`, `body_url`). Each filing: | Field | From SEC's column | |---|---| | `accession` | `accessionNumber` | | `form`, `filing_date`, `report_date`, `acceptance_datetime` | `form`, `filingDate`, `reportDate`, `acceptanceDateTime` | | `act`, `file_number`, `film_number`, `items`, `core_type`, `size` | `act`, `fileNumber`, `filmNumber`, `items`, `core_type`, `size` | | `is_xbrl`, `is_inline_xbrl` | `isXBRL`, `isInlineXBRL` (as booleans) | | `primary_document`, `primary_document_description` | `primaryDocument`, `primaryDocDescription` | | `primary_document_url`, `index_url` | built: the document and the `-index.htm` page (`null` URL when the primary document is in a sub-folder, e.g. ownership forms) | | `is_amendment` | built: the form ends in `/A` | ### GET /v1/sec/companies/{cik}/filings/latest?forms=10-K,20-F,40-F The newest filing (by filing date) of `forms` — default `10-K,20-F,40-F` (annual reports; amendments only when you list them, e.g. `10-K,10-K/A`): the filing fields above at the top level (`accession`, `form`, `filing_date`, `report_date`, `primary_document`, `primary_document_url`, `index_url`, `is_amendment`, …) plus `cik`, `cik_padded`, `name`, `tickers`, `exchanges`. Older filing pages are searched when the recent list has none; none at all: `not_found`. ### GET /v1/sec/filings/{cik}/{accession} The filing index: `form`, `filing_date`, `report_date`, `accepted`, `company`, `is_amendment`, `index_url`, `folder_url`, `accession`, `accession_nodash`, `index_source` (`index_htm`, or `index_json` when only the folder listing has the documents) and `documents`: `sequence`, `description`, `document` (file name), `type` (`10-K`, `EX-21`, `EX-101.SCH`, …), `size`, `url`. Sub-folders are not listed. ### POST /v1/sec/documents/fetch ```json {"cik": 100493, "accession": "0000100493-25-000070", "document": "tsn-20250927.htm", "max_bytes": 52428800} ``` Fetches one document of a filing (HTML/iXBRL, PDF, exhibit, XML) and answers **exactly like `POST /v1/fetch`** (`text`, `pdf_text_pages`, `content_kind`, `body_url`, `sha256`, `body_kind: origin`, …). Optional: `include_html` (default `true`), `accept_pdf`, `cache_ttl_s`, `timeout_s`, `project_ref`. Invalid identifiers answer `error: invalid_identifier` in that shape. ### GET /v1/sec/companies/{cik}/facts A summary of `data.sec.gov/api/xbrl/companyfacts/CIK##########.json`: `cik`, `cik_padded`, `name`, `taxonomies` (concepts per taxonomy), `concepts`, `bytes`; the complete JSON is at `body_url`. ### GET /v1/sec/search?q=…&forms=…&dateRange=… EDGAR full-text search (filings since 2001). Query: `q` (EDGAR syntax), `forms`, `dateRange` (`all`, `10y`, `5y`, `1y`, `30d`, or `custom` with `startdt`/`enddt`), `from` (offset, pages of 100). Answer: `query`, `total`, `total_relation`, `hits` (`accession`, `ciks`, `display_names`, `form`, `root_forms`, `file_type`, `file_description`, `file_date`, `period_ending`, `document`, `url`, `index_url`, `score`). ### Batch: POST /v1/batch/sec Up to 50,000 SEC operations as one job of kind `sec`: ```json { "items": [ {"op": "resolve", "ticker": "TSN"}, {"op": "latest", "ticker": "TSN", "forms": "10-K"}, {"op": "latest", "cik": 100493}, {"op": "document", "cik": 100493, "accession": "0000100493-25-000070", "document": "tsn-20250927.htm"} ], "idempotency_key": "chipiron-run-2026-10-20-sec-0001", "callback_url": "https://chipiron.example/hooks/cormorant" } ``` `op` is `resolve` (`ticker`, `cik` or `name`), `submissions` (`cik` or `ticker`, `forms`, `since`, `until`, `limit`, `history`), `latest` (`cik` or `ticker`, `forms`), `facts` (`cik` or `ticker`), `filing` (`cik`, `accession`) or `document` (`cik`, `accession`, `document`, `max_bytes`, `accept_pdf`, `include_html` — default `false` in a batch); `project_ref` on any item. Identifiers are checked when the job is created (`422 invalid_item` naming the item). Item results have the shape of the single endpoint (plus `op`); `document` results that of `POST /v1/sec/documents/fetch`. Items share the cluster-wide SEC rate limit with every other SEC request (at most 10 requests/s per egress IP) and are admitted a few at a time: a full limiter or a SEC backoff puts an item back in the queue instead of failing it. Each item is metered once as kind `sec` (a ticker resolved inside an item is not extra) and counts as `fetch` for key limits. ## GET /v1/usage?from=YYYY-MM-DD&to=YYYY-MM-DD Usage of the caller's scope and what it costs. Inclusive dates (UTC), default: current month to date, at most 400 days. Read from aggregates refreshed every 5 minutes, never from the request log, so it answers in the same time at any volume. **Who sees what.** A standard key (`cmr_…`) sees only its own requests. An account admin key (`cma_…`) sees every key and project of its customer — revoked keys included, and usage recorded before per-key attribution as `"key": null`. A project without a customer is its own account. Nothing ever crosses a customer boundary; a key or project you cannot see is `404`, like one that does not exist. **Parameters** | Parameter | Meaning | |---|---| | `from`, `to` | Range of UTC days (inclusive). | | `group_by` | Comma-separated, combinable: `key`, `project`, `category`, `day`, `hour` (hourly data: last 35 days). | | `key` | Only these keys (comma-separated prefixes, e.g. `cmr_AbCdEf`). | | `project` | Only these projects (comma-separated names). | | `category` | Only these billing categories: `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. | | `format` | `csv` (same as `Accept: text/csv`). | ```json { "project": "acme-crawler", "key": "cma_XyZaBc", "role": "account_admin", "scope": {"customer": "Acme", "projects": ["acme-crawler", "acme-search"], "keys": "all"}, "from": "2026-10-01", "to": "2026-10-19", "as_of": "2026-10-19T11:05:02+00:00", "final": false, "currency": "USD", "totals": {"requests": 9120, "succeeded": 8800, "failed": 320, "success_rate": 0.9649, "cached": 1200, "cache_hit_rate": 0.1316, "errors": {"http_404": 210, "all_tiers_failed": 110}, "latency_ms_p50": 900, "latency_ms_p95": 7400, "bytes": 812345678, "premium_requests": 90, "premium_share": 0.0113, "charge": "15.96", "billable": {"fetch_own": {"quantity": "8710", "included": "1000", "billable": "7710", "amount": "15.42"}, "fetch_premium": {"quantity": "90", "included": "0", "billable": "90", "amount": "0.36"}, "premium_gb": {"quantity": null, "included": null, "billable": null, "amount": "0.18"}}}, "search": {"requests": 1200, "cached": 300, "cache_hit_rate": 0.25, "succeeded": 1198, "success_rate": 0.998, "bytes": 0, "premium_bytes": 0, "latency_ms_p50": 1100, "latency_ms_p95": 3900, "fallbacks": 60, "fallback_rate": 0.0667, "providers": {"searxng": {"requests": 840, ...}, "brave": {...}}}, "fetch": {"requests": 7920, "cached": 900, "tiers": {"direct": {...}, "proxy": {...}}, "errors": {"http_404": 210}, "by_outcome": {"succeeded": {"requests": 7600, "bytes": 800000000, "premium_bytes": 8800000, "premium_requests": 80}, "failed": {"requests": 320, "bytes": 0, "premium_bytes": 323456, "premium_requests": 10}}, ...}, "company_profile": {}, // extended company profiles (kind company_profile), same fields; {} when none "group_by": ["key"], "groups": [{"key": "cmr_AbCdEf", "key_label": "crawler", "key_role": "standard", "key_status": "active", "project": "acme-crawler", "requests": 9000, ..., "charge": "15.70", "billable": {...}}], "keys": [{"prefix": "cmr_AbCdEf", "label": "crawler", "role": "standard", "project": "acme-crawler", "active": true, "requests": 9000, "cached": 1150, "succeeded": 8690, "search_requests": 1100, "fetch_requests": 7900, "company_requests": 0, "company_profile_requests": 0, "premium_bytes": 9123456, "charge": "15.70"}], "account": [{"period": "2026-10", "from": "2026-10-01", "to": "2026-10-19", "currency": "USD", "complimentary": true, "usage_charges": "15.96", "account_level_amount": "-15.96", "account_lines": [{"kind": "complimentary", "description": "Complimentary plan: 100 % discount", "amount": "-15.96"}], "list_total": "15.96", "net_total": "0.00"}] } ``` - **Metrics** (in `totals` and every group): `requests` (cache hits included), `succeeded`, `failed`, `cached`, `errors` (failures by error code, top 50), `latency_ms_p50`/`latency_ms_p95` of uncached requests (estimated from a latency histogram with bounds 100, 250, 500 ms, 1, 2, 5, 10, 20, 30, 60 and 120 s — exact to the bucket), `bytes`, `premium_requests` and `premium_share` (fetches that needed the premium routes, of all uncached fetches). - **Charges** (projects with a customer): `charge` is what the group costs at your plan's prices in your invoice currency, as a decimal string; `billable` lists each price-plan meter with the measured `quantity`, the `included` units it used (its pro-rata share of the plan's included units of the month), `billable` and `amount`. Meters billed on provider cost show only the amount. The period is rated exactly like the invoice and each usage line is split to the cent, first per key, then per day and category: **any grouping adds up to the same total**, and the keys add up to the usage lines of the estimate (`GET /v1/account`) or invoice. Base fee, minimum, contract discount and the complimentary credit are account-level lines: account admin keys see them per period in `account`. Complimentary accounts see list-price charges and the 100 % `complimentary` line. - **Provider cost** — `cost_usd` and the project `budget` (Cormorant's spend with its search and proxy providers) appear only for projects without an external customer (internal use). Customers see what they pay, never what Cormorant pays. - **Freshness**: `as_of` is the time up to which every request is included; `final: true` (top level, and per group with `group_by=day`) means the days are closed and will not change (a day becomes final two hours after it ends). - `search` and `fetch` keep the per-kind blocks of earlier versions (now for the caller's scope), with `providers`/`tiers` breakdowns. With `key` in `group_by`, `keys` keeps the earlier per-key list as well. - **Latency components**: every per-kind block has `latency_components` (`queue_ms`, `politeness_wait_ms`, `upstream_ms`, each `{p50, p95}` of uncached requests), and each entry of `tiers`/`providers` adds `latency_ms_p50/p95` and `<component>_p50/_p95`, so a slow tier can be told apart from a busy domain or a queue backlog. Estimated from histograms like the other percentiles; requests from before these were recorded are not included. - `by_outcome` (in every per-kind block) splits the same requests into `succeeded` and `failed`, each with `requests`, `bytes`, `premium_bytes`, `premium_requests` (fetch) and `cost_usd` (provider cost, internal projects only): `fetch.by_outcome.failed.premium_bytes` is what blocked pages cost before the fetch gave up. Whether failed requests are billed depends on your plan (`billable`). **CSV.** `Accept: text/csv` (or `format=csv`) returns one row per group — or one row of totals without `group_by` — with the dimensions, the metrics, `charge`, the meters as `<meter>_quantity`, `<meter>_included`, `<meter>_billable`, `<meter>_amount` and `errors` as `code=n;code=n`. ```sh curl -sS "$CORMORANT_URL/v1/usage?from=2026-10-01&to=2026-10-31&group_by=key,day" \ -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Accept: text/csv" -o usage.csv ``` ## GET /v1/usage/timeseries The same metrics per `granularity=hour` (last 35 days; default range: today) or `granularity=day` (default: month to date), one point per bucket up to now — buckets without usage are zero, so the points can be charted directly. `group_by` (`key`, `project`, `category`, combinable) gives one series per value; `key`, `project` and `category` filter as for `GET /v1/usage`. Same scope rules, same charges (an hour's charge is its share of the day's), CSV with `Accept: text/csv`. ```json {"granularity": "hour", "from": "2026-10-19", "to": "2026-10-19", "as_of": "2026-10-19T11:05:02+00:00", "final": false, "currency": "USD", "group_by": ["key"], "series": [{"key": "cmr_AbCdEf", "key_label": "crawler", "key_role": "standard", "key_status": "active", "project": "acme-crawler", "points": [{"t": "2026-10-19T00:00:00+00:00", "final": false, "requests": 410, "succeeded": 402, "failed": 8, "latency_ms_p95": 6100, "premium_share": 0.01, "charge": "0.82", ...}, ...]}]} ``` ## GET /v1/usage/domains The busiest fetched domains of the caller's keys over the range (kept 90 days): `requests`, `succeeded`, `success_rate`, `premium_requests` and `premium_share` per domain, busiest first (`limit`, default 50, at most 500). `group_by=key` returns a top list per key. Built from each key's 100 busiest domains per day; usage recorded before per-key attribution is not included. ```json {"from": "2026-10-01", "to": "2026-10-19", "domains": [{"domain": "www.example.com", "requests": 1200, "succeeded": 1150, "success_rate": 0.9583, "premium_requests": 40, "premium_share": 0.0333}]} ``` ## GET /v1/requests The request ledger, newest first and in real time: one row per search, fetch or batch item of the caller's scope (a standard key: its own requests; an account admin key: the whole customer). Kept as long as the request log (90 days; default range: the last 30 days). | Parameter | Meaning | |---|---| | `from`, `to` | Range of UTC days. | | `key`, `project`, `category` | As for `GET /v1/usage`. | | `kind` | `search`, `fetch`, `company` (`/v1/companies/…`), `sec` (SEC EDGAR: `/v1/sec/…` and fetches of SEC hosts) or `company_profile` (extended company profiles). | | `outcome` | `succeeded` or `failed`. | | `error`, `domain`, `project_ref`, `job_id`, `cached` | Exact matches. | | `limit` | Rows per page, 1–1000 (default 100). | | `cursor` | `next_cursor` of the previous page. | ```json {"from": "2026-09-20", "to": "2026-10-19", "as_of": "2026-10-19T11:05:02+00:00", "currency": "USD", "limit": 100, "next_cursor": "MjAyNi0xMC0xOVQxMTowMjoxMyswMDowMHw5ODc2", "requests": [{"request_id": "req_6b1f2c0e9a7d4b5c8e3f1a2b3c4d5e6f", "time": "2026-10-19T11:02:13+00:00", "key": "cmr_AbCdEf", "key_role": "standard", "project": "acme-crawler", "project_ref": "company:4711", "job_id": null, "kind": "fetch", "domain": "www.example.com", "tier": "direct", "status": 200, "ok": true, "error": null, "latency_ms": 840, "cached": false, "bytes": 48211, "premium": false, "category": "fetch_own", "charge": "0.001842"}]} ``` Page with `cursor` until `next_cursor` is `null`; the next page is also in the `Link: <…>; rel="next"` header (and `X-Next-Cursor`). Pages are keyset pages, so they stay fast and consistent however deep you go, while new requests arrive on top. `charge` is the request's share of its key's charges for that day and category (their average price, 6 decimals); it is `null` until the request is aggregated (within 5 minutes) and settles when the day is final. `GET /v1/requests/{request_id}` returns one request with `reason` and the tiers tried (`attempts`); another key's (standard keys) or another customer's request is `404`. CSV with `Accept: text/csv` (one page; the next cursor is in `X-Next-Cursor`). ## GET /v1/quota Every limit that applies to the caller this month, each with `limit`, `used`, `reserved`, `remaining`, `resets_at`, `projected_end_of_period` (linear extrapolation to the end of the month) and `status` (`ok`, `warning` from 80 %, `exhausted`): the key's spend cap (`monthly_spend`, money as decimal strings), request quotas (`monthly_requests.<category>`) and rate limit; the project's rate limit (and provider budget, for projects without an external customer). An account admin key gets every active key of the account and, for a customer, the plan's included units (`included.<meter>`). ```json {"key": "cmr_AbCdEf", "role": "standard", "period": "2026-10", "resets_at": "2026-11-01T00:00:00+00:00", "as_of": "2026-10-19T11:05:02+00:00", "usage_refreshed_at": "2026-10-19T11:05:00+00:00", "keys": [{"key": "cmr_AbCdEf", "key_label": "crawler", "key_role": "standard", "project": "acme-crawler", "is_self": true, "allowed_categories": null, "usage": {"requests": {"search": 120, "fetch": 81234, "fetch_premium": 412}, "spend": "38.12", "currency": "USD"}, "limits": [{"name": "monthly_requests.fetch", "unit": "requests", "limit": 100000, "used": 81234, "reserved": 0, "remaining": 18766, "resets_at": "2026-11-01T00:00:00+00:00", "projected_end_of_period": 132550, "status": "warning"}, {"name": "monthly_spend", "unit": "USD", "limit": "50.00", "used": "37.90", "reserved": "0.22", "remaining": "11.88", "status": "ok", ...}]}], "project": {"project": "acme-crawler", "limits": [{"name": "rate_limit", "unit": "requests/s", "limit": 200, "used": 3, ...}]}} ``` `reserved` is the spend estimated since the last aggregation (or, for the provider budget, spend held for calls in flight). `events` lists the thresholds crossed this month in your scope (see *Alerts* below). `alert_email` says whether alert e-mails reach your account's alert addresses (there may be several): `status` is `deliverable`, `not_set`, `partially_undeliverable` (some addresses) or `undeliverable` (all), with `addresses`, `undeliverable` (counts), and `reason` (`bounced` or `complained`) and `since` of the first undeliverable one. After a hard bounce or a spam complaint no further alert mail goes to that address until it is changed (contact us); webhooks are not affected. ## Alerts: POST, GET, DELETE /v1/alerts A webhook receives an event whenever something in its scope crosses **50, 80 or 100 %** of a limit or allowance in a month (each threshold fires once per month): a key's spend cap or request quota, a project's provider budget (projects without an external customer) or the plan's included units of the account. The same events are listed in `GET /v1/quota` (`events`). ```json POST /v1/alerts {"url": "https://hooks.example.com/cormorant", "scope": "customer", "thresholds": [80, 100]} 201 {"id": 7, "url": "https://hooks.example.com/cormorant", "scope": "customer", "key": null, "project": null, "thresholds": [80, 100], "created_at": "2026-10-19T11:00:00+00:00", "last_delivery_at": null, "last_status": null, "secret": "whsec_9f2c…"} ``` - `scope`: `key` (one key; `key` = its prefix, default the calling key), `project` (`project` = name; the project and its keys) or `customer` (the whole account). A standard key can only watch itself (`403 account_admin_required` otherwise); an account admin key anything in its account. Default: `key` for standard keys, `customer` for admin keys. - `thresholds`: any of `50`, `80`, `100` (default all). - `url` must be `https` and resolve to a public address; at most 20 webhooks per account. `secret` is returned **only once**. - `GET /v1/alerts` lists the webhooks you can see (never secrets) with `last_delivery_at` and `last_status`; `DELETE /v1/alerts/{id}` removes one. **Deliveries** are `POST <url>` with the event as JSON: ```json {"id": "evt_42", "created_at": "2026-10-19T11:05:03+00:00", "type": "usage.threshold", "scope": "key", "customer": "Acme", "project": "acme-crawler", "key": "cmr_AbCdEf", "key_role": "standard", "limit": {"name": "monthly_spend", "unit": "USD", "limit": "50.00", "used": "40.12", "threshold": 80, "period": "2026-10", "resets_at": "2026-11-01T00:00:00+00:00"}} ``` Headers: `X-Cormorant-Event: usage.threshold`, `X-Cormorant-Delivery: dlv_…` and `X-Cormorant-Signature: t=1760871903,v1=5257a8…`, where `v1` is the hex HMAC-SHA256 of `"<t>.<raw body>"` with your `secret`. Verify it in constant time and reject a `t` older than a few minutes: ```python import hashlib, hmac, time def verify(secret: str, body: bytes, header: str) -> bool: parts = dict(p.split("=", 1) for p in header.split(",")) mac = hmac.new(secret.encode(), f"{parts['t']}.".encode() + body, hashlib.sha256) fresh = abs(time.time() - int(parts["t"])) < 300 return fresh and hmac.compare_digest(mac.hexdigest(), parts["v1"]) ``` Any `2xx` answer counts as delivered; other answers and network errors are retried after 1, 5, 15, 60, 180, 360 and 720 minutes (8 attempts). Redirects are not followed, and each delivery connects to the address that passed the check (no DNS rebinding). Events are evaluated with the usage aggregates (every 5 minutes). Use `id` to ignore duplicates. ## GET /v1/keys and GET /v1/keys/self The keys the caller may see — by prefix, label and role, never the key itself: every key of the customer's projects for an account admin key (revoked keys included), only itself for a standard key. The prefix always shows the role: `cma_…` is an account admin key, `cmr_…` a standard key. ```json {"role": "account_admin", "keys": [{"prefix": "cma_XyZaBc", "label": "ops", "role": "account_admin", "project": "acme-crawler", "status": "active", "created_at": "2026-09-01T08:00:00+00:00", "revoked_at": null, "last_used_at": "2026-10-19T11:02:00+00:00", "is_self": true}, {"prefix": "cmr_AbCdEf", "label": "crawler", "role": "standard", "status": "revoked", ...}]} ``` Each key also has `limits` (`monthly_spend_cap`, `currency`, `monthly_requests`, `rate_limit_rps`, `allowed_categories`; `null` without limits). `GET /v1/keys/self` returns the calling key in the same shape plus its use this month (`usage`) and quota state (`quota`, the entries of `GET /v1/quota`). ## Account and invoices Keys belong to a project; a project may belong to a **customer** (the billed account). These endpoints show the key's own project usage plus the plan, current-period estimate and issued invoices of that customer — never another customer's data, never drafts. Money is a decimal **string** in the invoice currency (USD, CHF or EUR). Usage is aggregated every 5 minutes (`usage_refreshed_at`). See [billing.md](/docs/billing.md) for pricing rules. ### GET /v1/account ```json { "project": {"id": 7, "name": "acme-crawler"}, "period": {"period": "2026-10", "from": "2026-10-01", "to": "2026-10-31"}, "usage_refreshed_at": "2026-10-19T11:00:04+00:00", "project_usage": {"fetch_own": {"requests": 9120, "succeeded": 8800, "bytes": 812345678, "bytes_ok": 800000000, "premium_bytes": 0, "premium_bytes_ok": 0, "cost_usd": "0", "cost_ok_usd": "0"}, "search": {...}}, // cost_usd / cost_ok_usd: null for projects of an external customer "customer": {"name": "Acme", "company_name": "Acme Research AG", "currency": "USD", "status": "active", "complimentary": true}, "customer_usage": {...}, "plan": {"id": 1, "code": "standard-usd", "version": 1, "name": "Standard (USD)", "currency": "USD", "complimentary": false}, "estimate": {"period": "2026-10", "currency": "USD", "lines": [{"kind": "usage", "component": "fetch_own", "description": "Page fetches (1,000 included)", "quantity": "8800", "unit": "fetch", "unit_price": null, "amount": "15.60", "detail": {"included": "1000", "billable": "7800", "tiers": [{"from": "0", "to": "100000", "quantity": "7800", "unit_price": "0.002", "amount": "15.60"}]}}, {"kind": "complimentary", "description": "Complimentary plan: 100 % discount", "amount": "-15.60", ...}], "subtotal": "15.60", "discount_total": "15.60", "net_total": "0.00", "vat_rate": "8.1", "vat_amount": "0.00", "total": "0.00", "complimentary": true, "list_total": "15.60"} } ``` `keys` breaks the period down per API key. An account admin key (`cma_…`) sees every key of the customer's projects (or of this project without a customer); an additional key (`cmr_…`) sees only itself. Keys appear by `prefix`, `label` and `role`, never ids or secrets, with its usage by category and `amount`, its share of the usage charges at list price. Each usage line of the estimate is split between keys in proportion to their share of that meter (to the cent), so the amounts add up to the usage lines; base fee, minimum, discounts and the complimentary credit stay on the account. A row with `"prefix": null` is usage recorded before per-key attribution. ```json "keys": [{"prefix": "cma_XyZaBc", "label": "billing", "role": "account_admin", "project": "acme-crawler", "active": true, "requests": 9120, "succeeded": 8800, "usage": {"fetch_own": {"requests": 9120, ...}}, "amount": "15.60"}] ``` Projects without a customer get `"customer": null, "plan": null, "estimate": null` (and `amount: null` per key). Categories: `search`, `search_fallback`, `search_cached`, `fetch_own`, `fetch_premium`, `fetch_cached`, `company`, `company_cached`, `sec`, `sec_cached`, `company_profile`, `company_profile_cached`. ### GET /v1/account/invoices `{"invoices": [{"number": "CORM-2026-0001", "status": "issued|paid|void", "period": "2026-09", "period_start", "period_end", "currency", "plan_label", "complimentary", "subtotal", "discount_total", "net_total", "vat_rate", "vat_amount", "total", "vat_note", "issued_at", "due_at", "paid_at", "voided_at", "pdf_url": "/v1/account/invoices/CORM-2026-0001/pdf"}]}` ### GET /v1/account/invoices/{number} and /pdf The invoice with its lines (and tier breakdown), or the PDF. Another customer's number (or a draft) is `404`. ## Company data The company register: one source-attributed record (`company/v1`) per company Cormorant has been asked about, kept and versioned. Sources are official registers, GLEIF and other open, legitimate sources (`GET /v1/companies/sources`). Every data call is metered as **`company`** (billing category and meter `company`; one request per call or batch item). `GET /v1/companies/sources`, `GET /v1/schemas/company` and watch management are not metered. | Endpoint | Purpose | |---|---| | `GET /v1/companies/search?q=&country=&identifier=&status=&legal_form=&limit=&cursor=&live=` | Register matches (trigram/full-text over current, former and trading names; exact on `identifier`) plus live hits from the sources covering `country` on the first page (unless the register has a near-exact hit, score ≥ 0.95; `live=true` always, `live=false` never). `match_score` 0–1, `in_register`, `ref`; `live_sources[]` per source `{source, ok, status, error, ms}` | | `GET /v1/companies/{id}` | The record. `id` = `cmp_…` or `scheme:value`. `max_age`, `refresh`, `include=people,ownership,filings,financials,events,claims`, `sources=` | | `GET /v1/companies/{id}/history?field=` | Claims timeline: every value each source asserted, with `observed_at`, `valid_from`, `valid_to`, `current` | | `GET /v1/companies/{id}/{events,people,ownership,relationships,filings,financials,sources}` | Sub-resources with paging (`limit`, `cursor`); `people` accepts `include_former` | | `POST /v1/companies/resolve` | `{name, country, city, identifiers, isin, persist}` → `match` (confident only; added to the register unless `persist: false`, so it has its `cmp_` `id` and `in_register: true`) and ranked `candidates` | | `POST /v1/companies/batch`, `POST /v1/companies/batch/csv` | Job of kind `company`: `get`, `refresh` or `resolve` per item; JSON or CSV in; results via `GET /v1/jobs/{job_id}` or `GET /v1/companies/batch/{job_id}.csv` | | `GET /v1/companies/changes?since=&cursor=&country=` | Change feed of the register (company and changed top-level fields), oldest first | | `POST/GET /v1/companies/watches`, `DELETE /v1/companies/watches/{id}` | Signed `company.changed` webhooks for a list of companies | | `GET /v1/companies/sources` | Sources: countries, schemes, capabilities, precedence, licence URL, freshness classes, live health | | `GET /v1/schemas/company` | JSON Schema (2020-12) of `company/v1` | **Identifiers** are written `scheme:value` and normalised (spaces, dots and case do not matter; check digits are verified): `ch_uid` (`CHE-101.329.561`), `ch_chid`, `ch_ehraid`, `lei`, `us_cik`, `uk_crn`, `de_register` (`München:HRB:6684`), `fr_siren`, `fr_siret`, `no_orgnr`, `dk_cvr`, `fi_ytunnus`, `ee_regcode`, `cz_ico`, `pl_krs`, `be_kbo`, `au_abn`, `au_acn`, `nz_nzbn`, `jp_corporate_number`, `br_cnpj`, `sg_uen`, `ca_corp`, `vat` (`DE129273398`, `CHE101329561`), `isin` (`CH0038863350`: the issuer, found through GLEIF's ISIN-to-LEI mapping; the ISIN is then kept on the record), `ticker` (`XNAS:AAPL`), `wikidata`. An invalid identifier (e.g. a wrong ISIN check digit) is `422 invalid_identifier`. **Refresh on request.** A request answers from the register. Each source's data about the company is re-fetched first when it is older than the source's freshness class for that field group (e.g. registry core 24 h, people 7 days) or than `max_age` (seconds); `refresh=true` re-fetches everything. The answer's `refresh` lists per source and group what was `refreshed`, `fresh`, `failed` (error class; the register still answers), `not_found`, `in_progress`, `pending` (a refresh slower than 2 s, `error: "deadline"`, or of a source whose canaries fail, `error: "canary_failed"`: it finishes in the background and the next request sees its data) or `skipped` (the source backs off after repeated failures, `error: "backoff"`). Stale sources are refreshed concurrently and the answer waits at most about 2 s for them; batch items wait for every refresh. A company not yet in the register is looked up in the sources that know the identifier's scheme and added; unknown everywhere is `404 company_not_found`, and remembered (ISIN 24 h, other schemes 1 h; `refresh=true` asks again): within that time the sources are not asked and `refresh` shows `not_found` with `error: "cached"`. Companies nobody asks for are never re-scraped; official change feeds (e.g. GLEIF's daily updates) keep the ones in the register current. **Provenance.** Every value is a claim of a source; the record shows the resolved value of each field, `provenance` (source, `observed_at`, `confirmed_at`, `sha256` of the raw document, `conflict`), `quality` (`completeness`, `conflicts` where sources disagree) and `sources[]` (raw documents with `body_url` — downloadable with `GET /v1/bodies/{sha256}` once you have received the record). `include=claims` adds every current claim. Precedence when sources disagree: official register > official gazette > GLEIF > regulatory filings > crosswalks > commercial sources, then each field's declared rule. `names.legal` is the national register's primary entry and does not depend on refresh order (CH: `zefix_rest` > `lindas_zefix` > `uid_register` > `ch_cantonal_excerpt` > `sogc` > `gleif`); other sources' forms are in `names.alternates` (`{name, source}`), language versions in `names.translations` (with `source`). A merged record's `cmp_` id keeps working and answers with the surviving record and `redirected_from`. ```sh curl -s "https://api.cormorant.ch/v1/companies/ch_uid:CHE-101.329.561?include=ownership" \ -H "Authorization: Bearer $CORMORANT_API_KEY" ``` ```json { "schema_version": "company/v1", "id": "cmp_01JA2B3C4D5E6F7G8H9J0KMNPQ", "identifiers": [{"scheme": "ch_uid", "value": "CHE-101.329.561", "issuer": "Federal Statistical Office (BFS)", "status": "active", "parts": null, "sources": ["gleif"]}, {"scheme": "lei", "value": "BFM8T61CT2L1QCEMIK50", "issuer": "GLEIF", "status": "active", "parts": null, "sources": ["gleif"]}], "names": {"legal": "UBS AG", "previous": [{"name": "SBC AG", "lang": "en"}], "trading": [], "translations": [], "alternates": []}, "legal_form": {"local_code": "AG", "local_name": "Company limited by shares", "elf_code": "MVII", "category": "corporation"}, "status": {"value": "active", "local_value": "ACTIVE"}, "jurisdiction": {"country": "CH", "subdivision": null, "register": {"authority": "RA000549", "court": null, "entity_id": "CHE-101.329.561"}}, "addresses": [{"type": "registered", "lines": ["Bahnhofstrasse 45"], "street": "Bahnhofstrasse 45", "house_number": null, "postal_code": "8001", "city": "Zurich", "region": "CH-ZH", "country": "CH"}], "dates": {"incorporated": "1978-02-28", "registered": null, "deregistered": null, "fiscal_year_end": null}, "ownership": [{"type": "direct_parent", "holder": {"name": "UBS Group AG", "identifiers": [{"scheme": "lei", "value": "549300SZJ9VS8SGXAN81"}], "country": "CH"}, "source": "gleif"}], "provenance": {"names.legal": {"source": "gleif", "observed_at": "2026-10-19T08:00:00+00:00", "confirmed_at": "2026-10-19T08:00:00+00:00", "sha256": "9f86…", "conflict": false}}, "sources": [{"source": "gleif", "url": "https://api.gleif.org/…/BFM8T61CT2L1QCEMIK50", "retrieved_at": "2026-10-19T08:00:00+00:00", "sha256": "9f86…", "body_url": "/v1/bodies/9f86…", "content_type": "application/vnd.api+json", "licence": "https://www.gleif.org/en/meta/lei-data-terms-of-use", "parser_version": "gleif/1"}], "quality": {"completeness": 0.778, "conflicts": []}, "refresh": [{"source": "gleif", "group": "core", "status": "fresh"}, {"source": "gleif", "group": "ownership", "status": "refreshed", "at": "2026-10-19T08:00:01+00:00"}], "redirected_from": null, "updated_at": "2026-10-19T08:00:01+00:00" } ``` **Latency.** A company in the register is answered from it: `resolve` with a confident register match and `search` with a near-exact register hit ask no source. Otherwise the live sources are asked concurrently and waited for at most 2.5 s (batch `resolve` items: 12 s); slower sources finish in the background, sources backing off after repeated failures or with failing canaries are not asked. `live_sources` (search and resolve) reports each source as `ok`, `failed` (`error`: error class), `timeout` or `skipped` (`error`: `backoff` or `canary_failed`) with `ms`. A newly stored `resolve` match gets the other sources' data in the background. **Batch.** `POST /v1/companies/batch` takes `{action, items: [{ref} | {name, country, city, identifiers, isin}], include, max_age, persist, idempotency_key, callback_url}`; `POST /v1/companies/batch/csv` takes a CSV body (header `ref,name,country,city,isin,identifiers` or any subset, identifiers separated by `;`) with the same options as query parameters. An item with an invalid ISIN fails alone (`invalid_identifier`). Item results: `{company_id, record}` for `get`/`refresh` (the record as above), `{match, candidates, company_id}` for `resolve`, or `{error}` (`company_not_found`, `invalid_identifier`). **Change feed and watches.** `GET /v1/companies/changes` returns `{items: [{seq, company_id, kind: created|updated|merged|split, fields, country, changed_at}], next_cursor}`; start with `since` (ISO 8601) and continue with `cursor`. `POST /v1/companies/watches` with `{companies: [ids or scheme:value], url}` (HTTPS, public address) returns the watch with its `secret` once (at most 20 watches per project, 1000 companies each). Changes of watched companies are POSTed every minute as `{event: "company.changed", id, watch_id, changes}` with `X-Cormorant-Event`, `X-Cormorant-Delivery` and `X-Cormorant-Signature: t=<unix>,v1=<hex>` (HMAC-SHA256 of `"<t>.<raw body>"`, as for usage alerts). Any 2xx is delivered; otherwise retried after 1, 5, 15, 60 and 180 minutes, then hourly, without skipping changes. A watched record merged into another is followed. **Errors**: `404 company_not_found`, `422 invalid_identifier`, `422 invalid_parameter` (e.g. unknown `include`), `422 invalid_item` (batch); otherwise as for every endpoint (`401`, `403`, `429`). ## Service status Public status of the service for your monitoring, and the page at [`/status`](/status). **No authentication** (an `Authorization` header is accepted and ignored), CORS-enabled, refreshed at most every 15 s and shared by all callers (`Cache-Control: public, max-age=15`, `ETag` / `If-None-Match` → `304`). Limited to 120 requests per minute per client IP (`429` with `Retry-After`). The status never contains host names, addresses, instance URLs, project names, keys or internal error strings — only component ids, statuses, reason codes and aggregate numbers. The status endpoints answer **`200` whatever the service status**: the status is in the body. Alert on the body (`status` is not `operational`), and treat a timeout, a connection error or any other HTTP code as "Cormorant is unreachable" — a status endpoint that answered `503` during an outage could not be told apart from a broken proxy or network in between. | Endpoint | Returns | |---|---| | `GET /v1/status` | overall `status`, every component with `reasons` and `metrics`, incidents and maintenance | | `GET /v1/status/summary` | only `status`, `description`, `updated_at` (for uptime monitors) | | `GET /v1/status/components/{id}` | one component plus the incidents and maintenance naming it (`404` for an id not in this deployment) | | `GET /v1/status/history?hours=24` | worst status per 5 minutes (per hour beyond 24 h, up to 168 h) and uptime per component; overall as `_overall` | ```json { "status": "degraded", "description": "Degraded performance", "updated_at": "2026-10-19T09:41:15+00:00", "window_s": 600, "components": [ {"id": "api", "name": "API", "description": "…", "status": "operational", "reasons": [], "metrics": {}, "group": null}, {"id": "fetch", "name": "Fetch", "description": "…", "status": "degraded", "reasons": ["low_success_rate"], "metrics": {}, "group": null}, {"id": "company_gleif", "name": "Company data: GLEIF", "description": "…", "status": "operational", "reasons": [], "metrics": {}, "group": "company_data"} ], "incidents": [ {"id": "inc_3f9a1c0b7e", "title": "Elevated fetch failures", "status": "identified", "impact": "degraded", "components": ["fetch"], "created_at": "…", "updated_at": "…", "resolved_at": null, "updates": [{"at": "…", "status": "identified", "message": "…"}]} ], "maintenance": [] } ``` **Statuses**, best to worst: `operational`, `maintenance`, `degraded`, `partial_outage`, `major_outage`. The overall `status` is the worst component status, except that a major outage of a component other than `api`, `search`, `fetch` or `storage` counts as a partial outage, and it is `maintenance` when the only components not operational are under announced maintenance. An unresolved incident shows its components at least at its `impact`. Components not part of the deployment are not listed (`archive` when archived copies are disabled, `premium_routes` when no premium route is configured, `sec` while no SEC identity is configured, `company_profiles` while no premium route is configured) — do not alert on a missing component. Counts and rates cover the last `window_s` = 600 s. A component with fewer than its minimum sample there (20 uncached requests; 10 attempts for a tier) reports `requests`/`attempts`, `failures` and `success_rate` over the **last hour** instead; `metrics.window_s` (`<tier>_window_s` in `premium_routes`) says which window was used, and the status is judged on that window. `null` rates mean even the hour has too few samples. | Component | Metrics (rates 0–1; `failures` and `window_s` with every rate) | Not operational when | |---|---|---| | `api` | — | database or cache unreachable (major) | | `search` | `success_rate`, `requests`, `engines_healthy_share`, `engines`, `instances_healthy_share`, `fallback_available`, `fallback_share`, `fallback_success_rate`, `workers_alive`, `queue_lag_s` | success rate of uncached searches < 0.95 / 0.80 / 0.50 (degraded / partial / major); healthy search instances < 50 % (degraded) or none (partial with the fallback search service, major without); healthy engines < 34 % (degraded); no worker; queue lag. `engines_healthy_share`: share of search engines (per instance) whose latest answer in the last 15 minutes returned results (the instance share until searches were observed); `engines`: `[{engine, status}]` with `ok` (healthy everywhere), `degraded` (on some instances) or `down` (rate limited, CAPTCHA or suspended everywhere) | | `fetch` | `success_rate`, `requests`, `latency_p95_ms`, `workers_alive`, `sync_queue_backlog`, `sync_queue_lag_s`, `batch_queue_backlog` | success rate of uncached fetches < 0.85 / 0.60 / 0.30 (4xx answers of the site, robots.txt, invalid or blocked URLs, too large, PDF not accepted and deferred are not failures); p95 > 60 s (degraded); no worker (major); queue lag | | `browser_rendering` | `success_rate`, `attempts`, `workers_alive`, `sync_queue_backlog`, `sync_queue_lag_s`, `batch_queue_backlog` | browser attempts failing for infrastructure reasons: success < 0.80 / 0.50 / 0.10; no browser worker (major) | | `archive` | `success_rate`, `attempts` | archive unreachable or erroring: success < 0.80 / 0.50 / 0.10 | | `premium_routes` | `proxy_success_rate`, `proxy_attempts`, `rendering_success_rate`, `rendering_attempts` (configured tiers only) | provider errors: success < 0.80 / 0.50 / 0.10 | | `sec` | `success_rate`, `requests`, `throttle_wait_p95_ms` (p95 wait for a slot in the cluster-wide SEC rate limiter), `backoff_active`, `rate_limit_rps` | SEC backing off after 403/429/503 (degraded, `sec_backoff`); success rate of uncached SEC requests < 0.85 / 0.60 / 0.30 (4xx, invalid identifiers and not found are not failures); p95 wait > 5 s (degraded, `throttled`) | | `providers` | — | an outside service Cormorant relies on (search fallback, premium routes, alternative sources, notifications) currently refuses our access (degraded, `provider_access`); which one is not shown — the operator is alerted | | `company_profiles` | `success_rate`, `requests`, `latency_p95_ms` (uncached extended company profile operations), `cache_hit_rate`, `refused_share` (operations the source refused: `blocked`, `rate_limited`), `rendering_share` (share served by the rendering service rather than the residential proxy), `throttle_wait_p95_ms` (p95 wait for a slot in the shared rate limit), `backoff_active`, `proxy_skipped` (the residential proxy is being skipped after a recent refusal); always over the last 600 s | success rate of uncached operations < 0.85 / 0.60 / 0.30 (not found, invalid identifiers, robots.txt, premium routes not allowed and exhausted budgets are not failures; `null` below 10); refused share ≥ 0.20 (degraded, `upstream_refusing`); the site throttle backing off after 429 (degraded, `site_backoff`); p95 wait > 10 s (degraded, `throttled`) | | `batch_jobs` | `scheduler_last_run_s`, `workers_alive`, `running_jobs`, `oldest_waiting_job_s`, `queue_backlog` | scheduler last ran > 30 s (degraded) / > 120 s ago (partial); no batch worker (major); a job waiting to start > 15 min (degraded) / > 60 min (partial) | | `storage` | `database_ok`, `cache_ok`, `object_storage_ok` (`null`: not configured) | database or cache unreachable (major); object storage unreachable (degraded) | | `company_<source>` (group `company_data`, one per active company data source, e.g. `company_gleif`) | `success_rate`, `requests` (last hour of calls to the source; `null` below 10), `latency_p95_ms`, `parse_failure_rate`, `canary_ok`, `backoff_active`, `schema_drift` (fields that used to be filled and now come back empty), `change_feed_lag_s` | success < 0.85 / 0.50 / 0.20; canaries (known companies checked every 15 minutes) failing (partial); backoff after repeated failures, schema drift, parse failures > 10 % (degraded). Counts at most as `degraded` for the overall status | Queue lag (`queue_lag_s`, `sync_queue_lag_s`: how long the oldest synchronous request has waited for a worker) above 15 / 60 / 120 s is degraded / partial / major. Rates computed on fewer than 20 requests (10 tier attempts) are reported but do not change a status. New metric keys may be added. **Reason codes**: `no_workers`, `queue_lag`, `low_success_rate`, `high_latency`, `engines_reduced`, `engines_unavailable`, `tier_errors`, `scheduler_stalled`, `jobs_waiting`, `database_unreachable`, `cache_unreachable`, `object_storage_unreachable`, `sec_backoff`, `throttled`, `provider_access`, `upstream_refusing`, `site_backoff`, `incident`, `maintenance`; company data sources also `canary_failed`, `source_backoff`, `schema_drift`, `parse_failures`. **Incidents and maintenance** are posted by the operator. `incidents` lists unresolved incidents (`investigating`, `identified`, `monitoring`) and those `resolved` within 7 days, newest first, each with its timeline (`updates`, newest first). `maintenance` lists `scheduled` and `in_progress` windows and those `completed` within 7 days, soonest first; an empty `components` list means the whole service. **History**: `GET /v1/status/history` returns, per component, `statuses` (oldest first; `null` where nothing was recorded) and `uptime` over the period, where a `partial_outage` bucket counts as 30 % downtime and a `major_outage` bucket as 100 %. Kept 7 days. ## Health and metrics - `GET /health` → `200 {"status":"ok"}` (process is up). For monitoring the service as a whole use [`GET /v1/status`](#service-status). - `GET /ready` → `200` when PostgreSQL and Redis answer and at least one SearXNG instance is registered, else `503`; body lists the checks, never secrets. - `GET /metrics` → Prometheus text (private network only): requests by kind, provider and outcome, cost, bytes, premium bytes, search fallbacks, latency histogram, queue lengths, SearXNG instance count — aggregated across all hosts. ## Examples ```sh export CORMORANT_URL=https://api.cormorant.ch CORMORANT_API_KEY=cmr_... curl -s "$CORMORANT_URL/v1/search" -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H 'Content-Type: application/json' -d '{"query":"Holcim annual report 2025","country":"CH"}' curl -s "$CORMORANT_URL/v1/fetch" -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H 'Content-Type: application/json' -d '{"url":"https://www.holcim.com/investors"}' curl -s "$CORMORANT_URL/v1/batch/fetch" -H "Authorization: Bearer $CORMORANT_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"idempotency_key":"demo-1","items":[{"url":"https://example.com"},{"url":"https://example.org"}]}' ``` ```python import httpx client = httpx.Client( base_url=CORMORANT_URL, headers={"Authorization": f"Bearer {KEY}"}, timeout=180 ) page = client.post("/v1/fetch", json={"url": "https://www.holcim.com/investors"}).json() if page["error"] is None: text = page["text"] ``` Sync calls wait up to 150 s for the result (then `504` with the request id); use the batch endpoints for bulk work. <!-- source: https://www.cormorant.ch/docs/changelog.md --> # Changelog Changes to the Cormorant API, newest first. Within v1, fields are added, not removed; a renamed field or value is announced here with its old name, which requests may keep using for 30 days. The [API contract](/docs/api.md) is the reference for every field mentioned here. ## Unreleased ### Fetch: the residential route fails over between networks - The `proxy` route now runs over several residential networks. When a site refuses an exit or a network fails, the request is retried at once through another one; each try is its own `proxy` attempt with its status, reason and `premium_bytes`. Route names and prices are unchanged. See [Tiers and escalation](/docs/api.md). ### Debug requests (available on request) - `X-Cormorant-Debug: 1` on `POST /v1/search` and `POST /v1/fetch`, for keys that support enabled for debugging: the request is recorded in detail and the answer carries `debug_url`, a link for support. Ignored for other keys. See [API contract](/docs/api.md#authentication). ### Standard and premium routes: renamed fields Every call is billed. Fetches run on **standard routes** (the shared cache, a direct request, a headless browser, archives and alternative copies) and, only when a site blocks those, on **premium routes** (the residential network `proxy` and the rendering service `proxy_render`), which are priced higher. Fields, values and labels that named the routes by their price now use these names. **Requests.** Send the new names now. Until **2026-11-19** the old request field and filter values keep working; after that day they answer `422`. **Responses** use only the new names from this release on: if you read any of the response fields below, switch to the new name. #### Renamed fields | Where | Old name | New name | |---|---|---| | Request field (`POST /v1/fetch`, `/v1/batch/fetch` items) | `allow_paid` | `allow_premium` | | Billing category and plan meter (`billing_category`, `billing.category`, `billing.units[].meter`, `GET /v1/usage`, `?category=`, `GET /v1/requests`, invoices) | `fetch_paid` | `fetch_premium` | | Key quota, allowed category, quota header (`X-Cormorant-Quota-Remaining`), `GET /v1/quota` limit name, alert limit name | `fetch_paid`, `monthly_requests.fetch_paid` | `fetch_premium`, `monthly_requests.fetch_premium` | | Plan meter, included allowance, alert limit name | `paid_gb`, `included.paid_gb` | `premium_gb`, `included.premium_gb` | | Fetch attempts, `GET /v1/usage` (`by_outcome`, providers, keys), account usage | `paid_bytes`, `paid_bytes_ok` | `premium_bytes`, `premium_bytes_ok` | | Fetch result on failure | `paid_tiers_tried` | `premium_routes_tried` | | `GET /v1/usage`, `/v1/usage/timeseries`, `/v1/usage/domains` (JSON and CSV) | `paid_requests`, `paid_share` | `premium_requests`, `premium_share` | | `GET /v1/requests` rows and CSV column | `paid` | `premium` | | Attempt reason | `paid_skipped_domain_refuses` | `premium_skipped_domain_refuses` | | Attempt reason | `paywall_skip_paid` | `paywall_skip_premium` | | Attempt reason (batch items) | `deferred_before_paid` | `deferred_before_premium` | | Error code (also `?error=` of `GET /v1/requests`) | `paid_tier_required` | `premium_route_required` * | | Status component (`GET /v1/status`, `/v1/status/components/{id}`) | `paid_tiers` ("Paid tiers") | `premium_routes` ("Premium routes") * | | Account, plan and invoice flag (`GET /v1/account`, `/v1/account/invoices`) | `free` | `complimentary` * | | Per-request billing block | `free_plan` | `complimentary_plan` * | | Account and invoice line `kind` | `free` ("Free plan: 100 % discount") | `complimentary` ("Complimentary plan: 100 % discount") * | | MCP client tool parameter / environment variable | `allow_paid` / `CORMORANT_ALLOW_PAID` | `allow_premium` / `CORMORANT_ALLOW_PREMIUM` (the old variable is still read) | \* Not a plain `paid` → `premium` swap: tiers became routes, and accounts billed at a 100 % discount are *complimentary*. Not renamed: the payment state of an invoice (`status: paid`, `paid_at`) and the tier names themselves (`direct`, `browser`, `archive`, `proxy`, `proxy_render`, the `tiers` field). ### Extended company profiles: new names in status, usage and billing - Service status (`GET /v1/status`, the status page): extended company profiles are the component `company_profiles`, "Company profiles (extended)". See [Monitoring](/docs/monitoring.md). - Usage and billing name them `company_profile` (billing category and meter; `company_profile_cached` when answered from the cache) and request kind `company_profile`, with the label "Company profiles (extended)": the `company_profile` block and `keys[].company_profile_requests` in `GET /v1/usage`, `spend_budget.*.by_kind.company_profile`, quota and alert limits `included.company_profile`, `GET /v1/requests` rows and its `kind` and `category` filters, `GET /v1/account` and invoice lines. Ledger rows of these requests have `domain: null`. - If you filter or alert on the previous names, switch to these before 2026-11-19; until then the previous filter values keep working. ### Fetch: known hard domains, Chrome fingerprint through the residential proxy - The `proxy` tier now sends GET requests with Chrome's TLS/HTTP2 fingerprint, like `direct`. - The browser tier no longer shows `HeadlessChrome` in its User-Agent or client hints. - A shipped list of known hard domains (`nestle.com`, `nestleusa.com`, `beyondmeat.com`, `apnews.com`, `nytimes.com`, `researchgate.net`, …) sets the route per site: tiers to skip (`hard_domain_skip`), the exit country, the rendering difficulty, and Common Crawl on synchronous calls. - Errors for these sites carry `hint` and `alternative_urls`. See [Fetching → Known hard domains](/docs/fetching.md#known-hard-domains). ### Fetch: busy domains answer fast; polite per-domain bursts - A synchronous `POST /v1/fetch` now waits at most 15 s for one of a busy domain's polite slots. When the expected wait is longer, it answers at once with HTTP 503, `error: "domain_busy"`, `retry_after_s` and `Retry-After`. Batch items are unchanged: they wait in the queue. - New fetch field `domain_concurrency` (up to 4) for a short burst on one site, e.g. a company's own pages. It is ignored while the site pushes back. See [Limits & politeness](/docs/limits.md#per-domain-politeness). ### Service status: counts, failures and a one-hour fallback window Each component now reports its counts (`requests` / `attempts`, `failures`) and the `window_s` they cover; when the 10-minute window is short of the minimum sample, the last hour is used. See [Monitoring → Windows and minimum samples](/docs/monitoring.md#windows-and-minimum-samples). ### Latency breakdown per request and in usage Fetch results gain `timing` (`queue_ms`, `politeness_wait_ms`, `upstream_ms`) and each attempt `wait_ms` / `upstream_ms`. `GET /v1/usage` adds `latency_components` (p50/p95 of each component) per kind, and latency p50/p95 plus component percentiles per tier. ### Caching: redirect targets and short pages - A fetched page is now also cached under the URL its redirects ended at, for example after `http` → `https` or bare host → `www`. A later request for that URL is then a hit. - Pages that are simply short are cached for a day instead of being rendered again every time. These are pages where `direct` and the browser both gave less text than `min_text_chars` and nothing was blocked. - New guide: [Caching and reuse across runs](/docs/caching.md). It covers the cache classes and TTLs, the cache keys, and how to use `cache_ttl_s` so that a recurring job reuses earlier answers. ### Fetch: no premium bytes for domains that refuse every tier A domain whose pages were refused by the premium routes as well (block page, challenge, captcha) in 2 fetches in a row now skips `proxy` and `proxy_render` for 6 h (then 24 h, 3 days): those fetches go to the archive and alternative sources, or fail fast, without premium bytes (attempt reason `premium_skipped_domain_refuses`; 5 % re-probe, one premium success ends it). New fields: `attempts[].premium_bytes`, and on failures `error_detail` (every attempt in one line) and `premium_routes_tried`. `GET /v1/usage` adds `by_outcome` (`succeeded` / `failed`: requests, bytes, premium bytes, premium requests) to each per-kind block. ### Quota: alert e-mail deliverability `GET /v1/quota` has `alert_email` (`status`: `deliverable`, `not_set`, `partially_undeliverable` or `undeliverable`; `addresses`, `undeliverable`, `reason` `bounced` / `complained`, `since`). Alert e-mails go to every address when several are set; an address that hard-bounces or reports a mail as spam stops receiving them. ### SEC EDGAR: "not an SEC filer" versus "unknown identifier" The SEC company endpoints and `/v1/batch/sec` items now say which case a failed lookup is: - `unknown_identifier`: neither SEC nor Cormorant knows the ticker, CIK, name or ISIN; - `not_sec_filer`: the company is known — a listing on a non-US exchange (`NESN.SW`, `XSWX:NESN`), the company register, GLEIF for an ISIN — but is not registered with the SEC under this identifier. The new `hint` carries the `company`, its home `listing` and `sec_candidates`: SEC filers with its name, such as the ADR of a foreign private issuer (Novo Nordisk → `NVO`); - `not_found` keeps its meaning for filings and documents. `resolve` takes `?isin=` (batch: `isin` on `resolve`, `submissions`, `latest`, `facts`). ### Service status: `providers` component and per-engine search health - `/v1/status` and `/status` list a new component, `providers` ("Third-party services"). It is `degraded` with reason `provider_access` while an outside service Cormorant relies on refuses access; which one is not shown. It has no metrics. - The `search` component reports `engines_healthy_share` from the search engines themselves, a per-engine `engines` list (`ok`, `degraded`, `down`) and `instances_healthy_share`. ### What each request is charged, and spend against the caps - Every billable answer (search, fetch, batch items, `/v1/sec/…`, `/v1/companies/…`) has a `billing` block: `charge` at your plan price in your invoice currency, `within_allowance`, `billable` and the units per plan meter, priced like the invoice. - `GET /v1/quota` and `GET /v1/usage` have `spend_budget`: per key and per customer, `cap` (`null` when none is set), `spent`, `remaining` and `by_kind`. - See [Costs & billing → Capping the cost of a run](/docs/billing.md#capping-the-cost-of-a-run). ### Wikimedia: a policy-compliant User-Agent Fetches of Wikipedia, Wikidata and the other Wikimedia sites now identify as `CormorantBot/1.0` with a contact address, per Wikimedia's User-Agent policy. An allowlisted `user_agent` you send is used instead. See [Fetching → Wikipedia, Wikidata](/docs/fetching.md#wikipedia-wikidata-and-other-wikimedia-sites). ### Swiss company data: SOGC events and cantonal register excerpts The `sogc` and `ch_cantonal_excerpt` sources are available: the events timeline from the Swiss Official Gazette of Commerce, and officers and signatories (role, signing authority, since/until, origin or nationality, residence) from the cantonal register excerpts. See [Switzerland](/docs/companies-switzerland.md). ### Companies: ISIN as an identifier - `isin:` works wherever an identifier does (`GET /v1/companies/{id}`, `search?identifier=`, `resolve`, batch `ref`), a bare ISIN too; the check digit is verified (`422 invalid_identifier`). - `resolve` and batch items take `isin` (also as a CSV column). - An identifier no source knows is remembered (ISIN 24 h, other schemes 1 h; `refresh=true` asks again). ### SEC EDGAR usage SEC EDGAR requests (`/v1/sec/...` and fetches of SEC hosts) are metered as kind `sec` in the billing categories `sec` and `sec_cached` (`billing_category`, `GET /v1/usage?category=`, `GET /v1/requests?kind=sec`). `GET /v1/usage` adds a `sec` block, `keys[]` adds `sec_requests`. For key limits, SEC requests count as `fetch`. ### Companies: resolve returns the register id A confident `match` from `POST /v1/companies/resolve` is now added to the register and comes back with its `cmp_` id, `ref` = that id and `in_register: true`. `"persist": false` (batch: `persist` option, CSV: `?persist=false`) keeps the old behaviour. ### SEC EDGAR endpoints A new guide, [SEC EDGAR via Cormorant](/docs/sec-edgar.md), MCP tools `cormorant_sec_company`, `cormorant_sec_filings` and `cormorant_sec_document`, and new `/v1/sec/...` endpoints (see the [API contract](/docs/api.md#sec-edgar-v1sec)): resolve, submissions, latest annual report, filing documents, document fetch, XBRL facts and full-text search. `POST /v1/batch/sec` queues up to 50,000 SEC operations as one job. ### Companies: fast search and resolve `resolve` answers from the register when it has a confident match; `search` does the same for a near-exact register hit unless `live=true`. Live sources run concurrently and are waited for at most 2.5 s; slower ones finish in the background. `live_sources[]` gains `status` and `ms`. ### SEC EDGAR policy for fetches Fetches of `www.sec.gov`, `data.sec.gov` and `efts.sec.gov` follow SEC's fair-access rules automatically: Cormorant sends its own declared identity, only the direct tier is used, and one shared rate limit applies. New errors `rate_limited` and `sec_unavailable` (both with `retry_after_s`), `sec_not_configured` and `redirect_not_allowed`. `/v1/status` lists a `sec` component. ### Companies: stable legal name `names.legal` is the official national register's primary entry, whatever the refresh order. New: `names.alternates` (other sources' forms with `source`) and `source` on `names.translations` items. ### Company data New product area: the company register (`company/v1`). See [Company data](/docs/companies.md). - `GET /v1/companies/search`, `GET /v1/companies/{id}`, its history and sub-resources; `POST /v1/companies/resolve`; batch jobs of kind `company` (JSON and CSV). - `GET /v1/companies/changes` and signed `company.changed` webhooks (`POST/GET/DELETE /v1/companies/watches`). - `GET /v1/companies/sources`, `GET /v1/schemas/company` (JSON Schema). - Metered as the billing categories `company` and `company_cached`. - Status: the group `company_data` lists one component per active source. - MCP: `cormorant_company_search`, `cormorant_company_get`, `cormorant_company_resolve`. ### Account admin keys, usage per key - API keys have a role, fixed at creation and visible in the prefix: account admin keys start with `cma_`, standard keys with `cmr_`. Every authenticated `/v1` response carries `X-Cormorant-Key-Role`. - Requests are attributed to the API key that made them; `GET /v1/usage?group_by=key` and `GET /v1/account` show usage per key. ### Cache revalidation New fetch field `revalidated`: `true` when an expired cache entry was confirmed unchanged by the origin (conditional GET answered `304`). The cached copy is returned with `cached: true` and no cost. ### Usage, charges and keys - `GET /v1/usage` with `group_by` (`key`, `project`, `category`, `day`, `hour`), filters, totals, failures by error, latency, and the `charge` at your plan price; CSV with `Accept: text/csv`. - A standard key sees only its own usage; an account admin key the whole customer account. - `GET /v1/keys`, `GET /v1/keys/self`, `GET /v1/usage/timeseries`, `GET /v1/usage/domains`, `GET /v1/requests` (request ledger) and `GET /v1/requests/{request_id}`. - Per-key limits (monthly spend cap, monthly request quotas, requests per second, allowed categories), `GET /v1/quota`, rate-limit headers and usage alerts (`POST/GET/DELETE /v1/alerts`, signed webhooks). - New guide [Usage, spending and quotas](/docs/usage.md) and the MCP tool `cormorant_usage`. ### Charges, not provider costs - **Changed for customer accounts:** `cost_usd` is `null` for keys of a customer account; what you are charged is in `GET /v1/usage` and `GET /v1/requests` (`charge`). - New `billing_category` on every search and fetch result. ### Dead domains, archived copies for missing pages - A host whose name does not exist answers `error: "domain_dead"` at once for 6 hours. - With `allow_archive` (the default), a fetch that ends in `domain_dead` or `http_404` / `http_410` returns a recent Wayback snapshot when one exists (`served_from_archive_because`). ## 2026-10-19 ### Large PDFs in batch fetches Batch fetch items that leave `max_bytes` unset accept PDFs up to 200 MB. New response fields `pdf_page_count` and `text_truncated`. ### Service status `GET /v1/status`, `/v1/status/summary`, `/v1/status/components/{id}` and `/v1/status/history` — no API key needed — and the status page at [`/status`](/status); see the [monitoring guide](/docs/monitoring.md). ### Article metadata Fetch results have a `meta` object for HTML pages: `author`, `published_at`, `modified_at`, `language`, `canonical_url`, `publisher`, `site_name`, `description`, `section`, `word_count`. ### Developer portal Documentation at `/docs`: guides, an interactive API reference and the API contract; [`/llms.txt`](/llms.txt) and [`/llms-full.txt`](/llms-full.txt); every guide as Markdown at `/docs/<page>.md`; an [MCP server](/docs/mcp.md) for AI agents. ### Accounts, plans and invoices `GET /v1/account`, `GET /v1/account/invoices`, `GET /v1/account/invoices/{number}` and `…/pdf`. Keys of a suspended account get `403 customer_suspended`. ### Unavailable domains, honest empty searches - A domain where every tier failed in 3 consecutive fetches answers `error: "domain_unavailable"` for a while, with `retry_after_s`. - Search results carry `unresponsive_engines`; a search that found nothing is `results: []` with `error: null`. ### Rendering tier: retries and difficulty Busy `proxy_render` answers are retried once (two `attempts`), and the difficulty that worked for a domain is remembered; such attempts carry `detail` with the difficulty used (`low`, `medium`, `high`). ### Faster synchronous calls under batch load `POST /v1/search` and `POST /v1/fetch` run on dedicated queues, so large batch jobs no longer push synchronous calls into timeouts. ### Per-request fetch controls New fetch fields: `allow_premium`, `tiers`, `timeout_s`, `country`, `user_agent` (allowlisted) and `accept_language`. New fetch errors: `timeout`, `no_tier_allowed`. ### Cache hits honour the request A cached page is used only when it satisfies the request (`render`, `min_text_chars`, `max_bytes`, `tiers`). ### Provenance, content kinds and exact bytes Fetch results carry `redirects`, `headers`, `origin_content_type`, `content_kind`, `body_url` and `body_kind`; `GET /v1/bodies/{sha256}` streams the exact bytes to projects that retrieved them. ## 2026-10-18 ### v1 Search with SearXNG first and a fallback search API, fetch with tier escalation (direct, browser, archive, residential proxy, rendering tier), shared caches, batch jobs with idempotency keys and callbacks, per-project keys, rate limits, budgets and `GET /v1/usage`.