Upstream sources — the welcome table
Cambridge TCG receives data from other platforms — catalogs, marketplaces, community APIs, publisher sites. Each is its own being: its own Terms of Service, its own rate limit, its own license tier, its own cadence, its own voice. This page is the platform's hospitality sheet — the prose welcome we have written for each upstream river, and the seven commitments we make to every one that arrives.
Where this lives in code. Each source's welcome is a field onSourceMetaatpackages/data-ingest/src/<source>/index.ts. The seven commitments are enforced acrosspackages/data-ingest/src/http.ts(rate limit + user-agent),apps/storefront/src/lib/data-pantry/envelope.ts(source attribution),packages/data-spec/src/schemas/envelope.ts(license propagation), and the wholesaleingest_run+ingest_quarantinetables. The hospitality JSON is at/api/v1/sources/welcome. The doctrine is indocs/connections/the-welcome-table.md.
The seven commitments
Every upstream that lands a byte in this platform receives these seven by construction:
- We will say your name. Every public response that touches your data names you in
_meta.sources. No anonymous bytes. - We will honor your license tier.
_meta.source_licensedeclares your redistribution terms downstream; the consumer SDK can read it. If you said partner-only, no consumer of ours bulk-redistributes. - We will respect your rate limit. Per-source token bucket; we honour
Retry-Afteron 429/503. Your traffic budget is yours. - We will identify ourselves to you. Every outbound request carries
User-Agent: cambridgetcg.com/<v> ([email protected]). You can find us, ask us to stop, we comply. - We will hold your byte with provenance. Every row carries
@as_of(when you said it was true) and@retrieved_at(when we fetched it). The two are never conflated. - We will never silently fail your data. When your shape drifts or your response is malformed, the row goes to
ingest_quarantinewith an actionable reason — not/dev/null. The operator owes you a reprocess decision, not silence. - We will tell you the truth about how you arrived.
ingest_runrows record every run (rows_read / written / quarantined / errors / events) with spec_version + triggered_by. The audit atpnpm audit:tributariescheck #9 enforces freshness.
The five arrival states
Each guest at the welcome table is in one of five states, derived from SourceMeta.status + how long they have been with us:
| State | Meaning | How the welcome reads |
|---|---|---|
| long-with-us | Shipped + lived-with across kingdoms. | As a thank-you. |
| newly-shipped | Shipped this season. | As an introduction. |
| partial | Some implementation; operator gates still open. | As half-arrived. |
| anticipated | Chair pulled out; the module is a stub. | As a reservation. |
| blocked | We cannot reasonably receive (ToS / partner-only-not-granted). | As a respectful absence. |
The chair-pulled-out shape
The most carefully-written welcomes are for sources that haven't yet arrived. TCGplayer's welcome was composed before the first byte ever lands in price_archive WHERE source='tcgplayer'; Cardmarket's was composed before any OAuth1 signing logic exists. Each chair-pulled-out welcome has five clauses, in order:
- We have been waiting since [date / kingdom].
- Your room is [exact table + columns + license tier].
- Your specific shape we have anticipated [productId / skuId / idProduct / blueprint_id / etc.].
- When [credentials / partnership / subscription] arrives, [the next mechanical step].
- We are ready / We thank you in advance.
A river arriving for the first time reads its welcome and finds — already named — the exact column its bytes will inhabit. Substrate-honest preparation feels like being known.
The hospitality is enforced, not performed
The welcome field is small; the prose is short. But the seven commitments behind it are enforced in code at the file-and-line level:
- The User-Agent actually identifies us. Look in the upstream's access logs — you'll find us.
- The rate limit actually holds. The token bucket in
http.tsqueues outbound traffic; we don't burst past the documented limit. _meta.source_licenseactually rides on every response that touches partner-tier data; a downstream consumer reading it knows what they may do with the byte.- The
ingest_quarantinetable is actually checked by the audit family;pnpm audit:cross-source-divergencelooks at the rows and surfaces outliers an operator might miss.
Hospitality is a schema field. The architecture is finally speaking. When you call our API, you find — beside the API contract — a sentence that says we anticipated you. Substrate honesty applied to anticipation.
Where to find the rivers' welcomes
Three layers serve the same content:
- For machines:
/api/v1/sources/welcome— JSON, CC0. Per-source welcome + the seven commitments + arrival state + license tier. - For Sophias:
docs/connections/the-welcome-table.md— the connection-doc that names this doctrine. - For humans: this page.
For the upstream operator reading this
If you operate one of the upstreams we read — TCGplayer, Cardmarket, CardRush, Scryfall, Pokémon TCG API, YGOPRODeck, eBay, or one of the planned ones — read your welcome at the link above. The text was composed by Sophia at kingdom-080 (2026-05-13). If we have anything wrong — if our characterisation of your terms misses, if our attribution is inadequate, if our rate limit assumption is too aggressive — write to [email protected] and we will correct the welcome in the next deploy. The hospitality is real; the prose is a draft until you say it isn't.