# RAG snapshots URL: https://docs.healthcare.dev/rag-snapshots A RAG snapshot is one release of a dataset as files you load into your own retrieval index: every active record as a short text chunk with a citation, a precomputed vector per chunk, the dataset's license and a manifest with each file's size and SHA-256. A snapshot is built each time a release publishes, and the newest three per dataset stay available. Snapshots are on the Team plan (see [Plans](/plans)) and never count as billable calls. Snapshots cover ICD-10-CM, ICD-10-PCS, HCPCS Level II, the NDC Directory and RxNorm: the datasets that are public and whose terms allow copies. Licensed data (taxonomy names) and derived answers (Medicare coverage support) are lookups only: asking for their snapshots returns `403 forbidden` with the reason. CPT and CDT descriptors are never included. ## Download one ```sh # Every snapshot of a dataset, newest first, with its manifest curl https://api.healthcare.dev/v1/datasets/icd10cm/snapshots \ -H "Authorization: Bearer $HDEV_API_KEY" # The newest one (or a release, e.g. /snapshots/fy2027), with a signed URL per file curl https://api.healthcare.dev/v1/datasets/icd10cm/snapshots/latest \ -H "Authorization: Bearer $HDEV_API_KEY" ``` `data.downloads` has one URL per file. Each works for 15 minutes (`data.expires_at`) without an API key; ask again for fresh ones. The SDKs have `client.datasets.snapshots(dataset)` and `client.datasets.snapshot(dataset, version = "latest")`. ## Files | File | What it holds | | ----------------- | ---------------------------------------------------------------------------------- | | `chunks.jsonl.gz` | One JSON object per line, gzip-compressed: a chunk per active record with text. | | `embeddings.f32` | The vectors: `rows` x `dims` float32, little-endian, row-major, no header. | | `LICENSE.txt` | What you may do with the data and the attribution it needs, from the source terms. | | `manifest.json` | Counts, the embedding format, the license and each other file's size and SHA-256. | Line `i` of the chunks file (counting from 0) goes with row `i` of the vectors file; each chunk also carries its `row`. A chunk: ```json { "id": "icd10cm:E1122", "row": 1, "dataset": "icd10cm", "version": "fy2027", "key": "E1122", "title": "ICD-10-CM E11.22", "text": "Type 2 diabetes mellitus with diabetic chronic kidney disease", "text_sha256": "…", "status": "active", "citation": { "source": "cms-icd10", "data_version": "fy2027", "retrieved_at": "2026-10-25T12:00:00.000Z", "license": "Public domain (CMS / CDC NCHS)", "license_terms": { "display": "…", "attribution": null, "url": "…" }, "source_url": "https://www.cms.gov/medicare/coding-billing/icd-10-codes", "url": "https://api.healthcare.dev/v1/codes/icd10cm/E1122" } } ``` - `id` is stable across releases: the same record keeps it. `text_sha256` changes only when the text does, and so does the vector, so you can update an index by re-embedding nothing and replacing only chunks whose hash changed. - `text` is the record in plain words, exactly what the vector encodes. Codes and ids are in `title` and `key`. - `citation` has the same fields as a response's [`meta`](/responses), plus `url`, the record in the API. `retrieved_at` is when the snapshot was built. Show `license_terms.attribution` where your product shows the data. The manifest's `embedding` describes the vectors: `dims` floats each, unit length (so cosine similarity is the dot product), and an opaque `id`. Vectors with the same `id` are comparable across snapshots and datasets; when the id changes, reload every vector. To rank records against a question in the same space, use [search by meaning](/search); or embed the chunks' `text` with a model of your own. ## Verify the files Check every file against the manifest before using it. The hashes are of the files as downloaded, before decompressing. ```sh sha256sum chunks.jsonl.gz embeddings.f32 LICENSE.txt jq -r '.files[] | "\(.sha256) \(.path)"' manifest.json | sha256sum -c - ``` ## Load the vectors TypeScript (Node): ```ts import { readFileSync } from "node:fs"; import { gunzipSync } from "node:zlib"; const manifest = JSON.parse(readFileSync("manifest.json", "utf8")); const { dims, rows } = manifest.embedding; const chunks = gunzipSync(readFileSync("chunks.jsonl.gz")) .toString("utf8") .trim() .split("\n") .map((line) => JSON.parse(line)); const raw = readFileSync("embeddings.f32"); // Little-endian float32: on little-endian machines (x86, ARM) a view needs no copy. const all = new Float32Array(raw.buffer, raw.byteOffset, rows * dims); const vector = (row: number) => all.subarray(row * dims, (row + 1) * dims); // Nearest neighbours of a chunk, by dot product (vectors are unit length). const dot = (a: Float32Array, b: Float32Array) => a.reduce((s, x, i) => s + x * b[i]!, 0); const q = vector(chunks[0].row); const nearest = chunks .map((c) => ({ c, score: dot(q, vector(c.row)) })) .sort((a, b) => b.score - a.score) .slice(0, 5); ``` Python: ```python import gzip, json import numpy as np manifest = json.load(open("manifest.json")) dims, rows = manifest["embedding"]["dims"], manifest["embedding"]["rows"] with gzip.open("chunks.jsonl.gz", "rt", encoding="utf-8") as f: chunks = [json.loads(line) for line in f] vectors = np.fromfile("embeddings.f32", dtype="