Sampling dimension values
Sampled values let a request name a value and have it resolve. Knowing that Region holds California is what connects “revenue in California” to a filter; without it, a caller — or a model writing a query spec — is guessing at the spelling of something it has never seen.
zsql sample reads those values from the warehouse and writes them to samples/values.jsonl.gz. The file travels with the model on deploy, and zsql values, GET .../values/{dim} and the MCP values tool read it back.
The three field keys
Sampling is off until a dimension asks for it:
| key | default | means |
|---|---|---|
enable_sampling | false | collect this dimension’s values |
sample_size | 1000, at most 20000 | how many values, most frequent first |
sample_sql | none | your own query instead of the generated one: the value in the first column, an optional count in the second |
- type: dimension
name: Region
data_type: string
description: Sales region the order was booked in.
enable_sampling: true
expression:
sql: region
enable_sampling: true is obeyed whatever the field is. There is no type rule and no third state: the modeller knows the data and the loader does not, so the loader does not guess and does not argue. A sample_size over 20,000 is refused at load, naming the field and the bound, rather than clamped — one greedy dimension would otherwise push the file past the size at which it is read at all, and cost every other dimension its values.
What to enable, and what not
Enable sampling only on a string dimension whose values are short — under 255 characters, and in practice far shorter: a status, a category, a region, a channel, a product name.
Leave it off everywhere else:
- Long or unbounded text —
notes,comments,description,varchar(max),text. This is the mistake to avoid: prose in the catalogue matches every request a little and nothing well, which makes resolution worse than having no samples at all. - High-cardinality keys and identifiers — an id, an order number, an email, an account number. Millions of values nobody will ever name, and customer data you did not need to copy.
- Numbers, dates and booleans. A request filters these by range, not by value.
- Anything a security policy covers, and anything sensitive.
Nothing enforces this, but zsql sample warns three ways, naming the field each time. Two of them read the catalog before spending a query:
| warning | from |
|---|---|
declared 2048 wide, past the 255 AGENTS.md advises; check it is a value and not text | the column’s declared length in the catalog |
declared TEXT, unbounded text; check it is a value and not prose | a text or clob column with no declared length |
the values average 412 characters, which is text, not a value; sampling it will blur resolution — turn enable_sampling off, or give it a sample_sql that trims | the values that came back, averaging over 100 characters |
A warning means the dimension wants turning off, or wants a sample_sql that trims and filters what it returns. There is no truncation length and no format detection: a modeller who needs either writes the query.
The same guidance is written into the AGENTS.md that zsql init generates, so an agent editing the model reads it too.
zsql sample
$ zsql sample --dry-run
1 dimension: 2 queries
Contact Subchannel ID on Contact planned, up to 1000 values
nothing run, nothing written
$ zsql sample --yes
1 dimension: 2 queries
Contact Subchannel ID on Contact planned, up to 1000 values
sampled Contact Subchannel ID (cs): 6 values of 6
1 dimension -> samples/values.jsonl.gz
that file is customer data: it ships with the model on deploy and is gitignored by default; committing it is a decision about where those values may go
It spends warehouse time, so it says what it will run before it runs it and asks on a terminal. --yes skips the question; --budget N refuses before running anything when the plan needs more than N queries (100 when unsaid), which is the guard against a model where sampling was turned on more widely than anyone meant:
1 dimension: 2 queries
error: 2 queries is more than the budget of 0; narrow by naming dimensions or tables, or raise --budget
| flag | does |
|---|---|
NAMES... | only these dimensions or tables, by name or uid; every sampled dimension when none |
-d DATASOURCE | one datasource; every one when unsaid |
--dry-run | say what would be run and run nothing |
--budget N | refuse before running when more than N queries are needed |
-y, --yes | run without asking |
--json | the plan and the outcome as JSON — counts only; the values go to the file, never to stdout |
Two queries per dimension: one for the values, one for the distinct total.
The query it generates
A dimension with no sample_sql goes through the planner like any other query. The command writes one expression line per dimension:
<dimension>, calc(count(*), integer) as n desc, limit <sample_size>, hints (<lookup tables>)
and runs the statement the planner returns, on the datasource the planner chose. Four things about that are worth knowing.
Which table answers is the planner’s decision. The table a dimension is declared on is not necessarily the one it should be read from — a partitioned table may be a fresher copy of the same rows. The planner already ranks candidates by partitions, lookups and row estimates, so the command passes the tables whose expression for that dimension is marked lookup as hints and lets the planner choose. A calculated dimension, an array, a plain sql expression are all the planner’s to render, so none is skipped.
What it skips is only what the model forbids — a hidden dimension, and one a security policy names, which is reported as named by the security policy "..."; its values are what the policy protects. A planner refusal on one dimension is a warning on that dimension, not a reason to stop the run.
COUNT(*) counts the rows of the table the dimension lives on. On a dimension table that is dimension rows, so a near-unique column returns n = 1 for every value and “most frequent first” degenerates into arbitrary order. Weighting by a fact table needs a join this template cannot express; a fact-weighted count is a later question. Where it matters, write sample_sql.
Ties are the warehouse’s. The order is the count alone, so which of several equally frequent values a limit cuts is whatever the warehouse returns. The written file is sorted by count then value, so two runs over the same data produce the same record.
sample_sql
sample_sql: |
SELECT category, COUNT(*) FROM products WHERE category IS NOT NULL GROUP BY 1 ORDER BY 2 DESC
Value in the first column, optional count in the second. This is the escape hatch for a dimension that needs trimming, filtering or a join the template cannot write.
The file
One JSON object per line, gzipped:
{"uid":"contact-subchannel-id","name":"Contact Subchannel ID","table":"cs","sampled_at":"2026-10-24T11:18:33Z","sample_size":1000,"distinct":6,"values":[{"v":"SC1","n":1530},{"v":"SC4","n":1238}]}
distinct is the column’s total; values is what came back; sample_size is the limit the query ran with. Completeness is inferred from the three: a list as long as the limit is the sign that the sample is truncated rather than whole, which is what 6 of 6 values versus 1000 of 48217 values reports.
zsql values
$ zsql values 'Contact Subchannel ID'
contact-subchannel-id 6 of 6 values
SC1 1530
SC4 1238
SC5 872
SC2 769
SC6 356
SC3 235
By name or uid, with the same near-match resolution as the other field lookups: an unmatched term gets no dimension matches '...' and an ambiguous one is listed. --json prints the record. With -r it asks the server’s deployed copy instead of the file here.
zsql check counts what it found:
validated customer-service/working-tree: 3 tables, 110 fields, 2 joins, 136 paths, 0 policies, 1 sampled dimensions
That count is dimensions with values in the file, not dimensions with the flag on — so 0 after turning the flag on means zsql sample has not run yet.
Over the API
A deployed branch serves what it was given:
GET /projects/{uid}/branches/{branch}/values/{dim}
The answer is the same record zsql values --json prints. A branch with no samples answers 404 NotFound with no sampled values for <uid>, which is a different thing from a dimension that does not exist — the branch summary’s values count says which. See Discovery and the MCP server, whose values tool is the same read.
Deploying
zsql deploy sends samples/values.jsonl.gz with the model files and the tests. Secrets are stripped; sampled values are not, because serving them is the point. If those values may not leave your machine, do not sample the dimensions that hold them.
What this is for
Right now sampled values are a lookup: a caller or an agent asks what a dimension holds and writes a filter that resolves. Feeding them into how a model answers a question is the next step, and nothing in the format assumes it yet.