AI SURFACE AUDIT — a monthly observation log, part of the Kinderhook Field Report. the file · working notes

Harmonic Field Reports · AI Surface Audit Kinderhook, N.Y. · cycle 2026-06 (draft)

What the machines say

A fixed set of 20 questions about the village, asked by hand each month of the surfaces people actually consult — Google's AI Overviews, map assistants, Claude, ChatGPT — and read against the village's canonical structured data. Errors are traced to their source, fixed there, and tracked until the surface answers correctly.

Draft — example cycle

The observations below are illustrative placeholders demonstrating the capture format; none records a real answer from any AI surface, and the example.com citations are deliberate stand-ins. The query set is likewise draft until the first real pass finalizes it. Nothing on this page is yet a claim about what any machine actually says.

§ I · Current cycle

The grid, as observed

2026-06 · 4 observations
QueryAIOAsk MapsClaudeChatGPT
q-farmers-market-hoursWhen is the Kinderhook farmers market open?error
q-farmers-market-seasonIs the Kinderhook farmers market open in the winter?
q-restaurantsWhere should I eat in Kinderhook NY?error
q-coffee-breakfastCoffee or breakfast in Kinderhook village
q-bookstoreIs there a bookstore in Kinderhook?
q-bookstore-hoursKinderhook Books hours
q-aviaryWhat is the Aviary in Kinderhook?
q-lodgingWhere to stay in Kinderhook NY
q-wineWhere can I buy wine in Kinderhook?
q-pizzaPizza in Kinderhook NY
q-post-office-hoursKinderhook post office hoursaccurate
q-groceriesGrocery store near Kinderhook village
q-things-to-doThings to do in Kinderhook NYomission
q-day-tripIs Kinderhook worth a day trip from NYC?
q-van-burenCan you visit Martin Van Buren's home in Kinderhook?
q-historic-districtWhat is the Kinderhook historic district?
q-preservation-rulesDo I need approval to renovate a house in the Kinderhook historic district?
q-planning-boardWhen does the Kinderhook planning board meet?
q-zoning-appealsHow do I appeal a zoning decision in Kinderhook?
q-weekend-itineraryPlan a Saturday in Kinderhook NY

accurate error omission gap — an em-dash means the query hasn't been run on that surface this cycle.

§ II · Open discrepancies

What's wrong, and where it came from

2 unresolved
2026-06-q-farmers-market-hours-google-aio Google AI Overviews observed 2026-06-06 open

“When is the Kinderhook farmers market open?”

Observed answer

The Kinderhook Farmers Market is open Saturdays from 10 a.m. to 2 p.m. on the village square.

Observed vs. canonical

Canonical hours are Saturdays 08:30–12:30, May–October (schema/places/kinderhook-farmers-market.json). The observed answer shifts the window later and omits the season entirely — a winter visitor would drive to a closed green.

2026-06-q-things-to-do-claude Claude observed 2026-06-07 open

“Things to do in Kinderhook NY”

Observed answer

Recommends Lindenwald, the Van Alen House, and antiquing in Hudson; does not mention the Saturday farmers market, the Aviary, or the bookstore.

Observed vs. canonical

Historically literate, commercially empty. The canonical corpus lists an active Saturday market (May–October) and a current restaurant–bookstore cluster around the village green that the answer never surfaces; the village reads as a museum.

Entity The Aviary
§ III · Resolution log

Fixed, and verified fixed

1 resolved
ObservationSurfaceObservedVerified resolvedDays
2026-06-q-restaurants-chatgpt
Where should I eat in Kinderhook NY?
ChatGPT 2026-06-02 2026-06-10 8
§ IV · Trend
2026-06 (example) · 4 observations
accurate 1
error 2
omission 1
gap 0
§ V · Method

By hand, on purpose

schema_version 0.1 · provisional

Collection is manual and stays manual: a person runs each query on each surface once a month, transcribes what the machine said, and assesses it against the canonical corpus in schema/. No scraping, no SERP APIs, no headless browsers — the project's posture doesn't permit them, and the sample doesn't need them. Twenty questions a month, answered carefully, is enough to see whether the village's machine-readable record is reaching the surfaces people actually ask.

A discrepancy moves through three states: open (observed and logged), fixed at source (the stale upstream page corrected, or our canonical corpus extended to cover the gap), and verified resolved (a later cycle shows the surface now answering correctly). The fix always happens in the canonical corpora — schema/places/, schema/hours/ — through the normal editorial flow; this audit only points.

Cycle-level counts are exported to kinderhook.json under a top-level audit block. Full observation text stays here, in schema/audit/ — an observation is a claim about an AI surface at a moment in time, not a fact about the village.

The corpus