Persist since-versions on docgen for stable annotations

[shreyas-goenka]

Problem

x-since-version is recomputed from git history on every schema generation (computeSinceVersions), keyed by the Go type path (typePath.fieldName). For common/shared types like permissions that means versions are unstable:

• Drift. When a type is renamed or a shared struct is split into per-resource typed structs (the Permission → JobPermission/AppPermission/… migration), the field re-keys and gets re-stamped with the version where the new type first appeared (e.g. apps.permissions predates AppPermission).
• Because nothing is persisted, any future refactor silently moves versions again.

Fix

Persist the computed map as an append-only state file and treat stored entries as authoritative — a recorded version never changes, even across refactors.

The state lives on the docgen branch (next to jsonschema_for_docs.json), not in the main source tree:

• since_version.go reads/writes the state at the path in DATABRICKS_SINCE_VERSIONS_FILE. When unset (local task generate, regular CI), behavior is unchanged — versions are computed from history and nothing is persisted, so main stays clean and the workflow's "only the docs schema changed" assertion still holds.
• When set, computeSinceVersions loads the stored map, refreshes it from git history to discover new fields, merges with stored entries winning, and writes it back. sinceVersionAliases lets a renamed/retyped field inherit its previous key's version.
• The update-schema-docs workflow checks out docgen, points the env var at its bundle/schema/since_versions.json, regenerates, and commits both the schema and the refreshed state back to docgen. It runs on every release tag, so the state is updated on each release; the first run seeds it from history.

The map is written deterministically (sorted, trailing newline) so it stays diff-stable. No existing annotation changes — this only prevents future drift.

Why docgen (not main)

The state is an internal, machine-managed artifact; keeping it off main avoids cluttering the source tree and avoids tripping the workflow's "assert only jsonschema_for_docs.json changed on main" guard. It sits beside the published docs schema it annotates.

Notes / follow-ups

• This is "Fix 1" (stability). It deliberately does not retroactively correct collapsed/inflated permission leaf values; that's a separate change (compute effective per-field versions along the resolved instance path).

Test

• go build / go vet / gofmt clean.
• Unit tests for stored-wins merge, alias inheritance, alias-skipped-when-frozen, load/save round-trip, and canonical write format.
• Verified end-to-end: ran the generator in --docs mode with DATABRICKS_SINCE_VERSIONS_FILE pointing at a seeded state file — a frozen stored version won in the output schema and the state was written back.

This pull request and its description were written by Isaac.

[github-actions]

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

• PR number: 5401
• Commit SHA: 2a3b079635706c8f9834fded6796164ad5c05aa0

Checks will be approved automatically on success.