[SEA-NodeJS] SEA query parameters, metadata & getInfo#412
Conversation
Wire three SEA session-backend capabilities onto the merged-kernel napi surface, all as thin adapters over the kernel (which owns the heavy lifting): - Bound parameters: forward `ordinalParameters`/`namedParameters` to the kernel's `ExecuteOptions.positionalParams`/`namedParams`, reusing the existing `DBSQLParameter.toSparkParameter` codec (DECIMAL→DECIMAL(p,s), INTERVAL→INTERVAL, NULL→VOID). Positional and named are mutually exclusive. - Metadata: `getCatalogs`/`getSchemas`/`getTables`/`getColumns`/`getFunctions`/ `getTableTypes`/`getTypeInfo`/`getPrimaryKeys`/`getCrossReference` forward to the kernel `Connection.list*`/`get*` calls and wrap the returned Statement. The kernel owns SQL synthesis, column projection, and the client-side `TABLE_TYPE` filter — the driver only maps request fields to arguments. - getInfo: synthesized client-side (no SEA/kernel endpoint), mirroring the three TGetInfoType values the Databricks Thrift server answers and rejecting the rest, byte-identical to the Thrift path. `SeaInputValidation` is slimmed to the one bind-time gate the driver needs (`assertBindableValue`); marker counting is the kernel's job. Re-exports the kernel's `ExecuteOptions`/`TypedValueInput` types from `SeaNativeLoader` rather than re-declaring them, so the driver param shapes can't drift from the kernel. Regenerates the committed napi `.d.ts` snapshot against the merged kernel. Validated against a live warehouse: positional + named params, mutual-exclusion rejection, all metadata calls (incl. kernel-side tableTypes filter), and getInfo. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Code-review-squad #412 review (79/100). Validated each finding against the kernel source + a live warehouse before fixing: - F1 (HIGH): getPrimaryKeys turned an omitted catalog into `?? ''`, which the kernel's `Identifier::new` rejects with an opaque "identifier must not be empty" — a regression vs Thrift (which forwards undefined). The napi `getPrimaryKeys(catalog: string, …)` requires a non-empty catalog, so reject up front with a clear, actionable message and document the divergence. Added a test (omitted + empty-string catalog) asserting the kernel call is never reached. (getCrossReference is unaffected — its parent args are nullable and passed through without `?? ''`.) - F3 (MED): mutual-exclusivity of ordinal/named params threw HiveDriverError with a different message than the Thrift path, which throws ParameterError('Driver does not support both ordinal and named parameters.') (ParameterError extends Error, not HiveDriverError — so a caller catching it worked on Thrift and silently failed on SEA). Now throws the identical ParameterError + message. Test updated. - F2 (MED): empirically re-verified against the live warehouse over Thrift — the server returns the exact same three values we synthesize ("Spark SQL" / "Spark SQL" / "3.1.1") and errors on every other TGetInfoType (probed CLI_MAX_DRIVER_CONNECTIONS, CLI_DATA_SOURCE_NAME, …). So throwing on an unsupported type matches Thrift's effective behaviour rather than narrowing it; softened the "byte-for-byte" assertion into the validated fact in both the SeaServerInfo and getInfo doc comments. - F4 (LOW): getInfo's rejection now names the accepted enum identifiers/values (CLI_SERVER_NAME (13), CLI_DBMS_NAME (17), CLI_DBMS_VER (18)) so an agent/SDK consumer can self-correct. - F5 (LOW): metadata + bound-param execute failures now emit a debug-level breadcrumb (mapped error + session id) before throwing, via a shared `logAndMapError` helper, matching the rest of the SEA backend's logging. - F7 (LOW): dropped the vestigial `nativeStatement!` non-null assertion in executeStatement (typed the local, mirroring runMetadata). - F8 (test): added coverage for the getPrimaryKeys omitted-catalog path (F1), the INTERVAL *→INTERVAL collapse, and the DECIMAL precision clamp-to-38. F6 (kernel diagnostic getters unconsumed) is tracked as a follow-up — wiring them in must land with the errorDetailsJson size-guard, out of scope here. Validated live: getPrimaryKeys(with/without catalog), the ParameterError parity, and the named-identifier getInfo error all behave as asserted. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
The CI "Check code style" step runs `prettier . --check` (whole repo), not just eslint; these SEA test files were committed without prettier formatting and failed it. Formatting-only; no behavioural change. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
|
🟠 P1 (Important — should fix)
🟡 P2 (Minor)
|
…ic, drop leading-zero miscount Code-review #412 (gopalldb P1). `decimalPrecisionScale` derived precision by counting digit characters, which (a) counted insignificant leading zeros ("0.00001" → DECIMAL(6,5)) and (b) clamped precision to 38 while still sending the full value — so a 40-digit value produced DECIMAL(38,0) alongside an unrepresentable 40-digit value, which the kernel rejects or silently truncates. The prior test blessed that clamp as correct (wrong contract). Now: - Precision = significant integer digits (insignificant leading zeros stripped, matching Spark's decimal-literal rule) + fractional scale. "0.00001" → DECIMAL(5,5); "007.50" → DECIMAL(3,2); "0" → DECIMAL(1,0). - A value needing precision > 38 throws ParameterError at bind time instead of clamp-and-send (P1.1). - A non-numeric / exponential value ("1e+21", "abc", "") throws ParameterError with a clear message rather than mis-deriving (p,s) and surfacing an opaque kernel error (P1.2 / P2). Validated live: 0.00001 binds as DECIMAL(5,5) and round-trips; a 40-digit value is rejected client-side with ParameterError (no server round-trip). Unit tests updated to assert the new reject contract + leading-zero handling. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
|
@gopalldb thanks — the DECIMAL P1 was a real bug; fixed in the latest commit and validated live. P1.1 — digit-counting precision + >38 clamp-and-send. Confirmed both:
P1.2 / P2 — format surprises ( Validated live: P2 — getInfo This reply was written by Isaac. |
What
Consolidates three SEA session-backend capabilities into one focused PR, each a thin adapter over the merged kernel (which does the heavy lifting):
1. Bound parameters
ordinalParameters/namedParametersto the kernel'sExecuteOptions.positionalParams/namedParams.DBSQLParameter.toSparkParametercodec (the same type-inference the Thrift path uses), then adapts the type name to the kernel codec:DECIMAL→DECIMAL(p,s),INTERVAL *→INTERVAL, missing value →VOID/NULL.?) and named (:name) parameters are mutually exclusive — rejected up-front.2. Metadata
getCatalogs/getSchemas/getTables/getColumns/getFunctions/getTableTypes/getTypeInfo/getPrimaryKeys/getCrossReferenceforward to the kernelConnection.list*/get*methods and wrap the returnedStatement.TABLE_TYPEfilter — the driver only maps request fields to positional arguments. (The previously-proposed driver-sideSeaTableTypeFilteris therefore not needed and was dropped.)3. getInfo
getInfoendpoint), mirroring the threeTGetInfoTypevalues the Databricks Thrift server answers (server name / DBMS name / DBMS version) and rejecting every other value — byte-identical to the Thrift path.Notes
SeaInputValidationis slimmed to the single bind-time gate the driver genuinely needs (assertBindableValue); parameter-marker counting is the kernel's job, so the JS SQL-walk was removed to avoid drift.SeaNativeLoaderre-exports the kernel'sExecuteOptions/TypedValueInput/NamedTypedValueInputtypes rather than re-declaring them, so driver param shapes can't drift from the kernel contract..d.tssnapshot against the merged kernel (matches how prior SEA PRs [SEA-NodeJS] (2/8) napi-rs binding consumer — TS loader + build script #380 / [SEA-NodeJS] (1/3) SEA connect + auth (PAT + OAuth M2M/U2M) #409 landed binding updates).Testing
lib/lints clean.tableTypes=[VIEW]filter), andgetInfo(CLI_DBMS_NAME)returnsSpark SQL.This pull request and its description were written by Isaac.