refactor(qwp): simplify the wire protocol to one version with inline schemas#39
Open
mtopolnik wants to merge 3 commits into
Open
refactor(qwp): simplify the wire protocol to one version with inline schemas#39mtopolnik wants to merge 3 commits into
mtopolnik wants to merge 3 commits into
Conversation
Mirror the server-side QWP simplification on the Java client. The wire format drops the per-table-block schema mode byte and schema id; column schemas always travel inline. Egress RESULT_BATCH carries the schema only on the first batch (batch_seq == 0); continuation batches reuse it. Ingress encode: QwpColumnWriter writes columns inline (no mode byte or schema_id), QwpWebSocketEncoder and QwpUdpSender drop the useSchemaRef parameter, and QwpWebSocketSender drops the schema-id allocator and the maxSchemasPerConnection plumbing across its connect overloads. QwpTableBuffer loses its schemaId field. Egress decode: QwpResultBatchDecoder parses the inline schema only on batch_seq == 0 into a reused per-query schema, replacing the connection schema registry; QwpEgressIoThread invalidates it when a new query starts. QwpQueryClient always reads SERVER_INFO now that the version gate is gone. Constants collapse to a single VERSION = 1, and CACHE_RESET keeps only RESET_MASK_DICT. The Sender builder loses maxSchemasPerConnection() and the max_schemas_per_connection connect-string key (a breaking change; QWP is beta). Client unit tests updated to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up doc cleanup after removing QWP schema references and protocol v2. No logic changes. Reword comments that still described the removed two-version, schema-reference world: the CACHE_RESET doc no longer advertises a schema-cache mask bit, SERVER_INFO is "every connection" rather than "every v2 connection", and the capability-trailer comments are version-neutral instead of naming a nonexistent v2.0/v2.1. Expand the QwpResultBatchDecoder querySchema comment to record two consequences of dropping the schema registry: every query's batch 0 now re-parses the inline schema and allocates a column-name String per column, and the single schema slot is correct only while the IoThread runs one query at a time (pipelining would need request_id keying). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Un-glue eight connect() parameter lists in QwpWebSocketSender where removing the maxSchemasPerConnection parameter had collapsed two arguments onto a single line with stray indentation. No behaviour change -- whitespace only. Repoint three Javadoc/comment references that pointed at the deleted docs/qwp/*.md design docs to their published equivalents under https://questdb.com/docs/ -- in QwpColumnBatch, ColumnView, and WebSocketResponse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 3, 2026
Open
Contributor
Author
[PR Coverage check]😍 pass : 31 / 37 (83.78%) file detail
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Removes the QWP schema-reference mechanism and the protocol-v2 designation from the Java client. Column schemas now always travel inline on the wire, and the client runs a single protocol version.
This is the client (Phase 2) half of a tandem change; the server half is questdb/questdb#7200. The QWP wire format is breaking, but QWP is beta so there is no compatibility window to preserve.
Why
QWP shipped with a schema-reference optimisation: a sender could transmit a column schema once, receive a schema id, and reference that id on later frames. Store-and-forward retry made this unworkable -- a recorded frame can replay against a fresh server connection that never saw the registering frame, leaving the reference dangling. Senders already stopped using reference mode in production, so the wire kept paying for a mechanism nobody used.
Wire format
Ingress table block (sender -> server):
Egress
RESULT_BATCH(server -> client): the schema rides only the first batch of a query (batch_seq == 0); continuation batches carry rows and bind against the schema parsed there.The per-frame version byte stays at offset 4, hardcoded to
1and validated== 1. TheX-QWP-Max-Versionhandshake header and negotiation machinery stay so a future version bump can re-introduce a range.Notable changes
batch_seq == 0is now rejected rather than binding rows to a stale schema.Sender.builder().maxSchemasPerConnection(...)and themax_schemas_per_connectionconnection-string option are gone. Code or config strings that reference them now fail -- with a compile error and an "unknown option" error respectively.SERVER_INFOis always delivered post-upgrade (no v2 gate), so the failover role filter no longer has a v1 "no role available" branch.Trade-offs
maxSchemasPerConnectionbuilder method disappears; external callers that set it will no longer compile. Acceptable for beta.Test plan
mvn -pl core test(client unit + decoder-hardening suites)SelfSufficientFramesTesttrimmed to its symbol-dict-delta invariant.cutlass-test-suite, which runs this client against the matching server branch.Coordination
Tandem with questdb/questdb#7200. The wire format is breaking; merge the two together. This PR's branch is named to match the server branch so the client CI builds against the server PR instead of master.
🤖 Generated with Claude Code