Skip to content

[AURON #1861] Convert comparison operators and LIKE to native operators#2298

Open
weiqingy wants to merge 3 commits into
apache:masterfrom
weiqingy:AURON-1861-impl
Open

[AURON #1861] Convert comparison operators and LIKE to native operators#2298
weiqingy wants to merge 3 commits into
apache:masterfrom
weiqingy:AURON-1861-impl

Conversation

@weiqingy
Copy link
Copy Markdown
Contributor

@weiqingy weiqingy commented Jun 2, 2026

Which issue does this PR close?

Closes #1861

Rationale for this change

Flink Calc expressions that use comparison operators (=, <>, >, <, >=, <=) or LIKE could not run on the Auron native engine — the Flink RexCallConverter recognized only arithmetic and CAST, so any Calc containing a comparison fell back to the Flink engine. Comparisons are the backbone of WHERE/JOIN/CASE predicates, so without them most real queries cannot be accelerated.

What changes are included in this PR?

Extends the existing RexCallConverter (no proto, Rust, or dependency changes — the native plumbing already exists for all of these operators):

  • Comparison operatorsPhysicalBinaryExprNode with op strings Eq / NotEq / Gt / Lt / GtEq / LtEq. Operands are promoted to a common type before conversion (the native comparison kernel requires matching operand types and performs no implicit coercion); the boolean result needs no output cast.
  • LIKE → the dedicated PhysicalLikeExprNode. LIKE ... ESCAPE c (explicit, non-default escape) is reported unsupported so the Calc falls back, since the native like node has no escape field. NOT LIKE is represented by Calcite as NOT(LIKE(...)); it converts end-to-end once the logical NOT converter is in place, and falls back to Flink until then.

Two commits: one for the six comparison operators, one for LIKE.

A few existing tests used EQUALS as a known-unsupported operator to exercise the fallback path; now that comparisons are supported, they use SIMILAR TO (still unsupported) to keep testing fallback.

Are there any user-facing changes?

Flink queries whose Calc predicates use these comparison operators or LIKE now execute on the native engine instead of falling back to Flink. No API or configuration change.

How was this patch tested?

New unit tests in RexCallConverterTest cover each comparison operator's native op string, operand type promotion, LIKE, the NOT(LIKE) fallback, and the explicit-ESCAPE fallback. The full auron-flink-planner module test suite passes (82 tests), with 0 Checkstyle violations.

@github-actions github-actions Bot added the flink label Jun 2, 2026
Copy link
Copy Markdown
Contributor

@Tartarus0zm Tartarus0zm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiqingy Thanks for your contribution!
The Flink Calc operator integration flow is already working end-to-end. We can now add full query tests in ITCase.

@weiqingy
Copy link
Copy Markdown
Contributor Author

weiqingy commented Jun 2, 2026

@weiqingy Thanks for your contribution! The Flink Calc operator integration flow is already working end-to-end. We can now add full query tests in ITCase.

@Tartarus0zm Thanks for the review! Added 9 end-to-end SQL ITCase tests in AuronFlinkCalcITCase (commit 8c886da7).

They submit real queries over T1 through StreamTableEnvironment and assert the result rows, covering all six comparison operators (=, <>, >, <, >=, <=) in WHERE filters, a comparison used directly in a Boolean projection, an INT-vs-DOUBLE filter that exercises operand promotion, and a LIKE filter.

The run logs confirm these execute natively — e.g. FilterExec [string@0 LIKE Comment%] and ProjectExec [int@0 > 1] over the FFIReader — so the Calc integration flow is genuinely working end-to-end for these operators.

I also refreshed the now-outdated comment on AuronCalcRewriteITCase.testFilterAndProjectEndToEnd, which still described comparisons as unsupported / falling back. Full planner suite is green (91 tests) with 0 Checkstyle violations.

@weiqingy
Copy link
Copy Markdown
Contributor Author

weiqingy commented Jun 2, 2026

The 1 failing check is unrelated to this PR:

  • Test spark-4.0 JDK21 Scala-2.13 / Build Auron JAR: JVM/RPC crash in spark-4.0 integration tests — the auron-spark-tests-spark40 module died mid-suite when the driver's in-test Spark REPL class server (spark://...:43973) became unreachable (RemoteClassLoaderError / event executor terminated / connect-refused), followed by a fatal SIGSEGV in libjvm.so (core dump + adoptium bug-report footer). This PR changes only Flink planner files under auron-flink-extension/ — no Spark, native-engine, or proto code — and every other Spark build (3.0–3.5, 4.1) plus Test Flink 1.18 passed on the same commit. Evidence: failing job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Convert Comparison operators to Auron Native operators

2 participants