bench: add ASOF join benchmarks - #23833
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #23833 +/- ##
==========================================
- Coverage 81.88% 81.88% -0.01%
==========================================
Files 1133 1133
Lines 424522 424522
Branches 424522 424522
==========================================
- Hits 347622 347612 -10
- Misses 56288 56294 +6
- Partials 20612 20616 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
e826548 to
1ff6d2b
Compare
1ff6d2b to
c3d099b
Compare
# Conflicts: # datafusion/expr/src/logical_plan/builder.rs # datafusion/expr/src/logical_plan/plan.rs # datafusion/sql/src/relation/join.rs # datafusion/sql/src/unparser/plan.rs # datafusion/sqllogictest/test_files/asof_join.slt # datafusion/substrait/tests/cases/serialize.rs # docs/source/user-guide/sql/select.md
|
There are many optimizations to do, but I think it good to have a benchmark in first. This PR will be the last one of this stack, all other changes will be non-blocking, optional follow ups to do. Cc @2010YOUY01 and @jayzhan211, let's go! Time to close the issue created 5 years ago 🤓 |
jayzhan211
left a comment
There was a problem hiding this comment.
Thanks @Xuanwo
Similar to nlj, hj, smj it would be nice to have ./bench.sh run asof
Some suggestions for more cases covered
2010YOUY01
left a comment
There was a problem hiding this comment.
Thank you for the fast iteration on this project! Here are some suggestions
I suggest we move the SQL benchmark to the new SQL bench framework, like https://github.com/apache/datafusion/tree/main/benchmarks/sql_benchmarks/nlj, so it would be runnable with
# under benchmarks/
./bench.sh run asof_join
Optional: To better explain the workload covered, we can write comments like
# top comment: axes to vary
#
# size: vary left/right input with very small / medium / large sizes, like 100/100k/10m rows
# sortedness: does the input already have the required order, or do we have to sort?
# group cardinality / size: ...
# per-query comment: just tag the axes
Q1: left small, right large; no existing order; single group ...
We can do this as a follow-up and potentially examine and try to better cover the typical workload.
|
Thanks @jayzhan211 and @2010YOUY01 for the thoughtful reviews! This is much cleaner now 🙌 |
Which issue does this PR close?
Rationale for this change
This is the final core benchmark layer of the ASOF JOIN stack. It uses the
standard SQL benchmark framework so the workloads share the existing runner,
result format, and Criterion mode.
What changes are included in this PR?
asof_joinSQL benchmark suite, runnable with./bench.sh run asof_joinorbenchmark_runner.equality-group cardinality and skew, match direction, and payload width.
declare their ordering with
WITH ORDER, keeping input sorting outside thatmeasured query.
AsOfJoinExec.Are these changes tested?
Yes:
cargo fmt --all./ci/scripts/doc_prettier_check.sh --write --allow-dirtycargo clippy --all-targets --all-features -- -D warningscargo test -p datafusion-benchmarks --lib --binscargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --iterations 1 --partitions 4CARGO_COMMAND='cargo run' ./bench.sh run asof_join 1cargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --query 7 --partitions 4 --criterionThe pre-sorted workload's physical plan contains
AsOfJoinExecwithout aninput
SortExec. These smoke runs validate the suite and are not presented asperformance claims.
Are there any user-facing changes?
This adds ASOF workloads to the repository benchmark tooling. It does not
change query semantics or runtime behavior.
This is the final core item tracked by #23738. It does not depend on the
optional floating-point follow-up #24375.