IN LIST: retain short lists with specialized filters

[geoffreyclaude]

Which issue does this PR close?

• Part of Further improve performance of IN list evaluation #19241.
• Stacked on #24088.
• Requires #25186 to land first so retained floating-point lists preserve signed-zero equality.

Rationale for this change

The expression simplifier currently rewrites singleton IN lists, and two- or three-item lists on a column, into left-deep OR / AND comparison chains. That rule predates the specialized static filters in this series. For supported types, rewriting two- and three-item lists now prevents them from reaching a substantially faster branchless physical path.

The rewrite should therefore depend on the available physical representation, not only on list length. This PR keeps singleton-to-equality simplification, retains static two- and three-item InList expressions when a specialized filter can evaluate them, and preserves the existing comparison rewrite for dynamic or generic lists.

Although the singleton InList path is faster for several representations in isolated evaluation, equality remains the canonical logical form and can benefit other optimizer and pushdown paths that this microbenchmark does not measure.

What changes are included in this PR?

• Gives the short-list simplifier access to type information.
• Retains two- and three-item constant lists for supported primitive, temporal, decimal, fixed-size-binary, and inline byte-view representations, including dictionary wrappers.
• Keeps the existing OR / AND expansion for generic types, out-of-line byte views, and non-constant lists.
• Decouples the preimage rewrite limit from the short-list physical-filter decision; its existing three-value limit is unchanged.
• Adds a SQL-planned benchmark for list sizes one through four and updates affected plan snapshots.

Are these changes tested?

Coverage includes the retained and expanded boundaries; IN and NOT IN; typed nulls; dynamic lists; nested dictionaries; supported and unsupported fixed-size-binary widths; inline and out-of-line Utf8View / BinaryView values; and the independent preimage limit.

Focused validation:

• cargo test -p datafusion-optimizer --lib (796 tests)
• All 11 affected SQL logic test files
• cargo test -p datafusion-substrait roundtrip_inlist (5 tests)
• With IN LIST: treat signed zeros as equal #25186 applied: cargo test -p datafusion-sqllogictest --test sqllogictests -- negative_zero
• cargo check -p datafusion --bench in_list_rewrite
• cargo bench -p datafusion --bench in_list_rewrite -- --noplot

Are there any user-facing changes?

Once prerequisite #25186 has landed, no query results or public APIs change. Optimized plans retain two- and three-item InList expressions for supported constant-list representations instead of expanding them into comparison chains.

Local benchmark snapshot

The new benchmark parses and analyzes both forms from SQL, asserts their physical shapes and equal outputs, then measures only physical-expression evaluation. It covers five representative types; list sizes 1-4; batches of 1, 64, and 8,192 rows; input and list nulls; miss, balanced, and skewed match rates; and both IN and NOT IN: 300 paired scenarios and 600 Criterion functions in total.

The table focuses on the two sizes whose optimizer policy changes. Each ratio is the median comparison chain / retained InList time across 16 scenarios; values above 1 favor retaining InList.

Representation	2 items	Wins	3 items	Wins	Decision
FixedSizeBinary(8)	17.26x	16/16	17.99x	16/16	Retain
Float64	3.29x	16/16	3.96x	16/16	Retain
Int32	1.85x	16/16	2.27x	16/16	Retain
inline Utf8View	1.34x	14/16	1.35x	14/16	Retain
generic Utf8	1.25x	11/16	1.77x	12/16	Keep OR / AND

Generic Utf8 remains mixed, including regressions for skewed and null-containing lists, so this PR deliberately leaves it on the existing comparison rewrite. List sizes one and four serve as boundary controls; their optimizer policy is unchanged.