[Bug] abs reports a DataFusion-specific ANSI overflow message instead of Spark's ARITHMETIC_OVERFLOW text

[andygrove]

Describe the bug

Under datafusion.execution.enable_ansi_mode = true, datafusion-spark's abs
correctly raises on integral overflow, but the message is one DataFusion
invents rather than Spark's:

DataFusion error: Arrow error: Compute error: Int32 overflow on abs(-2147483648)

Spark 4.2.0 raises:

[ARITHMETIC_OVERFLOW] overflow. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error. SQLSTATE: 22003

The condition detected is the same. Only the text differs. DataFusion does not
model Spark's error classes or SQLSTATE values, so an exact reproduction is not
possible, but the message body is expressible.

This one matters more than a typical message mismatch because abs.rs is the
in-repo precedent that new datafusion-spark ANSI implementations are pointed
at. Its wording gets copied.

Cross-version note

The message text is version dependent, and 4.2.0 is the version that changed it:

Spark	Message for abs(-2147483648)
3.5.8	[ARITHMETIC_OVERFLOW] integer overflow. ...
4.0.4	[ARITHMETIC_OVERFLOW] integer overflow. ...
4.1.3	[ARITHMETIC_OVERFLOW] integer overflow. ...
4.2.0	[ARITHMETIC_OVERFLOW] overflow. ...

Spark 4.2.0 added a canonicalization step to
ExecutionErrors.arithmeticOverflowError that rewrites any message matching
\w+ overflow to plain overflow, so the JDK strings integer overflow and
long overflow and the MathUtils strings byte overflow and
short overflow all collapse to the same text. The reason given in the source
is JIT hot-throw behavior, JDK-8367990.

Before 4.2.0 the width appeared in the message, so DataFusion's current text is
closer in spirit to those versions than to 4.2.0, though equal to none of them.
DataFusion has no mechanism for version-specific expectations, tracked by
#23887.

To Reproduce

set datafusion.execution.enable_ansi_mode = true;

select abs((-2147483648)::INT);
-- DataFusion error: Arrow error: Compute error: Int32 overflow on abs(-2147483648)

The array path produces a second spelling, because the macro behind it
stringifies the Arrow array type rather than the scalar type:

select abs(a) FROM (VALUES (-2147483647::INT), ((-2147483648)::INT)) AS t(a);
-- DataFusion error: Arrow error: Compute error: Int32Array overflow on abs(-2147483648)

Both spellings are asserted today in
datafusion/sqllogictest/test_files/spark/math/abs.slt, four of each. Spark's
message does not vary by input shape.

Expected behavior

A message matching Spark 4.2.0's ARITHMETIC_OVERFLOW text, and the same
message on the scalar and array paths.

Scope

This is the abs half of the repository-wide question raised in
#23897. That issue notes that
changing pmod alone would leave the crate less consistent while abs keeps
its own wording. The two should be decided together.

The two paths in abs are not equally easy, and the array one is the reason
this is worth its own issue rather than a drive-by fix.

• Scalar path. The message is built by scalar_compute_op! in
  datafusion/spark/src/function/math/abs.rs. It is local to the spark crate
  and is two format! calls.
• Array path. The message is built by make_try_abs_function! in
  datafusion/functions/src/math/abs.rs, which datafusion-spark imports and
  reuses. Core DataFusion's own abs raises through the same macro, so the
  text cannot be changed there without changing the error message core
  DataFusion users see. Matching Spark on the array path means datafusion-spark
  needs its own kernel rather than borrowing core's.

That second point generalizes past abs: wherever a spark function reuses a
core macro or kernel, Spark-specific error text is not available without first
un-sharing the code.

Relevant code

• datafusion/spark/src/function/math/abs.rs
• datafusion/functions/src/math/abs.rs, make_try_abs_function!
• datafusion/sqllogictest/test_files/spark/math/abs.slt

Surfaced by the audit-datafusion-spark-expression skill.

[goutamadwant]

take

[goutamadwant]

@andygrove I have a fix with #24156.

It adds a Spark-local checked array kernel so scalar and array ANSI overflow paths return the Spark 4.2 ARITHMETIC_OVERFLOW message without changing core DataFusion behavior. Let me know if any suggestions. thanks!