Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

Learn built-in UDF return types by typing alone, defer the compiles - #612

Open
eriknw wants to merge 5 commits into
31-numba-cache-builtin-udfsfrom
32-udf-typing-only-types
Open

eriknw wants to merge 5 commits into
31-numba-cache-builtin-udfsfrom
32-udf-typing-only-types

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026 •

Copy link
Copy Markdown
Member

First touch of a built-in UDF binop compiled it for all 11 to 13 sample
dtypes, and threw nearly all of that away. The compiles existed only to
read back a return type per dtype for op.types; a program that then used
one dtype paid for the rest.

Get the return types from numba's typing pipeline alone (run_frontend
plus type_inference_stage, no lowering), apply the existing downcast
heuristic over the full result set, and defer each dtype's lowering,
cfunc wrapper, and GrB_BinaryOp_new to the first op[dtype] access via
_build_deferred/_materialize_deferred on OpBase. op.types is unchanged:
dumped for all five ops before and after and diffed, identical.

Numba exposes no public way to ask for a return type without also
compiling for it, so _infer_ret_types_typing_only reaches into
numba.core. We support numba back to 0.57 and this was verified against
0.66.0 only, so the helper returns None on a failed import or any
surprise from the call and _build falls back to the eager loop. Verified
by forcing _HAS_TYPING_ONLY = False: the suite passes with only
test_builtin_udf_types_precede_compilation failing, which is the intended
signal. A numba that moves these internals turns into a red test rather
than a silent return to the old cost.

Only the built-in UDFs take this path, keyed off the same flag that marks
them cacheable. Typing is more permissive than lowering, so a user
function whose dtype types but fails to lower would be wrongly advertised
in .types; register_new and register_anonymous stay eager and keep
validating lowerability per dtype. Semiring construction iterates
_typed_ops directly, so it materializes a deferred multiplier first.

Measured here, fresh process, warm numba cache, medians of 3, load
average 4 to 5 on a shared machine so these are indicative and the
before-numbers are the noisier half. First-touch floordiv: 474 -> 248 ms.
First-touch of all five: 753 -> 289 ms. The deferred per-dtype build does
not vanish, it moves to first use of that dtype.

This does not speed up a program that ends up using every dtype; it stops
programs that use one or two from paying for all of them. It does not
touch UDT auto-lift, which was already compiled on demand.

_finalize_typed_binaryop keeps error_model="numpy" on the cfunc, as the
call site it was extracted from had it and every other cfunc in the
package sets it. No test here distinguishes it: with it removed from the
cfunc, from the njit, or from both, and with the numba cache cleared,
integer floordiv by zero still returns numpy's answer.

Deferral surfaced one more direct consumer of _typed_ops: the typed
commutes_to properties asked whether the PARTNER op had already compiled
this dtype, so in a fresh process floordiv[INT64].commutes_to answered
None until rfloordiv happened to be built, and the answer depended on
access order. The membership test now consults .types, whose keyset is
identical to _typed_ops once materialized, letting getitem drive the
deferred build. test_deferred_commutes_to pins it in a subprocess, where
access order is controlled.

Added in the 2026-10-20 coverage round

Three behavior fixes, each with before and after in its commit message (main
too). Numba types some ops it has no lowering for, such as > on complex
values, which max uses, and then raises a bare NotImplementedError from
lowering:

  • The per-dtype build loops of UnaryOp, BinaryOp, IndexUnaryOp (and so
    SelectOp) and IndexBinaryOp caught only TypingError, so the whole
    registration failed. BinaryOp.register_anonymous(lambda x, y: max(x, y))
    raised NotImplementedError; it now gives an op for BOOL, the ints and the
    floats, without FC32 or FC64.
  • With is_udt=True, the same error escaped op["FC64"] and "FC64" in op.
    The first now raises the usual UdfParseError ("does not work with"), and
    in answers False.
  • Numba before 0.60 raises a LoweringError from lowering instead, and 0.58
    and 0.59 first warn NumbaPendingDeprecationWarning, which -W error (as in
    our tests) raises in its place. CI draws Numba 0.57 to 0.68, so on those
    versions the registration above still failed. The four loops now share
    _compile_for_dtype, which skips a dtype on TypingError,
    NotImplementedError, or a LoweringError raised from a NotImplementedError,
    and raises any other error. The warning is ignored only while Numba
    captures errors "old_style" (a MAINT note says to drop that once Numba
    0.61 is the minimum), and the UdfParseError of an is_udt op names the
    missing lowering ("No definition for lowering ...") on every version.

Stack created with GitHub Stacks CLI • Give Feedback 💬

@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 2 times, most recently from 7e70d25 to bc9e14e Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 2 times, most recently from c4efaf4 to e17f97f Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from e17f97f to a020a2c Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 2 times, most recently from 6fbd969 to ace2c7a Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from ace2c7a to c489e57 Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from c489e57 to 04b6930 Compare August 6, 2026 15:41
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 2 times, most recently from a9184d0 to f465e6d Compare August 6, 2026 20:41
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from f465e6d to 31c66f3 Compare August 7, 2026 02:49
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 31c66f3 to 3f97f50 Compare August 7, 2026 05:09
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 3f97f50 to ce3a876 Compare August 26, 2026 17:30
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from ce3a876 to 16bfdbc Compare September 18, 2026 03:09
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 16bfdbc to 5079067 Compare September 18, 2026 03:46
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 005655f to ff1d5fb Compare October 1, 2026 17:11
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 3 times, most recently from af2bba7 to 8a5e678 Compare October 3, 2026 04:57
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 8a5e678 to 0c69478 Compare October 4, 2026 06:21
@eriknw
eriknw removed this pull request from stack #634 October 8, 2026 21:02
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 0c69478 to a0fe06c Compare October 8, 2026 21:40
@eriknw
eriknw added this pull request to stack #635 October 8, 2026 21:46
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch 2 times, most recently from fe4e2dd to 486c4cd Compare October 9, 2026 05:25
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 486c4cd to 11c1db1 Compare October 14, 2026 18:45
@eriknw
eriknw removed this pull request from stack #635 October 19, 2026 07:45
@eriknw
eriknw changed the base branch from 31-numba-cache-builtin-udfs to main October 19, 2026 07:46
@eriknw
eriknw force-pushed the 32-udf-typing-only-types branch from 11c1db1 to d2776df Compare October 19, 2026 07:47
@eriknw
eriknw changed the base branch from main to 31-numba-cache-builtin-udfs October 19, 2026 07:48
@eriknw
eriknw added this pull request to stack #650 October 19, 2026 07:50
@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.54902% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.95%. Comparing base (9d95bcd) to head (ac8b757).

Additional details and impacted files
@@                       Coverage Diff                       @@
##           31-numba-cache-builtin-udfs     #612      +/-   ##
===============================================================
+ Coverage                        97.57%   98.95%   +1.38%     
===============================================================
  Files                              105      105              
  Lines                            29949    30115     +166     
  Branches                          3489     3509      +20     
===============================================================
+ Hits                             29223    29801     +578     
+ Misses                             621      203     -418     
- Partials                           105      111       +6     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

First touch of a built-in UDF binop compiled it for all 11 to 13 sample
dtypes, and threw nearly all of that away. The compiles existed only to
read back a return type per dtype for op.types; a program that then used
one dtype paid for the rest.

Get the return types from numba's typing pipeline alone (run_frontend
plus type_inference_stage, no lowering), apply the existing downcast
heuristic over the full result set, and defer each dtype's lowering,
cfunc wrapper, and GrB_BinaryOp_new to the first op[dtype] access via
_build_deferred/_materialize_deferred on OpBase. op.types is unchanged:
dumped for all five ops before and after and diffed, identical.

Numba exposes no public way to ask for a return type without also
compiling for it, so _infer_ret_types_typing_only reaches into
numba.core. We support numba back to 0.57 and this was verified against
0.66.0 only, so the helper returns None on a failed import or any
surprise from the call and _build falls back to the eager loop. Verified
by forcing _HAS_TYPING_ONLY = False: the suite passes with only
test_builtin_udf_types_precede_compilation failing, which is the intended
signal. A numba that moves these internals turns into a red test rather
than a silent return to the old cost.

Only the built-in UDFs take this path, keyed off the same flag that marks
them cacheable. Typing is more permissive than lowering, so a user
function whose dtype types but fails to lower would be wrongly advertised
in .types; register_new and register_anonymous stay eager and keep
validating lowerability per dtype. Semiring construction iterates
_typed_ops directly, so it materializes a deferred multiplier first.

Measured here, fresh process, warm numba cache, medians of 3, load
average 4 to 5 on a shared machine so these are indicative and the
before-numbers are the noisier half. First-touch floordiv: 474 -> 248 ms.
First-touch of all five: 753 -> 289 ms. The deferred per-dtype build does
not vanish, it moves to first use of that dtype.

This does not speed up a program that ends up using every dtype; it stops
programs that use one or two from paying for all of them. It does not
touch UDT auto-lift, which was already compiled on demand.

_finalize_typed_binaryop keeps error_model="numpy" on the cfunc, as the
call site it was extracted from had it and every other cfunc in the
package sets it. No test here distinguishes it: with it removed from the
cfunc, from the njit, or from both, and with the numba cache cleared,
integer floordiv by zero still returns numpy's answer.

Deferral surfaced one more direct consumer of _typed_ops: the typed
commutes_to properties asked whether the PARTNER op had already compiled
this dtype, so in a fresh process floordiv[INT64].commutes_to answered
None until rfloordiv happened to be built, and the answer depended on
access order. The membership test now consults .types, whose keyset is
identical to _typed_ops once materialized, letting __getitem__ drive the
deferred build. test_deferred_commutes_to pins it in a subprocess, where
access order is controlled.
conftest touches every delayed op and semiring before the tests run, and
building a semiring compiles every dtype of its binary op, so in the test
process OpBase.__getitem__ never built a deferred dtype; only the
subprocess tests did, and they count compiled dtypes without using the
op they get back. Build a new op from floordiv's function as _initialize
does, index it, and check that it compiles that dtype alone, returns it
on the next index too, and floors toward minus infinity. Without the
return of the deferred op, indexing gives a UDT-lifted floordiv instead.

A second test sets _HAS_TYPING_ONLY to False, as on a Numba whose
internals moved, and checks that the eager build advertises the same
types as the typing-only one.
Behavior fix. UnaryOp, BinaryOp, IndexUnaryOp (and so SelectOp) and
IndexBinaryOp compile a UDF for each sample dtype and skip the dtypes it
fails to type for, catching numba.TypingError. Numba types some ops it
has no lowering for, such as `>` on complex values, which `max` uses,
and raises a bare NotImplementedError from lowering. That escaped the
loops, so the whole registration failed:

    BinaryOp.register_anonymous(lambda x, y: max(x, y))
        before: NotImplementedError: No definition for lowering
                <built-in function gt>(complex64, complex64) -> bool
        after:  a BinaryOp for BOOL, the ints and the floats; no FC32 or FC64

The same for UnaryOp, IndexUnaryOp, SelectOp and IndexBinaryOp. Catch
NotImplementedError beside TypingError in all four loops. Numba's own
NumbaNotImplementedError is a TypingError already (numba 0.66 and 0.68).
Behavior fix. An is_udt=True UDF compiles for each dtype on first use,
through _compile_udf_for_udt, which turns a NumbaError into
UdfParseError ("does not work with"). Numba types some ops it has no
lowering for, such as `>` on complex values, which `max` uses, and
raises a bare NotImplementedError from lowering. That escaped, from `in`
too:

    op = BinaryOp.register_anonymous(lambda x, y: max(x, y), is_udt=True)
    op["FC64"]       # before: NotImplementedError: No definition for lowering
                     #   <built-in function gt>(complex128, complex128) -> bool
    "FC64" in op     # before: the same NotImplementedError
    op["FC64"]       # after: UdfParseError: binary.<lambda> does not work with
                     #   (FC64, FC64): No definition for lowering ...
    "FC64" in op     # after: False

The same for UnaryOp, IndexUnaryOp and SelectOp, which share the helper.
Catch NotImplementedError there too, as the per-dtype build loops now do.
Behavior fix. Registering a UDF skips the dtypes Numba types but has no
lowering for, such as complex for `max`, by catching the bare
NotImplementedError from lowering. Numba before 0.60 captures errors
"old_style" by default and raises a LoweringError from it instead, and
0.58 and 0.59 first warn NumbaPendingDeprecationWarning, which `-W error`
(as in our tests) raises in its place. CI draws Numba 0.57 to 0.68:

    BinaryOp.register_anonymous(lambda x, y: max(x, y))   # Numba 0.57 to 0.59
        before: LoweringError: Failed in nopython mode pipeline (step:
                native lowering); 0.58 and 0.59 also warn, and under
                -W error raise NumbaPendingDeprecationWarning instead
        after:  a BinaryOp with no FC32 or FC64, as on 0.60 and later

The same for UnaryOp, IndexUnaryOp, SelectOp and IndexBinaryOp. Their
per-dtype loops now share _compile_for_dtype, which also skips a
LoweringError raised from a NotImplementedError, and raises any other.

An is_udt UDF on such a dtype already raised UdfParseError, except under
-W error on 0.58 and 0.59. _numba_compile keeps that warning out of both
paths, and the message now names the missing lowering on every version:

    op = BinaryOp.register_anonymous(lambda x, y: max(x, y), is_udt=True)
    op["FC64"]   # Numba 0.57 to 0.59
        before: UdfParseError: ... does not work with (FC64, FC64): Failed
                in nopython mode pipeline (step: native lowering)
        after:  UdfParseError: ... does not work with (FC64, FC64): No
                definition for lowering <built-in function gt>(complex128,
                complex128) -> bool
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant