Repository navigation
Conversation
This was referenced Oct 28, 2026
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 11a-py315-maintenance #652 +/- ##
=========================================================
- Coverage 98.83% 98.57% -0.26%
=========================================================
Files 104 104
Lines 28923 29121 +198
Branches 3417 3439 +22
=========================================================
+ Hits 28586 28707 +121
- Misses 215 270 +55
- Partials 122 144 +22 🚀 New features to boost your workflow:
|
eriknw
force-pushed
the
53-jit-improvements
branch
from
October 28, 2026 13:21
43398c6 to
52836c1
Compare
…e names)
The conda repair looked for {baked-in name, cc, clang, gcc} in
$CONDA_PREFIX/bin only. Conda on Windows installs compilers in
%CONDA_PREFIX%\Library\bin, and their names end in .exe, so the repair
found nothing there and left SuiteSparse's build-time compiler, which a
user's machine does not have. It now looks in both places, with and
without .exe, and takes a Windows path's basename with either slash.
Tested with a fake prefix (no Windows machine at hand).
… types A lifted binary op (plus, minus, times, truediv, floordiv, min, max) whose operands are a scalar and an array or record UDT, in either order, or two UDTs of one layout with other element types, ran only through its Numba cfunc: the C definition was set for same-type pairs alone, with one C type for x, y and z. So a semiring such as plus_times[FP64, FP64[R]] ran SuiteSparse's generic kernel with a function call per multiply. Such an op now gets a C definition with a type for each of z, x and y (udt_codegen._make_jit_c_mixed_definition), named op_N_x_y with N the length of x's C name, so that no two ops share a name (SuiteSparse finds a compiled kernel in memory by its op's name): a scalar operand is read as (*x) for every element or leaf, and each operand is converted in C to the type the op computes in, as the cfunc converts it (udt._udt_op_types). Ops with no such form keep the cfunc: an array operand that broadcasts, an element not computed in the result's type (integer truediv), a record with an array leaf, and eq/ne on a mixed pair. The whole mxv with plus_times[FP64, FP64[R]] now compiles as one JIT kernel (burble: dot2, jit). test_udt_scalar_broadcast_skips_jit becomes test_udt_scalar_broadcast_jit_routing: mixed arithmetic now JITs, mixed eq/ne still does not.
…present The burble names a JIT kernel only when it compiles or loads it, so a kernel loaded earlier in the process prints no '(jit: ...)'. A UDT product has no kernel but the JIT one and the generic one, which burble names, so the test checks that 'generic' is absent with the JIT on and present with it off.
…SuiteSparse test_udt_mixed_jit_c_name_is_its_own built its vectors from constant values. SuiteSparse before 10.5 makes a vector built from equal values iso, and an ewise_mult of iso vectors gives an iso result computed without the JIT kernel the test is about, so on those versions it checked nothing. On them its to_dense also raised "TypeError: unsupported operand type(s) for +: 'int' and 'tuple'" (wave 2 CI, 17 jobs, e.g. 122633175811 and 122633175893), the iso array-UDT export bug fixed in #591. The values now differ by index.
eriknw
force-pushed
the
53-jit-improvements
branch
from
October 29, 2026 02:01
52836c1 to
1deed77
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #636. The first three commits were written on the Tensor branch (#651) and
moved down here, since nothing in them is Tensor-specific: no line in this PR imports or
names the Tensor; the fourth fixes one of their tests. The diff is
binary.py(oneelif),udt_codegen.py(the mixed C definitions),ss/jit_config.pyandtest_ssjit.py.ss.fix_jit_configfinds a conda compiler on Windows. The conda repair looked for{the baked-in compiler's name, cc, clang, gcc} in
$CONDA_PREFIX/binonly. Conda onWindows installs compilers in
%CONDA_PREFIX%\Library\bin, with.exenames, andconda-forge's GraphBLAS there names the build machine's
cl.exe, so the repair foundnothing and kept a compiler the user's machine does not have. It now looks in both
places, with and without
.exe, and takes a Windows path's basename with either slash.Tested with a fake prefix; there was no Windows machine at hand. A compiler found this
way that cannot take the baked-in flags fails
fix_jit_config's probe compile, whichleaves the JIT off, as before.
JIT C definitions for lifted ops on a scalar and a UDT, or on two UDT types. A
lifted binary op (plus, minus, times, truediv, floordiv, min, max) whose operands are a
scalar and an array or record UDT, in either order, or two UDTs of one layout with
other element types, ran only through its Numba cfunc: the C definition was set for
same-type pairs alone, with one C type for x, y and z. So a semiring such as
plus_times[FP64, FP64[R]]ran SuiteSparse's generic kernel with a function call permultiply. Such an op now gets a C definition with a C type for each of z, x and y
(
udt_codegen._make_jit_c_mixed_definition). A scalar operand is read as(*x)forevery element or leaf, and each operand is converted in C to the type the op computes
in, as the cfunc converts it (
udt._udt_op_types):Before,
jit_c_nameandjit_c_sourceof these ops were None. Ops with no such formkeep the cfunc: an array operand that broadcasts, an element not computed in the
result's type (integer truediv), a record with an array leaf, and eq/ne on a mixed
pair. The whole mxv with
plus_times[FP64, FP64[R]]now compiles as one JIT kernel(burble: dot2, jit).
Test the mixed semiring's JIT kernel by burble. Burble names a JIT kernel only when
it compiles or loads it, so a kernel loaded earlier in the process prints no
(jit: ...). A UDT product has no kernel but the JIT one and the generic one, whichburble does name, so the test checks that "generic" is absent with the JIT on and
present with it off.
Vary the mixed JIT C name test's inputs, so its kernel runs on every SuiteSparse.
test_udt_mixed_jit_c_name_is_its_ownbuilt its vectors from constant values.SuiteSparse before 10.5 makes a vector built from equal values iso, and an
ewise_mult of iso vectors gives an iso result computed without the JIT kernel the
test is about, so on those versions, which CI draws, it checked nothing. Its
to_densealso raised there (17 CI jobs), from the iso array-UDT export bug fixed inReject record UDT pairs whose leaf counts differ #591. The values now differ by index.
The C name is
op_N_x_y, with N the length of x's C name (times_6_double_cabove).SuiteSparse finds a compiled kernel in memory by its op's name alone: the op's hash is
GB_jitifyer_hash(op_name), every UDT has the same type code, so the encodings match too,and
GB_jitifyer_lookupnever compares the definition. A plainop_x_yis ambiguous:plus on UDTs
AandBwould beplus_A_B, the name of the one-type plus on a UDT whoseC name is
A_B, and whichever of the two ran second ran the other's kernel. Reviewreproduced both orders: wrong values (1.5000000000000022 and 5e-323 for 11.5), and 16-byte
elements written into an array of 2-byte elements. With the length in it, a mixed name
reads back one way only, and it cannot equal a one-type name (
{op}_{type}), since a Ctype name never starts with a digit. It needs a UDT named like a pair, so it is rare, but
the failure is silent.
test_udt_mixed_jit_c_name_is_its_ownpins it.Tests:
test_udt_scalar_broadcast_skips_jitbecomestest_udt_scalar_broadcast_jit_routing(mixed arithmetic now gets a C definition, mixedeq/ne still does not). New:
test_udt_mixed_jit_c_source,test_udt_mixed_jit_c_name_is_its_own,test_udt_mixed_jit_cfunc_parity,test_udt_mixed_semiring_runs_jitandtest_fix_from_conda_finds_windows_compilers. TheJIT tests need SuiteSparse:GraphBLAS 9 or later (the C definitions are set with its string
setters) and skip before it.
Checked locally (macOS arm64, SuiteSparse:GraphBLAS 10.5, clang):
float64, float32, int8, int32, uint8 and complex64, 2x3 arrays, aligned and packed
records) against 7 scalar types in both orders, and 10 UDT pairs in both orders, 7 ops
each, with NaN, inf, -0.0 and integer zeros in the data. All 878 ops that got a C
definition give the cfunc's bytes. A second sweep by the reviewer (12 UDTs, 10 scalar
types, nested records, integer min and max) got 1326 C definitions, all compiled, all
equal to the cfunc apart from the padding bytes of nested aligned records, which the
one-type path leaves differently too.
mxvandvxmwith plus_times, min_minus and max_minus on[FP64, FP64[4]]and[FP64[4], FP64], withAandA.T: the JIT and generic kernels agree.--runslow, without Numba on both backends, and in CI'sC-scalar pass; on Python 3.15, numpy 2.2, and SuiteSparse:GraphBLAS 8.2 (whose JIT works
locally, where these tests skip), the only failures come from the local environments
(
test_version, and pandas 3.1 in the Python 3.15 one, see Reject record UDT pairs whose leaf counts differ #591).For a reviewer:
code does not check that status, so
jit_c_nameandjit_c_sourcewould show adefinition SuiteSparse does not have while the op runs through the cfunc. The one-type
path has the same gap; mixed names make it likelier.
((GxB_FC64_t) (x)), which MSVC's_Dcomplexwould not accept. The one-type complex kernels already assume C99 complex.