Repository navigation
Conversation
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 5, 2026 17:44
ceeeea2 to
aaf597e
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 5, 2026 18:03
aaf597e to
8e2c61c
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
2 times, most recently
from
August 6, 2026 07:59
56e3ec8 to
eef9de4
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 6, 2026 15:39
eef9de4 to
3ddf548
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 6, 2026 15:41
3ddf548 to
95835be
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
2 times, most recently
from
August 6, 2026 20:41
37f2180 to
e43f3e9
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 7, 2026 02:49
e43f3e9 to
4430a06
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 7, 2026 05:09
4430a06 to
a30c491
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
August 26, 2026 17:30
a30c491 to
2989aac
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
September 18, 2026 03:09
2989aac to
30e8d2d
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
September 18, 2026 03:46
30e8d2d to
5f38cd6
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
September 18, 2026 17:34
5f38cd6 to
927d857
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
September 18, 2026 17:39
927d857 to
25ab688
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
October 1, 2026 17:11
20d6f8b to
20390f5
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
2 times, most recently
from
October 3, 2026 04:57
0326c60 to
d648d5a
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
2 times, most recently
from
October 4, 2026 06:21
136b0fa to
8484d41
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
October 8, 2026 21:40
8484d41 to
bf82cab
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
2 times, most recently
from
October 9, 2026 05:25
6a4d2c9 to
7e4aa85
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
October 14, 2026 18:45
7e4aa85 to
4f07220
Compare
eriknw
force-pushed
the
38-monoid-guard-udt-fallback-builtins
branch
from
October 19, 2026 07:47
4f07220 to
f4c88db
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 36-monoid-builtin-binaryop-guard #619 +/- ##
====================================================================
+ Coverage 98.70% 98.73% +0.03%
====================================================================
Files 105 105
Lines 29879 29929 +50
Branches 3486 3491 +5
====================================================================
+ Hits 29491 29550 +59
+ Misses 254 249 -5
+ Partials 134 130 -4 🚀 New features to boost your workflow:
|
The guard added with the monoid-from-built-in rejection keyed on "constructed with neither a Python func nor a numba func", which is true for most built-ins but not all of them: binary.any, first, pair, and second carry a Python orig_func as their UDT fallback while their typed ops are still TypedBuiltinBinaryOp. All four sailed past the guard and reproduced exactly what it exists to prevent: an opaque AttributeError partway through registration, with binary.<name>._monoid left pointing at the half-built monoid (binary.any.monoid is monoid.any flipped from True to False after the rejected call). binary.any is even named in the guard's own rationale as an op whose identity SuiteSparse ignores. Key the probe on the typed ops as well: any TypedBuiltinBinaryOp among them marks the op built-in. The two signals agree everywhere else, including under mapnumpy=True, where the mapped numpy binary ops hold the built-in's typed ops and no funcs, and are flagged by both tests alike; the numpy monoids themselves are mapped without passing through Monoid._build, so their registration is unaffected (verified under both mapnumpy settings). A UDF's typed ops are never TypedBuiltinBinaryOp, so nothing legitimate is newly rejected.
…/max
Behavior fix. monoid.min and monoid.max generate a UDT's identity field
by field, and _scalar_identity picked the integer arm with
np.issubdtype(dtype, np.integer). numpy files timedelta64 under
np.integer, so a record with a timedelta64 field reached np.iinfo, which
rejects it:
rec = dtypes.register_anonymous(np.dtype([("n", np.int64), ("t", "m8[s]")]))
monoid.min[rec] # before: ValueError: Invalid integer data type 'm'.
rec in agg.min # before: the same ValueError, not False
A datetime64 field already got "KeyError: monoid.min does not work with
...: field 't' has unsupported dtype datetime64[s]", and `in` answered
False. Test the dtype's kind instead, so a timedelta64 field takes that
path too:
monoid.min[rec] # after: KeyError ... field 't' has unsupported dtype timedelta64[s]
rec in agg.min # after: False
Real identities for timedelta64 fields (its own max and min, with NaT
in mind) are a separate enhancement.
Behavior fix. Monoid._compile_udt handed a binary op whose return type
differs from its input type straight to GrB_Monoid_new_UDT, which
rejects it with a DomainMismatch that has no message, also from `in`:
op = BinaryOp.register_anonymous(lambda x, y: x["a"] + y["a"], is_udt=True)
m = Monoid.register_anonymous(op, 0)
m[rec] # before: DomainMismatch (empty message)
rec in m # before: the same DomainMismatch
Check the compiled op's return type first and raise a TypeError, the
error the neighbouring check (two different input types) raises:
m[rec] # after: TypeError: monoid.<lambda> does not work with rec: a
# monoid's binary operator must return its input type, rec, not INT64
rec in m # after: False
Behavior fix. The lazy branch stored register_new's arguments for the
delayed call but left is_idempotent out, so the monoid built on first
access was never idempotent:
Monoid.register_new("m", op, 0, is_idempotent=True, lazy=True)
monoid.m.is_idempotent # before: False
# after: True
The eager branch already passed it.
The test registers lazy_idem in the binary and monoid namespaces, so
test_operator_types ignores it like the other names test_lazy_op makes.
Behavior fix. Monoid._compile_udt generated a field-wise identity
whenever the monoid had no identity and its name was plus, times, min or
max. A user monoid gets its name from its BinaryOp, which gets it from
the function, so a user op called `min` (or a monoid given name="plus")
silently took the built-in's identity and gave wrong results. With an op
that takes the field-wise max and identity None:
op = BinaryOp.register_anonymous(fieldwise_max, "min", is_udt=True)
v = [(1, 2), (5, -3)]; w = [(-1, -2), (-5, -3)]
Monoid.register_anonymous(op, None) # name "min"
before: identity (INT64_MAX, INT64_MAX); v.reduce -> (INT64_MAX, INT64_MAX)
Monoid.register_anonymous(op, None, "plus")
before: identity (0, 0); v.reduce -> (5, 2), w.reduce -> (0, 0)
both, after: KeyError: monoid.<name> does not work with ...: no identity value
Generate the identity only for a built-in monoid, the ones with no
_binaryop of their own. A user monoid with an identity is unaffected.
test_udt_monoid_op_must_return_its_input_type matched the error against "_BuiltinOpsRec", the name the record_udt fixture registers. But register_anonymous returns the one DataType for an np.dtype and gives it the newest name, and test_udt_op_pickle registers the same record dtype as "_PickleUdt". In random order, with a record_udt test, then test_udt_op_pickle, then this test, the message said "_PickleUdt" and the match failed (wave 2 CI: 13 jobs, e.g. 122634118659 and 122634118406, on both backends and with mapnumpy on or off). Match the fixture's name as it is when the error is raised.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #617. Overnight adversarial review of #617 found a hole in its guard:
_is_builtin_binaryopkeys on "no Python func and no numba func", but four built-ins (binary.any,first,pair,second) carry a Pythonorig_funcas their UDT fallback. All four sailed past the guard and reproduced exactly what #617 exists to prevent:binary.anyis even named in #617's own rationale as an op whose identity SuiteSparse ignores. This is an incomplete fix in #617 rather than a regression: the four common cases (plus/times/min/max) were already rejected cleanly.The fix keys the probe on the typed ops as well: any
TypedBuiltinBinaryOpamong them marks the op built-in. Verified that the two signals agree everywhere else, including undermapnumpy=True(where mapped numpy binary ops hold the built-in's typed ops and are flagged by both tests alike, and the numpy monoids never pass throughMonoid._build), and that a UDF's typed ops are neverTypedBuiltinBinaryOp, so nothing legitimate is newly rejected. The new test is teeth-checked: it fails with the opaque AttributeError without the fix.Gates: full pinned suite 1113 passed / 145 skipped; test_op + test_numpyops green under both mapnumpy settings; pre-commit all hooks pass.
Added in the 2026-10-20 coverage round
Four more monoid bugs, found while testing main's coverage gaps. Each is a behavior fix with before and after in its commit message, and each is on main too:
monoid.minandmonoid.maxon a record with a timedelta64 field raisedValueError: Invalid integer data type 'm'fromnp.iinfo(numpy files timedelta64 undernp.integer), and so didrec in agg.min. They now raise the KeyError a datetime64 field gets, naming the field, andinanswers False.in. It now raises a TypeError saying that a monoid's binary operator must return its input type, andinanswers False.Monoid.register_new(..., is_idempotent=True, lazy=True)droppedis_idempotent, so the monoid built on first access was not idempotent.minthat takes the field-wise max and identity None, reducing (1, 2) and (5, -3) gave (INT64_MAX, INT64_MAX) here and (1, -3) on main. It now raises KeyError "no identity value": identities are generated for the built-in monoids only.Each new test fails with its fix reverted.
A test fix:
test_udt_monoid_op_must_return_its_input_typematched the error against "_BuiltinOpsRec", the name therecord_udtfixture registers. Butregister_anonymousreturns the one DataType for an np.dtype and gives it the newest name, andtest_udt_op_pickleregisters the same record dtype as "_PickleUdt", so in random order the match failed (13 CI jobs, on both backends). It now matches the fixture's name as it is when the error is raised.