Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

Extend the monoid guard to built-ins that carry a UDT fallback func - #619

Open
eriknw wants to merge 6 commits into
36-monoid-builtin-binaryop-guardfrom
38-monoid-guard-udt-fallback-builtins
Open

eriknw wants to merge 6 commits into
36-monoid-builtin-binaryop-guardfrom
38-monoid-guard-udt-fallback-builtins

Conversation

@eriknw

@eriknw eriknw commented Aug 5, 2026 •

Copy link
Copy Markdown
Member

Stacked on #617. Overnight adversarial review of #617 found a hole in its guard: _is_builtin_binaryop keys on "no Python func and no numba func", but four built-ins (binary.any, first, pair, second) carry a Python orig_func as their UDT fallback. All four sailed past the guard and reproduced exactly what #617 exists to prevent:

from graphblas import binary, monoid
from graphblas.core.operator.monoid import Monoid

binary.any.monoid is monoid.any        # True
try:
    Monoid.register_anonymous(binary.any, 0)   # opaque AttributeError, not the clear TypeError
except AttributeError:
    pass
binary.any.monoid is monoid.any        # False! left pointing at the half-built monoid

binary.any is even named in #617's own rationale as an op whose identity SuiteSparse ignores. This is an incomplete fix in #617 rather than a regression: the four common cases (plus/times/min/max) were already rejected cleanly.

The fix keys the probe on the typed ops as well: any TypedBuiltinBinaryOp among them marks the op built-in. Verified that the two signals agree everywhere else, including under mapnumpy=True (where mapped numpy binary ops hold the built-in's typed ops and are flagged by both tests alike, and the numpy monoids never pass through Monoid._build), and that a UDF's typed ops are never TypedBuiltinBinaryOp, so nothing legitimate is newly rejected. The new test is teeth-checked: it fails with the opaque AttributeError without the fix.

Gates: full pinned suite 1113 passed / 145 skipped; test_op + test_numpyops green under both mapnumpy settings; pre-commit all hooks pass.

Added in the 2026-10-20 coverage round

Four more monoid bugs, found while testing main's coverage gaps. Each is a behavior fix with before and after in its commit message, and each is on main too:

  • monoid.min and monoid.max on a record with a timedelta64 field raised ValueError: Invalid integer data type 'm' from np.iinfo (numpy files timedelta64 under np.integer), and so did rec in agg.min. They now raise the KeyError a datetime64 field gets, naming the field, and in answers False.
  • A UDT monoid whose binary op returns another type raised a DomainMismatch with no message, also from in. It now raises a TypeError saying that a monoid's binary operator must return its input type, and in answers False.
  • Monoid.register_new(..., is_idempotent=True, lazy=True) dropped is_idempotent, so the monoid built on first access was not idempotent.
  • A field-wise identity was generated for any UDT monoid named plus, times, min or max that had none, so a user op with one of those names, or a monoid given one, silently took the built-in's identity. With an op called min that takes the field-wise max and identity None, reducing (1, 2) and (5, -3) gave (INT64_MAX, INT64_MAX) here and (1, -3) on main. It now raises KeyError "no identity value": identities are generated for the built-in monoids only.

Each new test fails with its fix reverted.

A test fix: test_udt_monoid_op_must_return_its_input_type matched the error against "_BuiltinOpsRec", the name the record_udt fixture registers. But register_anonymous returns the one DataType for an np.dtype and gives it the newest name, and test_udt_op_pickle registers the same record dtype as "_PickleUdt", so in random order the match failed (13 CI jobs, on both backends). It now matches the fixture's name as it is when the error is raised.

@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from ceeeea2 to aaf597e Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from aaf597e to 8e2c61c Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch 2 times, most recently from 56e3ec8 to eef9de4 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from eef9de4 to 3ddf548 Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 3ddf548 to 95835be Compare August 6, 2026 15:41
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch 2 times, most recently from 37f2180 to e43f3e9 Compare August 6, 2026 20:41
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from e43f3e9 to 4430a06 Compare August 7, 2026 02:49
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 4430a06 to a30c491 Compare August 7, 2026 05:09
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from a30c491 to 2989aac Compare August 26, 2026 17:30
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 2989aac to 30e8d2d Compare September 18, 2026 03:09
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 30e8d2d to 5f38cd6 Compare September 18, 2026 03:46
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 5f38cd6 to 927d857 Compare September 18, 2026 17:34
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 927d857 to 25ab688 Compare September 18, 2026 17:39
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 20d6f8b to 20390f5 Compare October 1, 2026 17:11
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch 2 times, most recently from 0326c60 to d648d5a Compare October 3, 2026 04:57
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch 2 times, most recently from 136b0fa to 8484d41 Compare October 4, 2026 06:21
@eriknw
eriknw removed this pull request from stack #634 October 8, 2026 21:02
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 8484d41 to bf82cab Compare October 8, 2026 21:40
@eriknw
eriknw added this pull request to stack #635 October 8, 2026 21:46
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch 2 times, most recently from 6a4d2c9 to 7e4aa85 Compare October 9, 2026 05:25
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 7e4aa85 to 4f07220 Compare October 14, 2026 18:45
@eriknw
eriknw removed this pull request from stack #635 October 19, 2026 07:45
@eriknw
eriknw changed the base branch from 37-networkx-int32-widen-pin to main October 19, 2026 07:46
@eriknw
eriknw force-pushed the 38-monoid-guard-udt-fallback-builtins branch from 4f07220 to f4c88db Compare October 19, 2026 07:47
@eriknw
eriknw changed the base branch from main to 36-monoid-builtin-binaryop-guard October 19, 2026 07:48
@eriknw
eriknw added this pull request to stack #650 October 19, 2026 07:50
@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.73%. Comparing base (e30824c) to head (14b0d86).

Additional details and impacted files
@@                         Coverage Diff                          @@
##           36-monoid-builtin-binaryop-guard     #619      +/-   ##
====================================================================
+ Coverage                             98.70%   98.73%   +0.03%     
====================================================================
  Files                                   105      105              
  Lines                                 29879    29929      +50     
  Branches                               3486     3491       +5     
====================================================================
+ Hits                                  29491    29550      +59     
+ Misses                                  254      249       -5     
+ Partials                                134      130       -4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The guard added with the monoid-from-built-in rejection keyed on
"constructed with neither a Python func nor a numba func", which is true
for most built-ins but not all of them: binary.any, first, pair, and
second carry a Python orig_func as their UDT fallback while their typed
ops are still TypedBuiltinBinaryOp. All four sailed past the guard and
reproduced exactly what it exists to prevent: an opaque AttributeError
partway through registration, with binary.<name>._monoid left pointing
at the half-built monoid (binary.any.monoid is monoid.any flipped from
True to False after the rejected call). binary.any is even named in the
guard's own rationale as an op whose identity SuiteSparse ignores.

Key the probe on the typed ops as well: any TypedBuiltinBinaryOp among
them marks the op built-in. The two signals agree everywhere else,
including under mapnumpy=True, where the mapped numpy binary ops hold
the built-in's typed ops and no funcs, and are flagged by both tests
alike; the numpy monoids themselves are mapped without passing through
Monoid._build, so their registration is unaffected (verified under both
mapnumpy settings). A UDF's typed ops are never TypedBuiltinBinaryOp,
so nothing legitimate is newly rejected.
…/max

Behavior fix. monoid.min and monoid.max generate a UDT's identity field
by field, and _scalar_identity picked the integer arm with
np.issubdtype(dtype, np.integer). numpy files timedelta64 under
np.integer, so a record with a timedelta64 field reached np.iinfo, which
rejects it:

    rec = dtypes.register_anonymous(np.dtype([("n", np.int64), ("t", "m8[s]")]))
    monoid.min[rec]       # before: ValueError: Invalid integer data type 'm'.
    rec in agg.min        # before: the same ValueError, not False

A datetime64 field already got "KeyError: monoid.min does not work with
...: field 't' has unsupported dtype datetime64[s]", and `in` answered
False. Test the dtype's kind instead, so a timedelta64 field takes that
path too:

    monoid.min[rec]       # after: KeyError ... field 't' has unsupported dtype timedelta64[s]
    rec in agg.min        # after: False

Real identities for timedelta64 fields (its own max and min, with NaT
in mind) are a separate enhancement.
Behavior fix. Monoid._compile_udt handed a binary op whose return type
differs from its input type straight to GrB_Monoid_new_UDT, which
rejects it with a DomainMismatch that has no message, also from `in`:

    op = BinaryOp.register_anonymous(lambda x, y: x["a"] + y["a"], is_udt=True)
    m = Monoid.register_anonymous(op, 0)
    m[rec]       # before: DomainMismatch (empty message)
    rec in m     # before: the same DomainMismatch

Check the compiled op's return type first and raise a TypeError, the
error the neighbouring check (two different input types) raises:

    m[rec]       # after: TypeError: monoid.<lambda> does not work with rec: a
                 #   monoid's binary operator must return its input type, rec, not INT64
    rec in m     # after: False
Behavior fix. The lazy branch stored register_new's arguments for the
delayed call but left is_idempotent out, so the monoid built on first
access was never idempotent:

    Monoid.register_new("m", op, 0, is_idempotent=True, lazy=True)
    monoid.m.is_idempotent    # before: False
                              # after: True

The eager branch already passed it.

The test registers lazy_idem in the binary and monoid namespaces, so
test_operator_types ignores it like the other names test_lazy_op makes.
Behavior fix. Monoid._compile_udt generated a field-wise identity
whenever the monoid had no identity and its name was plus, times, min or
max. A user monoid gets its name from its BinaryOp, which gets it from
the function, so a user op called `min` (or a monoid given name="plus")
silently took the built-in's identity and gave wrong results. With an op
that takes the field-wise max and identity None:

    op = BinaryOp.register_anonymous(fieldwise_max, "min", is_udt=True)
    v = [(1, 2), (5, -3)];  w = [(-1, -2), (-5, -3)]
    Monoid.register_anonymous(op, None)          # name "min"
        before: identity (INT64_MAX, INT64_MAX); v.reduce -> (INT64_MAX, INT64_MAX)
    Monoid.register_anonymous(op, None, "plus")
        before: identity (0, 0); v.reduce -> (5, 2), w.reduce -> (0, 0)
    both, after: KeyError: monoid.<name> does not work with ...: no identity value

Generate the identity only for a built-in monoid, the ones with no
_binaryop of their own. A user monoid with an identity is unaffected.
test_udt_monoid_op_must_return_its_input_type matched the error against
"_BuiltinOpsRec", the name the record_udt fixture registers. But
register_anonymous returns the one DataType for an np.dtype and gives it
the newest name, and test_udt_op_pickle registers the same record dtype
as "_PickleUdt". In random order, with a record_udt test, then
test_udt_op_pickle, then this test, the message said "_PickleUdt" and
the match failed (wave 2 CI: 13 jobs, e.g. 122634118659 and
122634118406, on both backends and with mapnumpy on or off). Match the
fixture's name as it is when the error is raised.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant