Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

Add Python 3.15; routine maintenance; zizmor in pre-commit and CI - #636

Open
eriknw wants to merge 17 commits into
11-udt-record-leaf-countfrom
11a-py315-maintenance
Open

eriknw wants to merge 17 commits into
11-udt-record-leaf-countfrom
11a-py315-maintenance

Conversation

@eriknw

@eriknw eriknw commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Maintenance on top of #591, so every PR above it runs on the refreshed CI.
Seventeen commits:

  • Pin the CI interpreter and its ABI in the update-env step: Pin the CI interpreter and its ABI in the update-env step #632,
    cherry-picked unchanged (see its description). Supersedes Pin the CI interpreter and its ABI in the update-env step #632.
  • Bump the GitHub Actions and pin them to commit SHAs, with version
    comments, as afar and python-suitesparse-graphblas do. Dependabot gets a
    7-day cooldown and one grouped PR a week. setup-miniconda's deprecated
    auto-activate-base becomes auto-activate. Supersedes Bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0 #572, Bump actions/upload-artifact from 4 to 7 #573, Bump conda-incubator/setup-miniconda from 3 to 4 #575,
    Bump codecov/codecov-action from 5 to 7 #577, Bump actions/checkout from 6 to 7 #578.
  • Update the pre-commit hooks (black 26.10, ruff 0.17, isort 9, flake8
    7.4, prettier 3.9, ...). pyroma stays at 5.0.1: 5.1 is still a beta. No
    hook reformats or flags any file, here or at the top of the stack.
    Supersedes chore: update pre-commit hooks #579.
  • Add numba 0.68, awkward 2.14, networkx 3.7 and psg 10.5.1.1 to the CI
    pools
    , with check_versions.sh to match.
  • Support Python 3.15. numba 0.68 has 3.15 builds, so numba is in from
    the start: the python_version< guards move to '3.16'. Classifier, black
    target, the pyver choices in test_and_build.yml and imports.yml, and the
    picker's 3.15 rules (scipy >=1.18, pandas >=3.0, awkward >=2.13, numba

    =0.68, psg >=10.5.1.1).

  • Run zizmor in pre-commit and CI, and fix what it reports. A new
    security.yml runs zizmor (with a token, for its online audits) and pysentry
    on PRs that touch pyproject.toml, .pre-commit-config.yaml or .github/, and
    weekly. Fixes: permissions: contents: read on four workflows that had
    the repository's read-write default token; step outputs reach scripts
    through env: instead of ${{ }}. .github/zizmor.yml is gone, with
    nothing left to ignore.
  • Re-date the numba#5395 notes: bool pointers still fail on numba 0.68.
  • Publish to PyPI with Trusted Publishing. The upload moves to its own
    job in the pypi environment (which asks a maintainer to approve it) with
    id-token: write, instead of a PyPI token secret. This assumes the trusted
    publisher registered on PyPI names workflow publish_pypi.yml and
    environment pypi. It removes zizmor's last inline suppression.
  • Leave py315 out of black's target versions for now. On Python 3.15,
    the newest black conda-forge can install is 26.1 (black 26.5 needs a
    pytokens with no 3.15 build yet), which rejects "py315" and failed the
    auto-generated code check on every 3.15 draw. A MAINT note in
    pyproject.toml says when to add it back.
  • Install the drawn psg version on wheel and source draws. psgver was a
    shell variable of the "Update env" step, so "Build extension module", in a
    new shell, installed the newest psg on PyPI for every wheel and source draw,
    whatever the log's "versions:" line said (since Test new wheels for python-suitesparse-graphblas on all OSes #385). It is now a step
    output, read through env:. That brings two kinds of draw to life: a source
    draw asks for the GraphBLAS that conda-forge pairs with its psg (psg
    9.4.5.0's sdist does not compile against GraphBLAS 10.5.1), and psg 7.4.3.2,
    which has manylinux x86_64 wheels only, leaves the wheel pool.
  • Run the JIT tests with --runslow in every Linux and macOS notebooks job.
    Most of test_ssjit.py needs the suitesparse backend, --runslow, Numba and a
    GraphBLAS that can JIT-compile in one pytest run. In one run in four or five
    no job had all four, so Codecov showed about 210 lines of test_ssjit.py as
    missed on Reject record UDT pairs whose leaf counts differ #591. A new step runs test_ssjit.py alone under coverage in the
    ubuntu-latest and macos-latest notebooks jobs, as the Windows-only "More
    tests for coverage" step does. Where Numba or the JIT is missing, those
    tests skip and the step passes.
  • Cancel a CI run's identical twin, keyed on the PR's head commit. When
    one push moves both a PR's head and its base, GitHub often delivers the
    pull_request event twice: in the last three restacks of this stack, 48 of
    111 PR updates got a duplicate Tests and Test imports run. Both workflows
    get a concurrency group keyed on the workflow, the PR number and its head
    commit, with cancel-in-progress. A run for an earlier commit of the PR is
    left to finish, and a push to main is never cancelled. This replaces the
    branch-keyed group test_and_build.yml had commented out since Don't install graphblas=8.0 yet #459.
  • Give the test and notebook steps a timeout: 120 minutes for "Unit
    tests" and "Unit tests (bizarro scalars)" (the slowest seen is 73.5) and 30
    for "Notebooks Execution check", instead of GitHub's default six hours.
  • Keep conda-forge's libomp out of macOS jobs that use the psg wheel. The
    wheel bundles its own libomp. With conda-forge's in the same process too
    (KMP_DUPLICATE_LIB_OK=TRUE lets both start), a GraphBLAS parallel region can
    wait forever in its join barrier, or a worker crashes; a macOS notebooks job
    hung that way in CI. Those draws now install OpenBLAS's pthreads build and
    set NUMBA_THREADING_LAYER=workqueue, closing conda-forge's two routes to its
    libomp. Draws that pin scipy 1.9 to 1.11 keep the openmp build, which those
    scipys require.
  • Give the other test steps a 30-minute timeout, as the notebooks step
    has
    : "Miscellaneous tests", "More tests for coverage" and "JIT tests
    for coverage" run graphblas too, so a hang like the libomp one above could
    hold their runner for six hours as well. In two green CI runs of Add an experimental n-dimensional Tensor (gb.Tensor, gb.einsum) #651,
    "Miscellaneous tests" took 0.1 to 0.5 minutes and "More tests for
    coverage" 1.6 to 2.1. "JIT tests for coverage" has not run in CI yet; with
    a cold JIT cache it took 62 seconds locally.
  • Add psg 10.5.2.0 to the CI picker. It has PyPI wheels, conda-forge
    builds and an sdist for Python 3.11 to 3.15 on all five CI platforms;
    check_versions.sh now looks for psg >=10.5.2. --validate: 200000
    combinations, 0 failures.
  • Install the GraphBLAS each pinned psg was built for. A conda-forge or
    source draw of psg X.Y.Z.W now also installs graphblas >=X.Y.Z,<X.Y.(Z+1)
    (source draws allowed <X.(Y+1) before). Since conda-forge published
    GraphBLAS 10.5.2, draws of psg 10.5.0.0 and 10.5.1.1 had been getting the
    10.5.2 library, a pairing psg was never built or tested for. Every psg in
    the pools has its GraphBLAS X.Y.Z on conda-forge on all five platforms.

Checked locally: pre-commit at every commit; zizmor reports nothing; the full
suite with --runslow passes on Python 3.14 and on 3.15 with both the newest
and the oldest pins, without numba, and in the C-scalar pass; 60 of 60
dry-run conda solves of the CI install succeed on all five platforms, all on
GIL builds. The later commits' messages give their own checks: dry-run solves
of the changed installs on the platforms they affect, ci_pick_versions.py --validate, and local runs of the new JIT step and of the libomp fix. The
workflow changes themselves can only be tested here.

The codecov step's token: input, which turns coverage reports back on, is
in #591; this PR only pins the action's version. The cap that keeps an
unpinned pandas below 3.1 in CI is on main, from #658.

@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.83%. Comparing base (7efd40b) to head (318fd8c).

Additional details and impacted files
@@                     Coverage Diff                      @@
##           11-udt-record-leaf-count     #636      +/-   ##
============================================================
+ Coverage                     98.65%   98.83%   +0.17%     
============================================================
  Files                           104      104              
  Lines                         28923    28923              
  Branches                       3417     3417              
============================================================
+ Hits                          28535    28586      +51     
+ Misses                          246      215      -31     
+ Partials                        142      122      -20     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@eriknw
eriknw added this pull request to stack #650 October 19, 2026 07:50
@eriknw

eriknw commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

LGTM

eriknw added 11 commits October 28, 2026 20:21
conda-forge ships two ABIs for CPython 3.13+, and the "Update env" step did not
restate ``python``, so the solver was free to replace the interpreter
setup-miniconda installed. Three of the last sixteen python 3.14 draws on
ubuntu-24.04-arm did exactly that: the log reads ``python
3.14.7-h6bfacdd_100_cp314 --> 3.14.7-hefd156b_0_cp314t`` and pytest then reports
"Python: 3.14.7 free-threading build".

Two of the three passed only because they also drew pandas 3.0. The third drew
pandas 2.3, whose extension raises "The global interpreter lock (GIL) has been
enabled to load module 'pandas._libs.pandas_parser'", and ``filterwarnings =
error`` turns that into a collection error. It took test_formatting.py and
test_recorder.py down with it, the second only because the first had left pandas
half-imported.

The window opened on 2026-08-19, when conda-forge uploaded python 3.14.7 for
linux-aarch64 in both ABIs. Re-solving the failing job's own spec set per
platform: linux-aarch64 and osx-arm64 both pick ``cp314t`` today, so
macos-latest is on the same fuse and has simply not drawn 3.14 with an unlucky
pandas yet. linux-64, osx-64 and win-64 pick the GIL build either way, which is
why this has been an arm-only symptom.

``python-gil`` pins ``python_abi``; restating ``python=`` pins the version,
since ``python-gil`` alone constrains only the ABI and its own version is left
open. setup-miniconda's ``python-version`` is equally ABI-free, so this step is
where both get settled. Re-solved with the pair added: every platform lands on a
``cp314`` build, and no draw becomes unsatisfiable or downgrades numpy, numba,
pandas or psg, including python 3.11 with numpy 1.24 and scipy 1.9. python-gil
has builds for 3.9 through 3.14, so one spec covers every version in the pool.
The pypy guard matches the neighbouring lines: ``pypy python-gil`` is
unsatisfiable, so an unguarded spec would hard-fail a pypy draw if one is ever
added back.

python-suitesparse-graphblas pins the same pair in
``.github/workflows/test.yml``, and asks for ``python-freethreading`` on the
matrix entries that want the other ABI. An entry like that would be worth having
here too, but it is a coverage decision rather than a fix, so it is not in this
commit.
Every action moves to its newest release: checkout 7.0.1, setup-python 7.0.0,
setup-miniconda 4.1.0, codecov-action 7.1.1, upload-artifact 7.0.2,
gh-action-pypi-publish 1.14.2, and action-tmate 3.24 (pre-commit/action stays
at 3.0.1, its newest). This supersedes the open Dependabot PRs #572, #573,
#575, #577 and #578.

Each is pinned to the commit of its release, with the version in a comment,
since a tag can be moved to other code. Dependabot updates the pin and the
comment together. The SHAs are the same ones python-suitesparse-graphblas and
afar pin. Dependabot now waits seven days after a release before proposing it
and groups all action bumps into one PR a week, as those two repositories do.

None of the major bumps changes an input this repository uses. Most are the
move to Node.js 24; setup-python 7 drops the ``pip-install`` input, checkout 7
refuses fork checkouts under pull_request_target and workflow_run, and
upload-artifact 7 adds unzipped single-file uploads. One input changes:

* setup-miniconda deprecated ``auto-activate-base`` in favor of
  ``auto-activate`` (already in 3.3), and every run logged a warning for it.
pre-commit autoupdate: isort 9.0.2, pyupgrade 3.22.0, black 26.10.1, ruff
0.17.1, flake8 7.4.1, prettier 3.9.10 and check-jsonschema 0.38.2, with
flake8's hand-pinned plugins moved to flake8-bugbear 26.9.30 and
flake8-simplify 0.31.1 (and scripts/check_versions.sh with them). This
supersedes pre-commit.ci's #579, which predates black 26.10.1, ruff 0.17 and
prettier 3.9.10.

pyroma stays on 5.0.1: autoupdate offers 5.1b1, and 5.1 is still in beta on
PyPI (5.1b2), which is what the comment above it waits for.

None of the new versions changes or flags any file, here or at the top of the
current stack.
… pools

Each is the newest release on conda-forge, and the audits named all four as
newer than any pin.

* numba 0.68 caps numpy at <2.6, the same as 0.67 (conda-forge's
  ``depends``).
* awkward 2.14 needs awkward-cpp 57 and Python >=3.11, which every Python in
  the pool has.
* networkx 3.7 requires Python >=3.12, so py3.11's pool stops at 3.6. It
  constrains numpy, scipy and pandas exactly as 3.5 and 3.6 do, so the
  existing ``>= (3, 5)`` rule covers it; audit_conda_versions.py now checks
  3.7's constraints too.
* psg 10.5.1.1 is the newest build of SuiteSparse:GraphBLAS 10.5.1, with
  numpy 2 builds for every Python we test.

numpy (2.5), scipy (1.18), pandas (3.0), sparse (0.19) and pyyaml (6.0) have
no newer series. scripts/check_versions.sh moves to the new minimums.

The rest of the files with dependency versions need no change: pyproject.toml
holds floors, environment.yml and dev-requirements.txt are unpinned, and
binder/environment.yml and docs/env.yml pin Pythons that are still
supported.

`ci_pick_versions.py --validate`, audit_pypi_versions.py --samples 200 and
audit_conda_versions.py --samples 100 all pass.
conda-forge has Python 3.15.0 final (cp315 and cp315t), and every dependency
CI installs has py315 builds: numba 0.68, numpy 2.5.2+, scipy 1.18.1, pandas
3.0.6, awkward 2.13+, psg 10.5.1.1, matplotlib 3.11.2 and coverage 7.16
(compiled, so the tracer works). PyPI has cp315 wheels of all of them except
pyyaml, which pip builds from its sdist. Numba supports 3.15 from the start,
so this is a single step rather than the usual two: the numba guards in
pyproject.toml move to `python_version<'3.16'`.

* pyproject.toml: the 3.15 classifier, the numba guards (all five, sparse's
  included), and py315 in black's target-version (black 26.10 knows it).
* test_and_build.yml and imports.yml draw 3.15. The interpreter pin from the
  update-env step makes it the GIL build (cp315).
* ci_pick_versions.py gets a 3.15 row in every per-Python table, and three
  rules for what has no py315 build: scipy <1.18, pandas <3.0, and awkward
  <2.13 (awkward is noarch, but each release pins one awkward-cpp, and 56 is
  the first with py315 builds). numba starts at 0.68 and psg at 10.5.1.1 for
  every source: its sdist is the first from after psg added 3.15 support.
  The validator learned the same three rules and now names the floor each
  failure needs; disabling any one of the new rules makes `--validate` fail.

Checked: `--validate` passes (200000 combos), as do audit_pypi_versions.py
and audit_conda_versions.py. A dry-run solve of the update-env install, built
the way the workflow builds it, succeeds for 60 draws: 3.15 on all five CI
platforms with every source and slowtask (numba skipped too), and 3.11-3.14
on four platforms. Every one lands on a GIL interpreter. The full test suite
with --runslow passes on 3.15 with the newest pins (numpy 2.5.4, numba 0.68.0,
psg 10.5.1.1, scipy 1.18.1, pandas 3.0.6, networkx 3.7, awkward 2.14.0,
sparse 0.19.2).
zizmor audits GitHub Actions workflows for security problems. It runs as a
pre-commit hook (and so on pre-commit.ci, offline), and security.yml runs it
again with a token for its online audits: advisories against the actions we
use, pins to commits outside an action's repository, and version comments
that no longer match their pin. security.yml also runs pysentry, which
pre-commit.ci skips for lack of network. Both run on PRs that touch
pyproject.toml, .pre-commit-config.yaml or .github/, and weekly for
advisories published since. The setup follows afar's.

pysentry now runs when pyproject.toml or .pre-commit-config.yaml changes
rather than at every commit, where it fails offline, and with
``--direct-only`` it audits the floors in pyproject.toml instead of the
newest versions uv resolves, which would hide a floor that allows a
vulnerable release. It finds nothing either way today.

What zizmor reported, and what changed:

* excessive-permissions: test_and_build.yml, imports.yml, debug.yml and
  publish_pypi.yml had no ``permissions:``, so their jobs got this
  repository's default token, which is read-write on everything. They now
  ask for ``contents: read``, which is all checkout needs; uploads to
  Codecov and of artifacts do not use the token.
* template-injection: the "Update env" and "Build extension module" steps
  pasted step outputs into their scripts with ``${{ }}``. The values come
  from fixed lists, so nothing could be injected, but they now reach the
  scripts as environment variables. That includes the
  ``format('python={0} python-gil', ...)`` spec, which zizmor does not
  flag but which is the same pattern.
* use-trusted-publishing: suppressed inline on the token, with a TODO. A
  ``pypi`` environment with required reviewers already exists, but moving
  to trusted publishing needs a publisher registered on PyPI, which cannot
  be checked from here.
* unpinned-uses and dependabot-cooldown were fixed by the previous commits.

.github/zizmor.yml ignored all three of the first findings for whole files;
nothing is left for it to ignore, so it is gone. Offline and online, zizmor
now reports no findings (one ignored, the inline suppression). The default
persona is the one afar uses; the pedantic one would also flag the matrix
values in run steps, which are not attacker-controlled.
The two MAINT notes on the BOOL-to-INT8 routing asked for a re-test. A
cfunc that reads ``CPointer(boolean)`` still fails with "cond is not i1: i8",
and one that writes it with "cannot store i1 to i8*", on Numba 0.66 and on
0.68, the release that adds Python 3.15. The routing stays; only the notes'
dates and versions change.
The publish step used a PyPI API token stored as a secret. A trusted
publisher is now registered on PyPI for this repository, so the upload
moves to its own job that exchanges a short-lived OIDC token instead: only
that job gets `id-token: write`, and it runs in the `pypi` environment,
which the PyPI registration names and which asks a maintainer to approve
the upload. The build job is unchanged and hands the files over as the
`releases` artifact it already uploads. This removes zizmor's last inline
suppression.
CI's bizarro job installs black from conda-forge, and on Python 3.15 the
newest black it can install there is 26.1, because the pytokens >=0.4 that
black 26.5 needs has no 3.15 build yet. Black 26.1 does not know "py315" and
exits with an error, so the auto-generated code check failed on every 3.15
draw. Any other black older than 26.5 refused to run on this repository for
the same reason. Nothing here uses syntax new in 3.15, so the formatting is
the same; a MAINT note says when to add it back.
The version picker draws a python-suitesparse-graphblas version for every
source type, but ``psgver`` was a shell variable of the "Update env" step.
"Build extension module", which pip-installs psg for wheel and source draws,
runs in a new shell, where ``suitesparse-graphblas${psgver}`` expanded to
plain ``suitesparse-graphblas``. So every wheel and source draw installed
the newest psg on PyPI, whatever the "versions:" line in its log said; only
conda-forge draws, which install psg in "Update env" itself, got the version
they drew. It has been this way since wheel and source draws began in #385.

It turned up in the CI logs read for why test_ssjit.py's JIT tests skip in
CI. In all four wheel and source jobs there that drew a pin (psg 8.0.2.1,
9.4.5.0, 10.1.1.0 and 10.3.1.0), pip installed 10.5.1.1.

"Update env" now passes psgver on as a step output, and "Build extension
module" reads it through ``env:`` like the other step outputs (zizmor's
template-injection audit). An output rather than ``$GITHUB_ENV``, so no
other step sees it.

Installing the drawn version brings two kinds of draw to life for the first
time, and neither could have worked:

* source: psg's sdist compiles against conda-forge's GraphBLAS, which was
  always the newest one (``graphblas>=7.4``). psg 9.4.5.0's sdist does not
  compile against GraphBLAS 10.5.1 (undeclared GxB_*_GPU_ID constants,
  changed GxB_Iterator and *_get signatures); against 9.4.5 it builds and
  imports. A source draw now asks for the GraphBLAS that conda-forge's
  build of the same psg depends on: python-suitesparse-graphblas X.Y.Z.W
  requires graphblas >=X.Y.Z,<X.(Y+1). An unpinned draw still gets the
  newest of both.
* wheel: psg 7.4.3.2 has manylinux x86_64 wheels only, and the picker, which
  does not know the OS, offered it to every Python 3.11 wheel draw with
  numpy 1.x. It leaves the wheel pool; conda-forge draws still test 7.4.

No other step after "Update env" reads a picked version.

Checked:

* PyPI has wheels on all five runner platforms (manylinux x86_64 and
  aarch64, macOS arm64 and x86_64, win_amd64) for every psg in the wheel
  pools, for every Python each one is offered on.
* A dry-run solve of the "Update env" install, built the way the workflow
  builds it, succeeds for source draws of psg 9.3.1.0 (Python 3.11),
  9.4.5.0 (3.13), 10.0.1.1 (3.12), 10.1.1.0 (3.13), 10.3.1.0 (3.14),
  10.4.0.0 (3.11) and 10.5.1.1 (3.15) on all five platforms, each landing
  on the GraphBLAS that conda-forge pairs with it.
* Locally (macOS arm64, Python 3.12), psg's 9.4.5.0 sdist builds and imports
  against conda-forge's graphblas 9.4.5 and fails against 10.5.1, while
  10.4.0.0's builds against 10.5.1 too.
* test_vector, test_matrix, test_scalar, test_op and test_ssjit (with
  --runslow) pass on psg's 8.0.2.1 wheel with numpy 1.25 (Python 3.11) and
  its 9.3.1.0 wheel with numpy 2.3 (Python 3.12).
* ``ci_pick_versions.py --validate`` passes.
Most of test_ssjit.py needs four things in one pytest run: the suitesparse
backend, --runslow, Numba, and a GraphBLAS that can JIT-compile. The backend
is drawn per pass, --runslow is on in one pass of each pytest job and in
neither pass of a notebooks job, Numba is skipped at random outside the
notebooks jobs, and psg's wheels never JIT. In one run in four or five no
job had all four, so codecov reported about 210 lines of test_ssjit.py as
missed on #591 even though the JIT works in CI.

That came from reading CI logs for why those tests skip. The C compiler is
fine: every Linux and macOS job on conda-forge's GraphBLAS (conda-forge,
source and upstream draws) passes the JIT probe, and
scripts/jit_diagnostics.py prints "Result: True". The skips come from psg's
wheels, which build GraphBLAS with GRAPHBLAS_USE_JIT=OFF, and from Windows,
where conda-forge's GraphBLAS names the build machine's cl.exe.

A new step runs test_ssjit.py on its own, under coverage, with
--runslow and --backend=suitesparse, in the ubuntu-latest and macos-latest
notebooks jobs, before the coverage report and upload. Those jobs install
Numba unless the version picker rules it out (some source draws on Python
3.11 and 3.12), and draw psg from all four sources. It mirrors the
Windows-only "More tests for coverage" step, and runs the file in order
(``-p no:randomly``), as local runs do.

No condition on Numba or the JIT: where either is missing, the tests that
need it skip themselves and the rest still run and count, so the step
passes. Locally (macOS arm64), the step's command:

* psg 10.5 with Numba and a working JIT: 48 passed, in about 45 s.
* The same with Numba blocked: 19 passed, 29 skipped.
* conda-forge's psg 9.3.1.0, 8.2.1.0 and 8.0.2.1, each with Numba and
  c-compiler as a notebooks job installs them: 47, 32 and 32 passed, the
  rest skipped for needing a newer SuiteSparse:GraphBLAS.
* psg's 8.0.2.1 and 9.3.1.0 wheels, without Numba: all passed or skipped.
When one push moves both a PR's head and its base branch, GitHub often
delivers that PR's pull_request event twice, and Tests and Test imports each
ran twice on the same merge commit. Restacking this repository's PR stack
moves every branch in one push: in the last three restacks, 37 PRs had their
base moved each time, and 48 of those 111 PR updates got a duplicate run
(15, 13 and 20), the twins arriving up to about 50 s apart. It is GitHub's
doing, not the push: gh-stack pushes once, with one ``git push`` and no API
call after it, and an ``--atomic`` push got the same duplicates. The PRs that
got one show ``base_ref_force_pushed`` beside ``head_ref_force_pushed`` on
their timelines. This came out of tracing the 40 queued duplicates left
after the last restack. The Security workflow, whose concurrency group
cancels the first twin within seconds, never kept one.

Both workflows now have a concurrency group keyed on the workflow, the PR
number and the PR's head commit, with cancel-in-progress, so the later twin
cancels the earlier one, which is nearly always still queued:

* A run for an earlier commit of the same PR is in another group and is left
  to finish, as before. The group test_and_build.yml had commented out since
  #459, keyed on the branch, would cancel it; this replaces it.
* A push to main has no pull_request, so the group falls back to the run id
  and the pushed commit: each push's run is a group of its own and is never
  cancelled. Neither workflow has another trigger; workflow_dispatch would
  key on its run id the same way.
* A re-run keeps its run's group, so re-running a cancelled twin while the
  other is still going cancels that one.
A run that hangs holds its runner for GitHub's default six hours. One way to
hang was seen locally: on macOS, a conda environment with a pip
suitesparse-graphblas wheel loads two copies of libomp (the workflow sets
KMP_DUPLICATE_LIB_OK=TRUE to allow that), and a full test run sat at 0% CPU
for seven hours in SuiteSparse's OpenMP join barrier.

"Unit tests" and "Unit tests (bizarro scalars)" now stop after 120 minutes
(the slowest seen is 73.5 minutes, a full --runslow on macos-15-intel with
the Tensor PR higher in the stack), and "Notebooks Execution check" after 30.
The suitesparse-graphblas wheel for macOS bundles its own libomp. When conda-forge's
libomp is in the same process too (KMP_DUPLICATE_LIB_OK=TRUE lets both start), the two
runtimes call into each other's barrier code: a GraphBLAS parallel region (a large
calloc's memset) can wait forever in its join barrier, or a worker crashes. A macOS
notebooks job hung that way in CI, and locally, with the wheel, numpy 2.1, scipy 1.13 and
numba 0.68, the Tensor test modules hung at the same test in every run (2 of 2), also with
NUMBA_THREADING_LAYER=workqueue (1 of 1).

conda-forge's libomp gets in two ways: OpenBLAS's openmp build (loaded with numpy) and
Numba's default "omp" threading layer. With both replaced, by OpenBLAS's pthreads build
and Numba's "workqueue" layer, the same runs finished (3 of 3, two seeds; 529 passed),
and the wheel's libomp was the only OpenMP runtime the process loaded. With the pthreads
build alone, Numba's omp layer still loaded conda-forge's libomp, and the run crashed in
the first parallel Numba kernel.

So "Update env" adds libopenblas=*=*pthreads* to the conda install and sets
NUMBA_THREADING_LAYER=workqueue for later steps on macOS draws of the wheel. conda-forge
has macOS pthreads builds from libopenblas 0.3.30 on only, and scipy 1.9-1.11 require an
older libopenblas, so draws pinning those keep the openmp build. Dry-run solves of the
install line with the pin resolve on osx-arm64 and osx-64 for Python 3.11 to 3.15
(libopenblas 0.3.34 pthreads).
"Miscellaneous tests", "More tests for coverage" and "JIT tests for
coverage" run graphblas too, so the hang the unit-test timeouts guard
against (two libomp copies on macOS) could hold their runner for the
6-hour limit as well. In the two green CI runs of 50-tensor on
2026-10-27, "Miscellaneous tests" took 0.1 to 0.5 minutes and "More tests
for coverage" 1.6 to 2.1. "JIT tests for coverage" has not run in CI yet;
with a cold JIT cache at the top of the stack it took 62 seconds locally,
compiling 227 kernels.
psg 10.5.2.0 is on PyPI and conda-forge. PyPI has cp311 to cp315 wheels for
every platform CI runs on (manylinux x86_64 and aarch64, macOS arm64 and
x86_64, win_amd64), and the sdist. conda-forge has builds for py311 to py315
on linux-64, linux-aarch64, osx-64, osx-arm64 and win-64, all depending on
graphblas >=10.5.2,<10.6, which conda-forge has on all five, so a "source"
draw's graphblas pin solves too. The builds need numpy >=1.24 (py3.11) or
>=1.25 (py3.12 on), which the numpy 2.x pool this list feeds always meets.

No Python floor moves: 10.5.2.0 builds for every Python 3.11 to 3.15 on every
source, so PSG_MIN_PYTHON stays as it is.
…ource draws

python-suitesparse-graphblas X.Y.Z.W wraps SuiteSparse:GraphBLAS X.Y.Z, and psg's
own test_version_tracks_libgraphblas asserts that the two match. conda-forge's psg
builds accept any graphblas >=X.Y.Z,<X.(Y+1), and so did this workflow's pin for
source draws, so once conda-forge published graphblas 10.5.2 (2026-10-28), a
conda-forge draw of psg 10.5.1.1 or 10.5.0.0 solved to graphblas 10.5.2, and a
source draw compiled the 10.5.1.1 or 10.5.0.0 sdist against 10.5.2's headers. The
"python-suitesparse-graphblas tests" step then reports
`assert (10, 5, 1) == (10, 5, 2)`, and the graphblas tests run against a library
the psg in the draw was not built for.

Pin graphblas to >=X.Y.Z,<X.Y.(Z+1) for a pinned psg on conda-forge and source
draws. Every psg pin in the pools (7.4.0 to 10.5.2.0) depends on graphblas
>=X.Y.Z on conda-forge, and graphblas X.Y.Z is there for linux-64,
linux-aarch64, osx-64, osx-arm64 and win-64. Unpinned psg, upstream and wheel
draws are unchanged.

Checked with dry-run solves of CI's spec list, 17 of 17 landing on graphblas
X.Y.Z: conda-forge draws of psg 10.5.1.1 on py3.15 on all five platforms, of an old
psg on py3.11 on each (7.4.0 on linux-64 and osx-arm64, 7.4.3.2 on linux-aarch64,
8.0.2.1 on osx-64, 7.4.1 on win-64), of 8.2.1.0, 9.3.1.0, 9.4.5.0 and 10.5.2.0,
and source draws of 9.4.5.0, 10.3.1.0 and 10.5.1.1. Without the pin, a py3.13
osx-arm64 conda-forge draw of 10.5.1.1 solves to graphblas 10.5.2.
@eriknw
eriknw force-pushed the 11a-py315-maintenance branch from 27654db to 318fd8c Compare October 29, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant