Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

Fix gh-559 memory growth: build .ss lazily instead of storing it - #598

Open
eriknw wants to merge 2 commits into
53-jit-improvementsfrom
18-ss-namespace-refcycle
Open

eriknw wants to merge 2 commits into
53-jit-improvementsfrom
18-ss-namespace-refcycle

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026

Copy link
Copy Markdown
Member

Every Matrix and Vector was born inside two reference cycles: the stored
self.ss held _parent back to the object, and so did ss.config. Instances
therefore died by the cyclic garbage collector rather than by refcount, and
their C-side GrB buffers accumulated between gc sweeps. With gc disabled, a
scaled version of the gh-559 batched-mxm loop grew without bound. The .ss
namespace is now built per access (a property inside the existing
class_property), nothing is stored on the parent, no cycle forms, and objects
free as soon as their refcount drops.

Two user-visible behavior changes:

A.ss is A.ss is now False. It was True, because .ss was a stored attribute;
each access now returns a fresh namespace object. Code that compares .ss by
identity, or that caches attributes on it, will see the difference.

Assigning A.ss now raises AttributeError. It previously succeeded and
silently replaced the namespace, because "ss" was in slots.

Class-level Matrix.ss and Vector.ss still resolve to the ss class, so the
import_* classmethods are unchanged.

Building the namespace per access costs roughly 215ns against roughly 78ns for
the stored attribute (timeit), so about 1% of a single small mxm plus to_dense
iteration, which runs about 15us. The .ss namespace is typically touched once
per user operation.

Six of the eight new tests fail without the fix. They assert on the reference
graph rather than on process memory, which a functional suite cannot see and
which an RSS delta would measure only statistically: a weakref must be dead the
instant the last strong reference drops with the cyclic collector switched off,
and a batched mxm loop must not raise the number of live Matrix objects
reported by gc.get_objects(). The remaining two cover invariants the fix has
to preserve, class-level access and views whose _parent is set, so they pass
either way.


Stack created with GitHub Stacks CLI • Give Feedback 💬

@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 9146314 to 37dca2b Compare August 4, 2026 16:12
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 37dca2b to 231b1d9 Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 231b1d9 to 609f44d Compare August 5, 2026 03:18
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 609f44d to 3e41cb9 Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 3e41cb9 to 949f5e1 Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 949f5e1 to cce705d Compare August 5, 2026 18:05
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from cce705d to 1706f84 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 1706f84 to be91313 Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from be91313 to 5913c15 Compare August 6, 2026 15:41
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 5913c15 to 03fffef Compare August 6, 2026 20:36
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 03fffef to 1c7e63b Compare August 6, 2026 20:42
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 1c7e63b to bfce0b2 Compare August 7, 2026 02:48
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from bfce0b2 to 7a5cb1e Compare August 7, 2026 05:09
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 7a5cb1e to 088e8fe Compare August 26, 2026 17:30
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch 3 times, most recently from 4ee7535 to e1f1e30 Compare September 26, 2026 05:23
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from e1f1e30 to 68f553d Compare September 26, 2026 05:25
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch 5 times, most recently from 90925bc to 05b4f6f Compare October 3, 2026 04:57
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch 2 times, most recently from 424724b to b33e2ed Compare October 4, 2026 06:21
@eriknw
eriknw removed this pull request from stack #634 October 8, 2026 21:02
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from b33e2ed to 2ce513e Compare October 8, 2026 21:40
@eriknw
eriknw added this pull request to stack #635 October 8, 2026 21:46
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 2ce513e to 6795304 Compare October 9, 2026 03:06
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from 6795304 to a9f7675 Compare October 9, 2026 05:25
@eriknw
eriknw force-pushed the 18-ss-namespace-refcycle branch from a9f7675 to 6c5ca88 Compare October 14, 2026 18:45
@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.64%. Comparing base (1deed77) to head (accb051).

Additional details and impacted files
@@                   Coverage Diff                   @@
##           53-jit-improvements     #598      +/-   ##
=======================================================
+ Coverage                98.49%   98.64%   +0.14%     
=======================================================
  Files                      104      105       +1     
  Lines                    29121    29204      +83     
  Branches                  3439     3438       -1     
=======================================================
+ Hits                     28684    28807     +123     
+ Misses                     273      260      -13     
+ Partials                   164      137      -27     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@eriknw

eriknw commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

Good fix 👍

Every Matrix and Vector was born inside two reference cycles: the stored
self.ss held _parent back to the object, and so did ss.config.  Instances
therefore died by the cyclic garbage collector rather than by refcount, and
their C-side GrB buffers accumulated between gc sweeps.  With gc disabled, a
scaled version of the gh-559 batched-mxm loop grew without bound.  The .ss
namespace is now built per access (a property inside the existing
class_property), nothing is stored on the parent, no cycle forms, and objects
free as soon as their refcount drops.

Two user-visible behavior changes:

  A.ss is A.ss is now False.  It was True, because .ss was a stored attribute;
  each access now returns a fresh namespace object.  Code that compares .ss by
  identity, or that caches attributes on it, will see the difference.

  Assigning A.ss now raises AttributeError.  It previously succeeded and
  silently replaced the namespace, because "ss" was in __slots__.

Class-level Matrix.ss and Vector.ss still resolve to the ss class, so the
import_* classmethods are unchanged.

Building the namespace per access costs roughly 215ns against roughly 78ns for
the stored attribute (timeit), so about 1% of a single small mxm plus to_dense
iteration, which runs about 15us.  The .ss namespace is typically touched once
per user operation.

Six of the eight new tests fail without the fix.  They assert on the reference
graph rather than on process memory, which a functional suite cannot see and
which an RSS delta would measure only statistically: a weakref must be dead the
instant the last strong reference drops with the cyclic collector switched off,
and a batched mxm loop must not raise the number of live Matrix objects
reported by gc.get_objects().  The remaining two cover invariants the fix has
to preserve, class-level access and views whose _parent is set, so they pass
either way.
test_batched_mxm_loop_does_not_accumulate_matrices restores the cyclic
collector only if it was on when the test started. It always is under
pytest, so the other arm of that if never runs and Codecov reported a
partial branch. The same check in _dies_by_reference_count is not
reported: its try block returns, and coverage.py gives a finally block
reached that way only one arc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant