Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wide Struct maps with reverse or shuffled keys repeatedly scan field names after the expected-field comparison fails. Add a lazy shared
StrLookupfor map Structs with at least 64 fields, while retaining that comparison and the linear fallback for input keys longer than 64 UTF-8 bytes. Build the table during type preparation, protect initialization with a CPython critical section, traverse/clear it with the class, and recheck cached type information after reentrant allocations.Adds semantic tests across JSON, MessagePack, TOML, YAML and
convert, plus JSON/MessagePack benchmarks in the existing performance suite. No public API changes. Related: #4 and #119.Measured on a new cloud VM with 6 dedicated vCPU under KVM, using one pinned core per microbenchmark: CPython 3.14.7 with GIL, GCC 14.2.0, identical Linux x86-64 builds. The C source in this PR matches the measured candidate. For 256 fields:
Full matrix: change in mean decoding time, candidate versus base
Negative values mean faster. Each row includes all three input orders.
The 72 cases use flat Structs with ASCII keys
field_0000onward, integer values of 1 and a fixed shuffle seed of 42; separate unique-value controls verify field routing. JSON/MessagePack use prepared Decoders; TOML/YAML include parsing and conversion (stdlibtomllib, PyYAML 6.0.3CSafeLoader). Type/Decoder setup is excluded; instance allocation is included. Two rounds alternate A/B/B/A and B/A/A/B, with 576 jobs and 80 timed values per arm/case, using pyperf.The gains apply to wide unordered inputs, not applications generally. Small unordered MessagePack cases still take up to 1.60% longer; ordered MessagePack means are 0.10-1.91% lower here, with mixed round signs at 32 fields. All observations are retained, including 288 jobs with pyperf warnings and two flagged for throttling. Worker values are clustered; small differences do not establish equivalence or portability. These results are separate from earlier ARM64 runs.
The change adds one pointer to each Struct class and an additional lazy table for wide map types; cold preparation and retained memory are not included in the timing table. Known input keys longer than the hashing cap retain linear lookup. The thresholds are implementation choices, not public guarantees.
Validation: fresh native macOS ARM64 and local Linux ARM64 builds each passed 6737 unit tests (154 skipped), including 159 new cases. Typing compatibility checks, stubtest and project hooks pass. The full Linux suite also passes ASan/UBSan (leak detection disabled). Free-threaded CPython 3.14.6 checks cover concurrent first preparation and the 159 semantic cases with the GIL disabled. Source-wide mypy produces the same 91 diagnostics as the base. Windows and GitHub CI are pending.
Closes #1289.