Repository navigation
Validate text codec names in development mode - #9481
youknowdot wants to merge 1 commit into
Conversation
Match CPython preflight lookup order before text fast paths and empty decoding. Enable after core codec bootstrap and avoid finalization checks. Restore existing bytes and str development-mode checks through update_lib. Assisted-by: OpenAI Codex: model version unavailable
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [ ] test: cpython/Lib/test/test_bytes.py (TODO: 10) dependencies: dependent tests: (no tests depend on bytes) [x] test: cpython/Lib/test/test_str.py (TODO: 1) dependencies: dependent tests: (no tests depend on str) Legend:
|
Merging this PR will not alter performance
Comparing 
Footnotes
|
| } | ||
|
|
||
| pub(crate) fn finish_core_codec_bootstrap(&self) { | ||
| self.inner.write().core_codecs_ready = true; |
There was a problem hiding this comment.
CPython gates this check on interp->unicode.fs_codec.encoding, which becomes available before stdio initialization—not on general interpreter readiness (guard, initialization order). RustPython currently has no equivalent nullable filesystem-codec state. Its initialized/ready flags become true after stdio; import_bootstrap_complete also controls frozen-import fallback. Registry cache/search-path contents are not substitutes: they can be incomplete during bootstrap and later cleared by users, and a core-only embedding may have no search callbacks.
If we retain the before-stdio behavior, a separate interpreter-local completion marker is therefore the smallest available representation. The current transition occurs only after successful core codec registration, including core-only embeddings. This is a RustPython phase analogue, not identical filesystem-codec initialization, and post-startup test passes alone do not establish lifecycle equivalence.
A small clarification/simplification is possible: determine first whether either name actually requires lookup, and return immediately when both are builtin shortcuts. Only the remaining cases need to read the readiness marker. This preserves encoding-before-error-handler lookup and does not add state or change the bootstrap boundary. I would keep broader startup or synchronization changes out of this patch.
Summary
-X devmode, matching CPython'sunicode_check_encoding_errorslookup order and common-name shortcuts.scripts/update_lib migratefrom CPythond220409a13b77de7a6c30abe16ea90db01528edc.General
codecs.encode/decode, private Python helpers, normal-mode lazy error-handler behavior, and dependencies are unchanged. This implements development-mode checks; it does not equate Rust debug builds with CPython'sPy_DEBUGpolicy.CPython basis
Pinned CPython source:
unicode_check_encoding_errors,PyUnicode_FromEncodedObject,PyUnicode_Decode, andPyUnicode_AsEncodedString. Encoding lookup precedes error-handler lookup; validation failures are propagated before transformation-error notes are added. Codec lookup can invoke registered search functions, but validation does not call encode/decode or the selected error handler.Validation
AI assistance
OpenAI Codex assisted with implementation, CPython source comparison, and validation. The model version was unavailable.