diff options
| author | Petr Viktorin <encukou@gmail.com> | 2024-06-21 17:19:31 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-06-21 17:19:31 +0200 |
| commit | 6f1d448bc110633eda110310fd833bd46e7b30f2 (patch) | |
| tree | 39555edee103339beea386042b7ffcbcd56ddcda /Python/marshal.c | |
| parent | 7595e6743ac78ac0dd19418176f66d251668fafc (diff) | |
| download | cpython-6f1d448bc110633eda110310fd833bd46e7b30f2.tar.gz cpython-6f1d448bc110633eda110310fd833bd46e7b30f2.zip | |
gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520)
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
Diffstat (limited to 'Python/marshal.c')
| -rw-r--r-- | Python/marshal.c | 17 |
1 files changed, 13 insertions, 4 deletions
diff --git a/Python/marshal.c b/Python/marshal.c index ca22d6d679a..a46fc0ce881 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -14,6 +14,7 @@ #include "pycore_long.h" // _PyLong_DigitCount #include "pycore_setobject.h" // _PySet_NextEntry() #include "marshal.h" // Py_MARSHAL_VERSION +#include "pycore_pystate.h" // _PyInterpreterState_GET() #ifdef __APPLE__ # include "TargetConditionals.h" @@ -1184,8 +1185,12 @@ r_object(RFILE *p) v = PyUnicode_FromKindAndData(PyUnicode_1BYTE_KIND, ptr, n); if (v == NULL) break; - if (is_interned) - PyUnicode_InternInPlace(&v); + if (is_interned) { + // marshal is meant to serialize .pyc files with code + // objects, and code-related strings are currently immortal. + PyInterpreterState *interp = _PyInterpreterState_GET(); + _PyUnicode_InternImmortal(interp, &v); + } retval = v; R_REF(retval); break; @@ -1217,8 +1222,12 @@ r_object(RFILE *p) } if (v == NULL) break; - if (is_interned) - PyUnicode_InternInPlace(&v); + if (is_interned) { + // marshal is meant to serialize .pyc files with code + // objects, and code-related strings are currently immortal. + PyInterpreterState *interp = _PyInterpreterState_GET(); + _PyUnicode_InternImmortal(interp, &v); + } retval = v; R_REF(retval); break; |