aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Python/marshal.c
Commit message (Collapse)AuthorAge
* gh-131942: Use the Python-specific `Py_DEBUG` macro rather than `_DEBUG` in ↵Xuehai Pan2025-05-08
| | | | Windows-related C code (GH-131944)
* gh-131238: Add explicit includes to pycore headers (#131257)Victor Stinner2025-03-17
|
* gh-127936, PEP 757: Convert marshal module to use import/export API for ints ↵Sergey B Kirpichev2025-01-23
| | | | | | (#128530) Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-125063: marshal: Add version 5, improve documentation (GH-126829)Petr Viktorin2024-11-15
| | | | | | | | | | * Document that slices can be marshalled * Deduplicate and organize the list of supported types in docs * Organize the type code list in marshal.c, to make it more obvious that this is a versioned format * Back-fill some historical info Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125194)Victor Stinner2024-10-09
| | | | | Replace PyUnicode_New(0, 0), PyUnicode_FromString("") and PyUnicode_FromStringAndSize("", 0) with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
* gh-125063: Emit slices as constants in the bytecode compiler (#125064)Michael Droettboom2024-10-08
| | | | | | | | | | | | | | | | | | | | | | | * Make slices marshallable * Emit slices as constants * Update Python/marshal.c Co-authored-by: Peter Bierma <zintensitydev@gmail.com> * Refactor codegen_slice into two functions so it always has the same net effect * Fix for free-threaded builds * Simplify marshal loading of slices * Only return SUCCESS/ERROR from codegen_slice --------- Co-authored-by: Mark Shannon <mark@hotpy.org> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
* gh-121905: Consistently use "floating-point" instead of "floating point" ↵Serhiy Storchaka2024-07-19
| | | | (GH-121907)
* gh-121040: Use __attribute__((fallthrough)) (#121044)Victor Stinner2024-06-27
| | | | | | | | | | | | | Fix warnings when using -Wimplicit-fallthrough compiler flag. Annotate explicitly "fall through" switch cases with a new _Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if available. Replace "fall through" comments with _Py_FALLTHROUGH. Add _Py__has_attribute() macro. No longer define __has_attribute() macro if it's not defined. Move also _Py__has_builtin() at the top of pyport.h. Co-Authored-By: Nikita Sobolev <mail@sobolevn.me>
* gh-113993: Allow interned strings to be mortal, and fix related issues ↵Petr Viktorin2024-06-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. * Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
* gh-116322: Add Py_mod_gil module slot (#116882)Brett Simmers2024-05-03
| | | | | | | | | | | | | | This PR adds the ability to enable the GIL if it was disabled at interpreter startup, and modifies the multi-phase module initialization path to enable the GIL when loading a module, unless that module's spec includes a slot indicating it can run safely without the GIL. PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148. A warning will be issued up to once per interpreter for the first GIL-using module that is loaded. If `-v` is given, a shorter message will be printed to stderr every time a GIL-using module is loaded (including the first one that issues a warning).
* gh-114099: Modify preprocessor symbol usage to support older macOS SDKs ↵Russell Keith-Magee2024-04-19
| | | | | (GH-118073) Co-authored-by: Joshua Root jmr@macports.org
* gh-112069: Add _PySet_NextEntryRef to be thread-safe. (gh-117990)Donghee Na2024-04-19
|
* gh-114099: Additions to standard library to support iOS (GH-117052)Russell Keith-Magee2024-03-28
| | | | | Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Malcolm Smith <smith@chaquo.com> Co-authored-by: Ned Deily <nad@python.org>
* gh-113626: Add allow_code parameter in marshal functions (GH-113648)Serhiy Storchaka2024-01-16
| | | | | Passing allow_code=False prevents serialization and de-serialization of code objects which is incompatible between Python versions.
* gh-106320: Move private _PySet API to the internal API (#107041)Victor Stinner2023-07-22
| | | | | | | | * Add pycore_setobject.h header file. * Move the following API to the internal C API: * _PySet_Dummy * _PySet_NextEntry() * _PySet_Update()
* gh-104922: remove PY_SSIZE_T_CLEAN (#106315)Inada Naoki2023-07-02
|
* gh-101006: Improve error handling when read marshal data (GH-101007)Serhiy Storchaka2023-06-29
| | | | | | | | * EOFError no longer overrides other errors such as MemoryError or OSError at the start of the object. * Raise more relevant error when the NULL object occurs as a code object component. * Minimize an overhead of calling PyErr_Occurred().
* gh-106084: Remove old PyObject call aliases (#106085)Victor Stinner2023-06-26
| | | | | | | | | | | | | | Remove old aliases which were kept backwards compatibility with Python 3.8: * _PyObject_CallMethodNoArgs() * _PyObject_CallMethodOneArg() * _PyObject_CallOneArg() * _PyObject_FastCallDict() * _PyObject_Vectorcall() * _PyObject_VectorcallMethod() * _PyVectorcall_Function() Update code which used these aliases to use new names.
* gh-103906: Remove immortal refcounting in compile/marshal.c (gh-103922)Dong-hee Na2023-06-05
|
* gh-105184: document that marshal functions can fail and need to be checked ↵Irit Katriel2023-06-02
| | | | with PyErr_Occurred (#105185)
* gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)Eric Snow2023-05-05
| | | Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules. We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
* GH-101291: Rearrange the size bits in PyLongObject (GH-102464)Mark Shannon2023-03-22
| | | | | | | | | | * Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts. * Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints. * Add functions to hide some internals of long object, and for setting sign and digit count. * Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().
* GH-101291: Refactor the `PyLongObject` struct into object header and ↵Mark Shannon2023-01-30
| | | | PyLongValue struct. (GH-101292)
* gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)Victor Stinner2022-11-23
| | | Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
* gh-99300: Replace Py_INCREF() with Py_NewRef() (#99530)Victor Stinner2022-11-16
| | | | Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef() and Py_XNewRef().
* gh-99300: Use Py_NewRef() in Python/ directory (#99302)Victor Stinner2022-11-10
| | | | Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in C files of the Python/ directory.
* gh-98925: Lower marshal recursion depth for WASI (GH-98938)Brett Cannon2022-11-01
| | | | | For wasmtime 2.0, the stack depth cost is 6% higher. This causes the default max `marshal` recursion depth to blow the stack. As the default marshal depth is 2000 and Windows is set to 1000, split the difference and choose 1500 for WASI to be safe.
* gh-78214: marshal: Stabilize FLAG_REF usage (GH-8226)Inada Naoki2022-05-04
| | | | | | | | | Use FLAG_REF always for interned strings. Refcounts of interned string is very unstable. When compiling same source, refcounts of interned string in the output may be 1 or >1. It makes FLAG_REF usage unstable. To help reproducible build, use FLAG_REF for interned string even if refcnt(obj)==1.
* GH-88116: Use a compact format to represent end line and column offsets. ↵Mark Shannon2022-04-21
| | | | | | | | | | | | (GH-91666) * Stores all location info in linetable to conform to PEP 626. * Remove column table from code objects. * Remove end-line table from code objects. * Document new location table format
* bpo-35134: Remove the Include/code.h header file (GH-32385)Victor Stinner2022-04-07
| | | | | | Remove the Include/code.h header file. C extensions should only include the main <Python.h> header file. Python.h includes directly Include/cpython/code.h instead.
* bpo-46841: Quicken code in-place (GH-31888)Brandt Bucher2022-03-21
| | | | | | | | | | | | | | | | | | | * Moves the bytecode to the end of the corresponding PyCodeObject, and quickens it in-place. * Removes the almost-always-unused co_varnames, co_freevars, and co_cellvars member caches * _PyOpcode_Deopt is a new mapping from all opcodes to their un-quickened forms. * _PyOpcode_InlineCacheEntries is renamed to _PyOpcode_Caches * _Py_IncrementCountAndMaybeQuicken is renamed to _PyCode_Warmup * _Py_Quicken is renamed to _PyCode_Quicken * _co_quickened is renamed to _co_code_adaptive (and is now a read-only memoryview). * Do not emit unused nonzero opargs anymore in the compiler.
* bpo-46906: Add PyFloat_Pack8() to the C API (GH-31657)Victor Stinner2022-03-12
| | | | | | | | | | | | | | | | | | | | | | Add new functions to pack and unpack C double (serialize and deserialize): * PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8() * PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8() Document these functions and add unit tests. Rename private functions and move them from the internal C API to the public C API: * _PyFloat_Pack2() => PyFloat_Pack2() * _PyFloat_Pack4() => PyFloat_Pack4() * _PyFloat_Pack8() => PyFloat_Pack8() * _PyFloat_Unpack2() => PyFloat_Unpack2() * _PyFloat_Unpack4() => PyFloat_Unpack4() * _PyFloat_Unpack8() => PyFloat_Unpack8() Replace the "unsigned char*" type with "char*" which is more common and easy to use.
* bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized ↵Eric Snow2022-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541
* bpo-35134: Move classobject.h to Include/cpython/ (GH-28968)Victor Stinner2021-10-15
| | | | | | | | | Move classobject.h, context.h, genobject.h and longintrepr.h header files from Include/ to Include/cpython/. Remove redundant "#ifndef Py_LIMITED_API" in context.h. Remove explicit #include "longintrepr.h" in C files. It's not needed, Python.h already includes it.
* bpo-35134: Add Include/cpython/floatobject.h (GH-28957)Victor Stinner2021-10-14
| | | | | Split Include/floatobject.h into sub-files: add Include/cpython/floatobject.h and Include/internal/pycore_floatobject.h.
* bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)Victor Stinner2021-10-12
| | | | | | | * Move _PyObject_CallNoArgs() to pycore_call.h (internal C API). * _ssl, _sqlite and _testcapi extensions now call the public PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs(). * _lsprof extension is now built with Py_BUILD_CORE_MODULE macro defined to get access to internal _PyObject_CallNoArgs().
* bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)Victor Stinner2021-10-12
| | | | | Fix typo in the private _PyObject_CallNoArg() function name: rename it to _PyObject_CallNoArgs() to be consistent with the public function PyObject_CallNoArgs().
* bpo-45094: Add Py_NO_INLINE macro (GH-28140)Victor Stinner2021-09-03
| | | | | * Rename _Py_NO_INLINE macro to Py_NO_INLINE: make it public and document it. * Sort macros in the C API documentation.
* bpo-37596: Clean up the set/frozenset marshalling code (GH-28068)Brandt Bucher2021-08-31
|
* bpo-37596: Make `set` and `frozenset` marshalling deterministic (GH-27926)Brandt Bucher2021-08-25
|
* bpo-44530: Add co_qualname field to PyCodeObject (GH-26941)Gabriele N. Tornetta2021-07-07
|
* bpo-43950: Add code.co_positions (PEP 657) (GH-26955)Pablo Galindo2021-07-02
| | | | | | | | | | | | | | | This PR is part of PEP 657 and augments the compiler to emit ending line numbers as well as starting and ending columns from the AST into compiled code objects. This allows bytecodes to be correlated to the exact source code ranges that generated them. This information is made available through the following public APIs: * The `co_positions` method on code objects. * The C API function `PyCode_Addr2Location`. Co-authored-by: Batuhan Taskaya <isidentical@gmail.com> Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
* Fix compiler errors for unused variables in marshal.c (GH-26977)Pablo Galindo2021-06-30
|
* bpo-41180: Replace marshal code.__new__ audit event with marshal.load[s] and ↵Steve Dower2021-06-30
| | | | marshal.dumps (GH-26961)
* bpo-43693: Turn localspluskinds into an object (GH-26749)Guido van Rossum2021-06-21
| | | Managing it as a bare pointer to malloc'ed bytes is just too awkward in a few places.
* bpo-43693: Un-revert commits 2c1e258 and b2bf2bc. (gh-26577)Eric Snow2021-06-07
| | | | | | | | | | These were reverted in gh-26530 (commit 17c4edc) due to refleaks. * 2c1e258 - Compute deref offsets in compiler (gh-25152) * b2bf2bc - Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388) This change fixes the refleaks. https://bugs.python.org/issue43693
* bpo-43693: Revert commits 2c1e2583fdc4db6b43d163239ea42b0e8394171f and ↵Pablo Galindo2021-06-04
| | | | | | | | | | | | | b2bf2bc1ece673d387341e06c8d3c2bc6e259747 (GH-26530) * Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)" This reverts commit b2bf2bc1ece673d387341e06c8d3c2bc6e259747. * Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)" This reverts commit 2c1e2583fdc4db6b43d163239ea42b0e8394171f. These two commits are breaking the refleak buildbots.
* bpo-43693: Add new internal code objects fields: co_fastlocalnames and ↵Eric Snow2021-06-03
| | | | | | | | | | | | | | | | | co_fastlocalkinds. (gh-26388) A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs. Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars. At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections. This is something we can easily address in the compiler instead. This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free). These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily). Aside from the (mild) performance impact, there are a number of other benefits: * there's now a clear, direct relationship between locals-plus and variables * code that relies on the locals-plus-to-name mapping is simpler * marshaled code objects are smaller and serialize/de-serialize faster Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs). Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array. It would also allow us to shrink marshaled code objects even further. https://bugs.python.org/issue43693
* bpo-43693: Add _PyCode_New(). (gh-26375)Eric Snow2021-05-27
| | | | | This is an internal-only API that helps us manage the many values used to create a code object. https://bugs.python.org/issue43693
* bpo-40222: "Zero cost" exception handling (GH-25729)Mark Shannon2021-05-07
| | | | | | | | "Zero cost" exception handling. * Uses a lookup table to determine how to handle exceptions. * Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements. * Reduces the size of the frame object by about 60%.
generated by cgit v1.2.3 (git 2.39.1) at 2025-06-07 12:47:38 +0000