aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Python/pylifecycle.c
Commit message (Collapse)AuthorAge
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-05
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* gh-109860: Use a New Thread State When Switching Interpreters, When ↵Eric Snow2023-10-03
| | | | | | | | | Necessary (gh-110245) In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread. However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it. One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
* gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994)Eric Snow2023-10-02
| | | This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
* gh-110014: Include explicitly <unistd.h> header (#110155)Victor Stinner2023-09-30
| | | | | | | | | | | * Remove unused <locale.h> includes. * Remove unused <fcntl.h> include in traceback.h. * Remove redundant <assert.h> and <stddef.h> includes. They are already included by "Python.h". * Remove <object.h> include in faulthandler.c. Python.h already includes it. * Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS is defined. * Fix also warnings in pthread_stubs.h: don't redefine macros if they are already defined, like the __NEED_pthread_t macro.
* gh-110079: Remove extern "C" { ...} in C code (#110080)Victor Stinner2023-09-29
|
* gh-109611: Add convenient C API function _PyFile_Flush() (GH-109612)Serhiy Storchaka2023-09-23
|
* GH-108716: Turn off deep-freezing of code objects. (GH-108722)Mark Shannon2023-09-08
|
* gh-108765: Cleanup #include in Python/*.c files (#108977)Victor Stinner2023-09-06
| | | Mention one symbol imported by each #include.
* gh-108634: PyInterpreterState_New() no longer calls Py_FatalError() (#108748)Victor Stinner2023-09-01
| | | | | | | | | | | | pycore_create_interpreter() now returns a status, rather than calling Py_FatalError(). * PyInterpreterState_New() now calls Py_ExitStatusException() instead of calling Py_FatalError() directly. * Replace Py_FatalError() with PyStatus in init_interpreter() and _PyObject_InitState(). * _PyErr_SetFromPyStatus() now raises RuntimeError, instead of ValueError. It can now call PyErr_NoMemory(), raise MemoryError, if it detects _PyStatus_NO_MEMORY() error message.
* gh-108634: Py_TRACE_REFS uses a hash table (#108663)Victor Stinner2023-08-31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python built with "configure --with-trace-refs" (tracing references) is now ABI compatible with Python release build and debug build. Moreover, it now also supports the Limited API. Change Py_TRACE_REFS build: * Remove _PyObject_EXTRA_INIT macro. * The PyObject structure no longer has two extra members (_ob_prev and _ob_next). * Use a hash table (_Py_hashtable_t) to trace references (all objects): PyInterpreterState.object_state.refchain. * Py_TRACE_REFS build is now ABI compatible with release build and debug build. * Limited C API extensions can now be built with Py_TRACE_REFS: xxlimited, xxlimited_35, _testclinic_limited. * No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2() functions to PyModule_Create2TraceRefs() and PyModule_FromDefAndSpec2TraceRefs(). * _Py_PrintReferenceAddresses() is now called before finalize_interp_delete() which deletes the refchain hash table. * test_tracemalloc find_trace() now also filters by size to ignore the memory allocated by _PyRefchain_Trace(). Test changes for Py_TRACE_REFS: * Add test.support.Py_TRACE_REFS constant. * Add test_sys.test_getobjects() to test sys.getobjects() function. * test_exceptions skips test_recursion_normalizing_with_no_memory() and test_memory_error_in_PyErr_PrintEx() if Python is built with Py_TRACE_REFS. * test_repl skips test_no_memory(). * test_capi skisp test_set_nomemory().
* gh-108314: Add PyDict_ContainsString() function (#108323)Victor Stinner2023-08-24
|
* gh-108308: Replace _PyDict_GetItemStringWithError() (#108372)Victor Stinner2023-08-23
| | | | | | | Replace _PyDict_GetItemStringWithError() calls with PyDict_GetItemStringRef() which returns a strong reference to the item. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-108308: Replace PyDict_GetItem() with PyDict_GetItemRef() (#108309)Victor Stinner2023-08-23
| | | | | | | | | Replace PyDict_GetItem() calls with PyDict_GetItemRef() or PyDict_GetItemWithError() to handle errors. * Replace PyLong_AS_LONG() with _PyLong_AsInt() and check for errors. * Check for PyDict_Contains() error. * pycore_init_builtins() checks for _PyType_Lookup() failure.
* Ignore _Py_write_noraise() result: cast to (void) (#108291)Victor Stinner2023-08-22
| | | | Code using _Py_write_noraise() usually cannot report. Ignore errors is the least surprising behavior for users.
* gh-108014: Add Py_IsFinalizing() function (#108032)Victor Stinner2023-08-18
| | | Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for ↵Mark Shannon2023-08-17
| | | | performance. (GH-108036)
* gh-107630: Initialize Each Interpreter's refchain Properly (gh-107733)Eric Snow2023-08-07
| | | This finishes fixing the crashes in Py_TRACE_REFS builds. We missed this part in gh-107567.
* gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)Eric Snow2023-08-03
| | | The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
* gh-106320: Move private _PySet API to the internal API (#107041)Victor Stinner2023-07-22
| | | | | | | | * Add pycore_setobject.h header file. * Move the following API to the internal C API: * _PySet_Dummy * _PySet_NextEntry() * _PySet_Update()
* GH-104584: Miscellaneous fixes for -Xuops (GH-106908)Brandt Bucher2023-07-20
|
* Remove private _PyCodec_Lookup() function (#106269)Victor Stinner2023-06-30
| | | | | | | | | | | | | | | Remove the following private functions of the C API: * _PyCodecInfo_GetIncrementalDecoder() * _PyCodecInfo_GetIncrementalEncoder() * _PyCodec_DecodeText() * _PyCodec_EncodeText() * _PyCodec_Forget() * _PyCodec_Lookup() * _PyCodec_LookupTextEncoding() Move these functions to a new pycore_codecs.h internal header file. These functions are no longer exported.
* gh-106084: Remove _PyObject_CallMethod() function (#106159)Victor Stinner2023-06-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the following private functions from the public C API: * _Py_CheckFunctionResult() * _PyObject_CallMethod() * _PyObject_CallMethodId() * _PyObject_CallMethodIdNoArgs() * _PyObject_CallMethodIdObjArgs() * _PyObject_CallMethodIdOneArg() * _PyObject_MakeTpCall() * _PyObject_VectorcallMethodId() * _PyStack_AsDict() Move these functions to the internal C API (pycore_call.h). No longer export the following functions: * _PyObject_Call() * _PyObject_CallMethod() * _PyObject_CallMethodId() * _PyObject_CallMethodIdObjArgs() * _PyObject_Call_Prepend() * _PyObject_FastCallDictTstate() * _PyStack_AsDict() The following functions are still exported for stdlib shared extensions: * _Py_CheckFunctionResult() * _PyObject_MakeTpCall() Mark the following internal functions as extern: * _PyStack_UnpackDict() * _PyStack_UnpackDict_Free() * _PyStack_UnpackDict_FreeNoDecRef()
* gh-104584: Baby steps towards generating and executing traces (#105924)Guido van Rossum2023-06-26
| | | | | Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose). All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.
* gh-105927: finalize_modules_clear_weaklist() uses _PyWeakref_GET_REF() (#105971)Victor Stinner2023-06-21
| | | | | finalize_modules_clear_weaklist() now holds a strong reference to the module longer than before: replace PyWeakref_GET_OBJECT() with _PyWeakref_GET_REF().
* GH-100987: Allow objects other than code objects as the "executable" of an ↵Mark Shannon2023-06-14
| | | | | | | | | | internal frame. (GH-105727) * Add table describing possible executable classes for out-of-process debuggers. * Remove shim code object creation code as it is no longer needed. * Make lltrace a bit more robust w.r.t. non-standard frames.
* gh-104812: Run Pending Calls in any Thread (gh-104813)Eric Snow2023-06-13
| | | For a while now, pending calls only run in the main thread (in the main interpreter). This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
* gh-105603: Change the PyInterpreterConfig.own gil Field (gh-105620)Eric Snow2023-06-13
| | | We are changing it to be more flexible that a strict bool can be for possible future expanded used cases.
* gh-100227: Lock Around Use of the Global "atexit" State (gh-105514)Eric Snow2023-06-08
| | | The risk of a race with this state is relatively low, but we play it safe anyway.
* gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current ↵Eric Snow2023-06-01
| | | | | | | Thread (gh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.)
* gh-105145: Remove old functions to config Python init (#105154)Victor Stinner2023-06-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the following old functions to configure the Python initialization, deprecated in Python 3.11: * PySys_AddWarnOptionUnicode() * PySys_AddWarnOption() * PySys_AddXOption() * PySys_HasWarnOptions() * PySys_SetArgvEx() * PySys_SetArgv() * PySys_SetPath() * Py_SetPath() * Py_SetProgramName() * Py_SetPythonHome() * Py_SetStandardStreamEncoding() * _Py_SetProgramFullPath() Most of these functions are kept in the stable ABI, except: * Py_SetStandardStreamEncoding() * _Py_SetProgramFullPath() Update Doc/extending/embedding.rst and Doc/extending/extending.rst to use the new PyConfig API. _testembed.c: * check_stdio_details() now sets stdio_encoding and stdio_errors of PyConfig. * Add definitions of functions removed from the API but kept in the stable ABI. * test_init_from_config() and test_init_read_set() now use PyConfig_SetString() instead of PyConfig_SetBytesString(). Remove _Py_ClearStandardStreamEncoding() internal function.
* gh-103295: expose API for writing perf map files (#103546)gsallam2023-05-21
| | | | | Co-authored-by: Aniket Panse <aniketpanse@fb.com> Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Carl Meyer <carl@oddbird.net>
* GH-101520: Move tracemalloc functionality into core, leaving interface in ↵Mark Shannon2023-05-17
| | | | Modules. (#104508)
* gh-103763: Implement PEP 695 (#103764)Jelle Zijlstra2023-05-15
| | | | | | | | | | | | | | This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization ↵Eric Snow2023-05-15
| | | | | (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.
* gh-101819: Isolate `_io` (#101948)Erlend E. Aasland2023-05-15
| | | | Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-99113: Make Sure the GIL is Acquired at the Right Places (gh-104208)Eric Snow2023-05-06
| | | This is a pre-requisite for a per-interpreter GIL. Without it this change isn't strictly necessary. However, there is no real downside otherwise.
* gh-99113: Add PyInterpreterConfig.own_gil (gh-104204)Eric Snow2023-05-05
| | | | | We also add PyInterpreterState.ceval.own_gil to record if the interpreter actually has its own GIL. Note that for now we don't actually respect own_gil; all interpreters still share the one GIL. However, PyInterpreterState.ceval.own_gil does reflect PyInterpreterConfig.own_gil. That lie is a temporary one that we will fix when the GIL really becomes per-interpreter.
* gh-103323: Remove PyRuntimeState_GetThreadState() (#104171)Victor Stinner2023-05-04
| | | | | This function no longer makes sense, since its runtime parameter is no longer used. Use directly _PyThreadState_GET() and _PyInterpreterState_GET() instead.
* gh-104109: Expose Py_NewInterpreterFromConfig() in the Public C-API (gh-104110)Eric Snow2023-05-02
| | | | | We also expose PyInterpreterConfig. This is part of the PEP 684 (per-interpreter GIL) implementation. We will add docs as soon as we can. FYI, I'm adding the new config field for per-interpreter GIL in gh-99114.
* gh-94673: Properly Initialize and Finalize Static Builtin Types for Each ↵Eric Snow2023-05-01
| | | | | Interpreter (gh-104072) Until now, we haven't been initializing nor finalizing the per-interpreter state properly.
* gh-98040: Remove just the `imp` module (#98573)Barry Warsaw2023-04-28
|
* gh-94673: Ensure Builtin Static Types are Readied Properly (gh-103940)Eric Snow2023-04-27
| | | There were cases where we do unnecessary work for builtin static types. This also simplifies some work necessary for a per-interpreter GIL.
* gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)Eric Snow2023-04-24
| | | | | | | | | | | | | | This is strictly about moving the "obmalloc" runtime state from `_PyRuntimeState` to `PyInterpreterState`. Doing so improves isolation between interpreters, specifically most of the memory (incl. objects) allocated for each interpreter's use. This is important for a per-interpreter GIL, but such isolation is valuable even without it. FWIW, a per-interpreter obmalloc is the proverbial canary-in-the-coalmine when it comes to the isolation of objects between interpreters. Any object that leaks (unintentionally) to another interpreter is highly likely to cause a crash (on debug builds at least). That's a useful thing to know, relative to interpreter isolation.
* gh-84436: Implement Immortal Objects (gh-19474)Eddie Elizondo2023-04-22
| | | | | | | | | This is the implementation of PEP683 Motivation: The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime. Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
* gh-101659: Add _Py_AtExit() (gh-103298)Eric Snow2023-04-05
| | | | | The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on gh-101660.)
* gh-100227: Revert gh-102925 "gh-100227: Make the Global Interned Dict Safe ↵Eric Snow2023-03-27
| | | | | | | for Isolated Interpreters" (gh-103063) This reverts commit 87be8d9. This approach to keeping the interned strings safe is turning out to be too complex for my taste (due to obmalloc isolation). For now I'm going with the simpler solution, making the dict per-interpreter. We can revisit that later if we want a sharing solution.
* gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters ↵Eric Snow2023-03-22
| | | | | | | | | (gh-102925) This is effectively two changes. The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.). The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly. Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict). _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters. This is especially important as we move the obmalloc state to each interpreter (gh-101660), as well as, potentially, the GIL (PEP 684). https://github.com/python/cpython/issues/100227
* gh-94673: Isolate the _io module to Each Interpreter (gh-102663)Eric Snow2023-03-21
| | | | | Aside from sys and builtins, _io is the only core builtin module that hasn't been ported to multi-phase init. We may do so later (e.g. gh-101948), but in the meantime we must at least take care of the module's static types properly. (This came up while working on gh-101660.) https://github.com/python/cpython/issues/94673
* gh-98608: Fix Failure-handling in new_interpreter() (gh-102658)Eric Snow2023-03-21
| | | | | The error-handling code in new_interpreter() has been broken for a while. We hadn't noticed because those code mostly doesn't fail. (I noticed while working on gh-101660.) The problem is that we try to clear/delete the newly-created thread/interpreter using itself, which just failed. The solution is to switch back to the calling thread state first. https://github.com/python/cpython/issues/98608
* gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545)Eric Snow2023-03-21
| | | | | Moving it valuable with a per-interpreter GIL. However, it is also useful without one, since it allows us to identify refleaks within a single interpreter or where references are escaping an interpreter. This becomes more important as we move the obmalloc state to PyInterpreterState. https://github.com/python/cpython/issues/102304
generated by cgit v1.2.3 (git 2.39.1) at 2025-06-03 07:42:09 +0000