bpo-41631: _ast module uses again a global state (#21961)

Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c:
"bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)".

Using a module state per module instance is causing subtle practical
problems.

For example, the Mercurial project replaces the __import__() function
to implement lazy import, whereas Python expected that "import _ast"
always return a fully initialized _ast module.

Add _PyAST_Fini() to clear the state at exit.

The _ast module has no state (set _astmodule.m_size to 0). Remove
astmodule_traverse(), astmodule_clear() and astmodule_free()
functions.

1 files changed, 6 insertions, 0 deletions

diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
index ab5a676786..75d57805c0 100644
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -1259,6 +1259,12 @@ flush_std_files(void)
 static void
 finalize_interp_types(PyThreadState *tstate)
 {
+    // The _ast module state is shared by all interpreters.
+    // The state must only be cleared by the main interpreter.
+    if (_Py_IsMainInterpreter(tstate)) {
+        _PyAST_Fini(tstate);
+    }
+
     _PyExc_Fini(tstate);
     _PyFrame_Fini(tstate);
     _PyAsyncGen_Fini(tstate);