bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)

When trying to extract the error line for the error message there
are two distinct cases:

1. The input comes from a file, which means that we can extract the
   error line by using `PyErr_ProgramTextObject` and which we already
   do.
2. The input does not come from a file, at which point we need to get
   the source code from the tokenizer:
   * If the tokenizer's current line number is the same with the line
     of the error, we get the line from `tok->buf` and we're ready.
   * Else, we can extract the error line from the source code in the
     following two ways:
     * If the input comes from a string we have all the input
       in `tok->str` and we can extract the error line from it.
     * If the input comes from stdin, i.e. the interactive prompt, we
       do not have access to the previous line. That's why a new
       field `tok->stdin_content` is added which holds the whole input for the
       current (multiline) statement or expression. We can then extract the
       error line from `tok->stdin_content` like we do in the string case above.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

1 files changed, 3 insertions, 0 deletions

diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
index 864422390ad..eb70d7b4e49 100644
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -209,6 +209,9 @@ class ExceptionTests(unittest.TestCase):
         check('x = "a', 1, 7)
         check('lambda x: x = 2', 1, 1)
         check('f{a + b + c}', 1, 2)
+        check('[file for str(file) in []\n])', 1, 11)
+        check('[\nfile\nfor str(file)\nin\n[]\n]', 3, 5)
+        check('[file for\n str(file) in []]', 2, 2)
 
         # Errors thrown by compile.c
         check('class foo:return 1', 1, 11)