bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993)

There are two errors that this commit fixes:

* The parser was not correctly computing the offset and the string
  source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
  exception happened in rules involving optionals ('?', [...]) as we
  always make them return valid results by using the comma operator. We
  need to check first if we don't have an error before continuing.

1 files changed, 4 insertions, 0 deletions

diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
index 02489a2bd92..0f8a8f134b6 100644
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -209,6 +209,10 @@ class ExceptionTests(unittest.TestCase):
                     src = src.decode(encoding, 'replace')
                 line = src.split('\n')[lineno-1]
                 self.assertIn(line, cm.exception.text)
+    
+    def test_error_offset_continuation_characters(self):
+        check = self.check
+        check('"\\\n"(1 for c in I,\\\n\\', 2, 2)
 
     def testSyntaxErrorOffset(self):
         check = self.check