py/lexer: Allow conversion specifiers in f-strings (e.g. !r).

PEP-498 allows for conversion specifiers like !r and !s to convert the
expression declared in braces to be passed through repr() and str()
respectively.

This updates the logic that detects the end of the expression to also stop
when it sees "![rs]" that is either at the end of the f-string or before
the ":" indicating the start of the format specifier. The "![rs]" is now
retained in the format string, whereas previously it stayed on the end
of the expression leading to a syntax error.

Previously: `f"{x!y:z}"` --> `"{:z}".format(x!y)`
Now: `f"{x!y:z}"` --> `"{!y:z}".format(x)`

Note that "!a" is not supported by `str.format` as MicroPython has no
`ascii()`, but now this will raise the correct error.

Updated cpydiff and added tests.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

1 files changed, 4 insertions, 14 deletions

diff --git a/tests/cpydiff/core_fstring_repr.py b/tests/cpydiff/core_fstring_repr.py
index df80abf795..d37fb48db7 100644
--- a/tests/cpydiff/core_fstring_repr.py
+++ b/tests/cpydiff/core_fstring_repr.py
@@ -1,18 +1,8 @@
 """
 categories: Core
-description: f-strings don't support the !r, !s, and !a conversions
-cause: MicroPython is optimised for code space.
-workaround: Use repr(), str(), and ascii() explicitly.
+description: f-strings don't support !a conversions
+cause: MicropPython does not implement ascii()
+workaround: None
 """
 
-
-class X:
-    def __repr__(self):
-        return "repr"
-
-    def __str__(self):
-        return "str"
-
-
-print(f"{X()!r}")
-print(f"{X()!s}")
+f"{'unicode text'!a}"