summaryrefslogtreecommitdiffstatshomepage
path: root/py/lexer.c
Commit message (Collapse)AuthorAge
* extmod/vfs_fat: Remove MICROPY_READER_FATFS component.Damien George2017-01-30
|
* extmod: Add generic VFS sub-system.Damien George2017-01-27
| | | | | | | | | | | | This provides mp_vfs_XXX functions (eg mount, open, listdir) which are agnostic to the underlying filesystem type, and just require an object with the relevant filesystem-like methods (eg .mount, .open, .listidr) which can then be mounted. These mp_vfs_XXX functions would typically be used by a port to implement the "uos" module, and mp_vfs_open would be the builtin open function. This feature is controlled by MICROPY_VFS, disabled by default.
* py/lexer: Permanently disable the mp_lexer_show_token function.Damien George2016-12-22
| | | | | The lexer is very mature and this debug function is no longer used. If it's really needed one can uncomment it and recompile.
* py/lexer: Remove unnecessary check for EOF in lexer's next_char func.Damien George2016-12-22
| | | | | | | | This check always fails (ie chr0 is never EOF) because the callers of this function never call it past the end of the input stream. And even if they did it would be harmless because 1) reader.readbyte must continue to return an EOF char if the stream is exhausted; 2) next_char would just count the subsequent EOF's as characters worth 1 column.
* py/lexer: Remove unreachable code in string tokeniser.Damien George2016-12-22
|
* tests/basics/lexer: Add a test for newline-escaping within a string.Damien George2016-12-22
|
* py/lexer: Make lexer use an mp_reader as its source.Damien George2016-11-16
|
* py/lexer: Rewrite mp_lexer_new_from_fd in terms of mp_reader.Damien George2016-11-16
|
* py/lexer: Provide generic mp_lexer_new_from_file based on mp_reader.Damien George2016-11-16
| | | | | | If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then lexer.c now provides an implementation of mp_lexer_new_from_file using the mp_reader_new_file function.
* py/lexer: Rewrite mp_lexer_new_from_str_len in terms of mp_reader_mem.Damien George2016-11-16
|
* py/lexer: Remove unnecessary code, and unreachable code.Damien George2016-10-12
| | | | | | | | | Setting emit_dent=0 is unnecessary because arriving in that part of the if-logic will guarantee that emit_dent is already zero. The block to check indent_top(lex)>0 is unreachable because a newline is always inserted an the end of the input stream, and hence dedents are always processed before EOF.
* py/vstr: Remove vstr.had_error flag and inline basic vstr functions.Damien George2016-09-19
| | | | | | | | | | | | | | | | | | The vstr.had_error flag was a relic from the very early days which assumed that the malloc functions (eg m_new, m_renew) returned NULL if they failed to allocate. But that's no longer the case: these functions will raise an exception if they fail. Since it was impossible for had_error to be set, this patch introduces no change in behaviour. An alternative option would be to change the malloc calls to the _maybe variants, which return NULL instead of raising, but then a lot of code will need to explicitly check if the vstr had an error and raise if it did. The code-size savings for this patch are, in bytes: bare-arm:188, minimal:456, unix(NDEBUG,x86-64):368, stmhal:228, esp8266:360.
* py: Declare constant data as properly constant.Damien George2016-05-20
| | | | | Otherwise some compilers (eg without optimisation) will put this read-only data in RAM instead of ROM.
* py: add async/await/async for/async with syntaxpohmelie2016-04-13
| | | | | | | | They are sugar for marking function as generator, "yield from" and pep492 python "semantically equivalents" respectively. @dpgeorge was the original author of this patch, but @pohmelie made changes to implement `async for` and `async with`.
* py: Add MICROPY_DYNAMIC_COMPILER option to config compiler at runtime.Damien George2016-02-25
| | | | | | | | | | | | | | This new compile-time option allows to make the bytecode compiler configurable at runtime by setting the fields in the mp_dynamic_compiler structure. By using this feature, the compiler can generate bytecode that targets any MicroPython runtime/VM, regardless of the host and target compile-time settings. Options so far that fall under this dynamic setting are: - maximum number of bits that a small int can hold; - whether caching of lookups is used in the bytecode; - whether to use unicode strings or not (lexer behaviour differs, and therefore generated string constants differ).
* py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts.Damien George2015-12-18
| | | | | | | | | | | | MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler, which is useful when only loading of pre-compiled bytecode is supported. It is enabled by default. MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin functions. By default they are only included if MICROPY_ENABLE_COMPILER is enabled. Disabling both options saves about 40k of code size on 32-bit x86.
* py/lexer: Properly classify floats that look like hex numbers.Damien George2015-09-07
| | | | Eg 0e0 almost looks like a hex number but in fact is a float.
* py/lexer: Raise SyntaxError when unicode char point out of range.Damien George2015-09-07
|
* py/lexer: Raise NotImplError for unicode name escape, instead of assert.Damien George2015-09-07
|
* py/lexer: Raise SyntaxError when str hex escape sequence is malformed.Damien George2015-07-23
| | | | Addresses issue #1390.
* py: Cast argument for printf to int, to be compatible with more ports.Damien George2015-06-22
| | | | This allows stmhal to be compiled with MICROPY_DEBUG_PRINTERS.
* py: Support unicode (utf-8 encoded) identifiers in Python source.Damien George2015-06-09
| | | | Enabled simply by making the identifier lexing code 8-bit clean.
* extmod: Add ubinascii.unhexlifyDave Hylands2015-05-20
| | | | This also pulls out hex_digit from py/lexer.c and makes unichar_hex_digit
* py: Allow to compile with extra warnings (sign-compare, unused-param).Damien George2015-03-19
|
* py: Parse big-int/float/imag constants directly in parser.Damien George2015-02-08
| | | | | | | | | Previous to this patch, a big-int, float or imag constant was interned (made into a qstr) and then parsed at runtime to create an object each time it was needed. This is wasteful in RAM and not efficient. Now, these constants are parsed straight away in the parser and turned into objects. This allows constants with large numbers of digits (so addresses issue #1103) and takes us a step closer to #722.
* py: Convert CR to LF and CR LF to LF in lexer.Damien George2015-01-30
| | | | | Only noticeable difference is how newlines are encoded in triple-quoted strings. The behaviour now matches CPython3.
* py: Be more precise about unicode type and disabled unicode behaviour.Damien George2015-01-28
|
* py, unix: Allow to compile with -Wsign-compare.Damien George2015-01-16
| | | | See issue #699.
* py: Put all global state together in state structures.Damien George2015-01-07
| | | | | | This patch consolidates all global variables in py/ core into one place, in a global structure. Root pointers are all located together to make GC tracing easier and more efficient.
* py: Move to guarded includes, everywhere in py/ core.Damien George2015-01-01
| | | | Addresses issue #1022.
* py: Fix printing of size_t entity; fix qemu-arm for changes to lexer.Damien George2014-12-05
|
* py: Optimise lexer by exposing lexer type.Damien George2014-12-05
| | | | | | | | | mp_lexer_t type is exposed, mp_token_t type is removed, and simple lexer functions (like checking current token kind) are now inlined. This saves 784 bytes ROM on 32-bit unix, 348 bytes on stmhal, and 460 bytes on bare-arm. It also saves a tiny bit of RAM since mp_lexer_t is a bit smaller. Also will run a bit more efficiently.
* py: Add further checks for failed malloc in lexer init functions.Damien George2014-10-09
|
* py: Change lexer stream API to return bytes not chars.Damien George2014-07-30
| | | | Lexer is now 8-bit clean inside strings.
* lexer: Convert type (u)int to mp_(u)int_t.Damien George2014-07-03
|
* lexer, vstr: Add unicode support.Chris Angelico2014-06-27
|
* py: Include mpconfig.h before all other includes.Paul Sokolovsky2014-06-21
| | | | | | It defines types used by all other headers. Fixes #691.
* Prefix ARRAY_SIZE with micropython prefix MP_Emmanuel Blot2014-06-19
|
* unicode: String API is const byte*.Paul Sokolovsky2014-06-14
| | | | | We still have that char vs byte dichotomy, but majority of string operations now use byte.
* py: Instead of having "debug on" var, have "optimization level" var.Paul Sokolovsky2014-06-03
| | | | | This allows to have multiple "optimization" levels (CPython has two (-OO removes docstrings), we can have more).
* lexer: Add another comment for somewhat obscure way __debug__ is handled.Paul Sokolovsky2014-06-02
|
* Tidy up some configuration options.Damien George2014-05-21
| | | | | | | | | | MP_ALLOC_* -> MICROPY_ALLOC_* MICROPY_PATH_MAX -> MICROPY_ALLOC_PATH_MAX MICROPY_ENABLE_REPL_HELPERS -> MICROPY_HELPER_REPL MICROPY_ENABLE_LEXER_UNIX -> MICROPY_HELPER_LEXER_UNIX MICROPY_EXTRA_* -> MICROPY_PORT_* See issue #35.
* py: Add support for __debug__ constant.Damien George2014-05-12
| | | | | | | | | __debug__ now resolves to True or False. Its value needs to be set by mp_set_debug(). TODO: call mp_set_debug in unix/ port. TODO: optimise away "if False:" statements in compiler.
* py, lexer: Add allocation policy config; return NULL if can't allocate.Damien George2014-05-10
|
* Add license header to (almost) all files.Damien George2014-05-03
| | | | | | | Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
* py: Fix lexer so it doesn't allow ! and ..Damien George2014-04-10
|
* py, lexer: Fix parsing of raw strings (allow escaping of quote).Damien George2014-04-10
|
* py: Clean up includes.xbe2014-03-17
| | | | Remove unnecessary includes. Add includes that improve portability.
* Add vstr_ins and vstr_cut_out; improve stmhal readline.Damien George2014-03-15
|
* Implement proper exception type hierarchy.Damien George2014-02-15
| | | | | | | | | | | | | | Each built-in exception is now a type, with base type BaseException. C exceptions are created by passing a pointer to the exception type to make an instance of. When raising an exception from the VM, an instance is created automatically if an exception type is raised (as opposed to an exception instance). Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper. Handling of parse error changed to match new exceptions. mp_const_type renamed to mp_type_type for consistency.
generated by cgit v1.2.3 (git 2.39.1) at 2025-06-03 03:22:09 +0000