summaryrefslogtreecommitdiffstatshomepage
path: root/py/parse.c
Commit message (Collapse)AuthorAge
* py/parse: Make parser error handling cleaner, less spaghetti-like.Damien George2015-10-12
|
* py: Move constant folding from compiler to parser.Damien George2015-10-12
| | | | | | | | | | It makes much more sense to do constant folding in the parser while the parse tree is being built. This eliminates the need to create parse nodes that will just be folded away. The code is slightly simpler and a bit smaller as well. Constant folding now has a configuration option, MICROPY_COMP_CONST_FOLDING, which is enabled by default.
* py/parse: Factor logic when creating parse node from and-rule.Damien George2015-10-08
|
* py: Allocate parse nodes in chunks to reduce fragmentation and RAM use.Damien George2015-10-02
| | | | | | | | With this patch parse nodes are allocated sequentially in chunks. This reduces fragmentation of the heap and prevents waste at the end of individually allocated parse nodes. Saves roughly 20% of RAM during parse stage.
* unix-cpy: Remove unix-cpy. It's no longer needed.Damien George2015-08-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | unix-cpy was originally written to get semantic equivalent with CPython without writing functional tests. When writing the initial implementation of uPy it was a long way between lexer and functional tests, so the half-way test was to make sure that the bytecode was correct. The idea was that if the uPy bytecode matched CPython 1-1 then uPy would be proper Python if the bytecodes acted correctly. And having matching bytecode meant that it was less likely to miss some deep subtlety in the Python semantics that would require an architectural change later on. But that is all history and it no longer makes sense to retain the ability to output CPython bytecode, because: 1. It outputs CPython 3.3 compatible bytecode. CPython's bytecode changes from version to version, and seems to have changed quite a bit in 3.5. There's no point in changing the bytecode output to match CPython anymore. 2. uPy and CPy do different optimisations to the bytecode which makes it harder to match. 3. The bytecode tests are not run. They were never part of Travis and are not run locally anymore. 4. The EMIT_CPYTHON option needs a lot of extra source code which adds heaps of noise, especially in compile.c. 5. Now that there is an extensive test suite (which tests functionality) there is no need to match the bytecode. Some very subtle behaviour is tested with the test suite and passing these tests is a much better way to stay Python-language compliant, rather than trying to match CPy bytecode.
* py/parse: Fix handling of empty input so it raises an exception.Damien George2015-07-24
|
* py/parse: De-duplicate and simplify code for parser "or" rule.Damien George2015-07-24
|
* py: Improve allocation policy of qstr data.Damien George2015-07-14
| | | | | | | | | | | | | | | | | | | | Previous to this patch all interned strings lived in their own malloc'd chunk. On average this wastes N/2 bytes per interned string, where N is the number-of-bytes for a quanta of the memory allocator (16 bytes on 32 bit archs). With this patch interned strings are concatenated into the same malloc'd chunk when possible. Such chunks are enlarged inplace when possible, and shrunk to fit when a new chunk is needed. RAM savings with this patch are highly varied, but should always show an improvement (unless only 3 or 4 strings are interned). New version typically uses about 70% of previous memory for the qstr data, and can lead to savings of around 10% of total memory footprint of a running script. Costs about 120 bytes code size on Thumb2 archs (depends on how many calls to gc_realloc are made).
* py: Clean up some bits and pieces in parser, grammar.Damien George2015-04-21
|
* py: Update parse.c&mpconfig.h to reflect rename of mp_lexer_show_token.nhtshot2015-02-23
| | | | | This function is only used when DEBUG_PRINTERS and USE_RULE_NAME are enabled.
* py: Expose compile.c:list_get as mp_parse_node_extract_list.Damien George2015-02-13
|
* py: Initialise variables in mp_parse correctly, to satisfy gcc warning.Damien George2015-02-08
|
* py: Parse big-int/float/imag constants directly in parser.Damien George2015-02-08
| | | | | | | | | Previous to this patch, a big-int, float or imag constant was interned (made into a qstr) and then parsed at runtime to create an object each time it was needed. This is wasteful in RAM and not efficient. Now, these constants are parsed straight away in the parser and turned into objects. This allows constants with large numbers of digits (so addresses issue #1103) and takes us a step closer to #722.
* py: Protect mp_parse and mp_compile with nlr push/pop block.Damien George2015-02-07
| | | | | | | | | | To enable parsing constants more efficiently, mp_parse should be allowed to raise an exception, and mp_compile can already raise a MemoryError. So these functions need to be protected by an nlr push/pop block. This patch adds that feature in all places. This allows to simplify how mp_parse and mp_compile are called: they now raise an exception if they have an error and so explicit checking is not needed anymore.
* py: Be more machine-portable with size of bit fields.Damien George2015-01-24
|
* py, unix, stmhal: Allow to compile with -Wshadow.Damien George2015-01-20
| | | | See issue #699.
* py, unix: Allow to compile with -Wsign-compare.Damien George2015-01-16
| | | | See issue #699.
* py: Add "default" to switches to allow better code flow analysis.Damien George2015-01-14
| | | | | This helps compiler produce smaller code. Saves 124 bytes on stmhal and bare-arm.
* py: Never intern data of large string/bytes object; add relevant tests.Damien George2015-01-13
| | | | | | | | | | Previously to this patch all constant string/bytes objects were interned by the compiler, and this lead to crashes when the qstr was too long (noticeable now that qstr length storage defaults to 1 byte). With this patch, long string/bytes objects are never interned, and are referenced directly as constant objects within generated code using load_const_obj.
* py: Move to guarded includes, everywhere in py/ core.Damien George2015-01-01
| | | | Addresses issue #1022.
* py: Remove unnecessary RULE_none and PN_none from parser.Damien George2014-12-20
|
* py: Add blank and ident flags to grammar rules to simplify parser.Damien George2014-12-20
| | | | This saves around 100 bytes code space on stmhal, more on unix.
* py: Save a few code bytes in parser; make vars local where possible.Damien George2014-12-20
|
* py: Optimise lexer by exposing lexer type.Damien George2014-12-05
| | | | | | | | | mp_lexer_t type is exposed, mp_token_t type is removed, and simple lexer functions (like checking current token kind) are now inlined. This saves 784 bytes ROM on 32-bit unix, 348 bytes on stmhal, and 460 bytes on bare-arm. It also saves a tiny bit of RAM since mp_lexer_t is a bit smaller. Also will run a bit more efficiently.
* py: Properly free string parse-node; add assertion to gc_free.Damien George2014-10-23
|
* py: Free non-interned strings in the parser when not needed.Damien George2014-09-23
| | | | | | | | | | | mp_parse_node_free now frees the memory associated with non-interned strings. And the parser calls mp_parse_node_free when discarding a non-used node (such as a doc string). Also, the compiler now frees the parse tree explicitly just before it exits (as opposed to relying on the caller to do this). Addresses issue #708 as best we can.
* py: Allow viper to have type annotations.Damien George2014-08-15
| | | | | | | | | Viper functions can now be annotated with the type of their arguments and return value. Eg: @micropython.viper def f(x:int) -> int: return x + 1
* parser: Convert (u)int to mp_(u)int_t.Damien George2014-07-03
|
* Rename machine_(u)int_t to mp_(u)int_t.Damien George2014-07-03
| | | | See discussion in issue #50.
* py: Include mpconfig.h before all other includes.Paul Sokolovsky2014-06-21
| | | | | | It defines types used by all other headers. Fixes #691.
* py: Fix check of small-int overflow when parsing ints.Damien George2014-05-28
| | | | Also unifies use of SMALL_INT_FITS macro across parser and runtime.
* Change const byte* to const char* where sensible.Damien George2014-05-25
| | | | | This removes need for some casts (at least, more than it adds need for new casts!).
* py: Don't automatically intern strings in parser.Damien George2014-05-25
| | | | | | | | This completes non-automatic interning of strings in the parser, so that doc strings don't take up RAM. It complicates the parser and compiler, and bloats stmhal by about 300 bytes. It's complicated because now there are 2 kinds of parse-nodes that can be strings: interned leaves and non-interned structs.
* Merge branch 'keep-strings-uninterned' of github.com:pfalcon/micropython ↵Damien George2014-05-25
|\ | | | | | | | | | | | | into pfalcon-keep-strings-uninterned Conflicts: py/parse.c
| * Draft approach towards resolving ↵Paul Sokolovsky2014-05-08
| | | | | | | | https://github.com/micropython/micropython/issues/560#issuecomment-42213955
* | Tidy up some configuration options.Damien George2014-05-21
| | | | | | | | | | | | | | | | | | | | MP_ALLOC_* -> MICROPY_ALLOC_* MICROPY_PATH_MAX -> MICROPY_ALLOC_PATH_MAX MICROPY_ENABLE_REPL_HELPERS -> MICROPY_HELPER_REPL MICROPY_ENABLE_LEXER_UNIX -> MICROPY_HELPER_LEXER_UNIX MICROPY_EXTRA_* -> MICROPY_PORT_* See issue #35.
* | py: Improve handling of memory error in parser.Damien George2014-05-10
|/ | | | | | | | | | | Parser shouldn't raise exceptions, so needs to check when memory allocation fails. This patch does that for the initial set up of the parser state. Also, we now put the parser object on the stack. It's small enough to go there instead of on the heap. This partially addresses issue #558.
* py, parser: Add commented-out code to discard doc strings.Damien George2014-05-06
| | | | | | Doesn't help with RAM reduction because doc strings are interned as soon as they are encountered, which is too soon to do any optimisations on them.
* py: Turn down amount of RAM parser and compiler use.Damien George2014-05-05
| | | | | | | | | | | | | | | | | | | | | | | | There are 2 locations in parser, and 1 in compiler, where memory allocation is not precise. In the parser it's the rule stack and result stack, in the compiler it's the array for the identifiers in the current scope. All other mallocs are exact (ie they don't allocate more than is needed). This patch adds tuning options (MP_ALLOC_*) to mpconfig.h for these 3 inexact allocations. The inexact allocations in the parser should actually be close to logarithmic: you need an exponentially larger script (absent pathological cases) to use up more room on the rule and result stacks. As such, the default allocation policy for these is now to start with a modest sized stack, but grow only in small increments. For the identifier arrays in the compiler, these now start out quite small (4 entries, since most functions don't have that many ids), and grow incrementally by 6 (since if you have more ids than 4, you probably have quite a few more, but it wouldn't be exponentially more). Partially addresses issue #560.
* Add license header to (almost) all files.Damien George2014-05-03
| | | | | | | Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
* py: Check explicitly for memory allocation failure in parser.Damien George2014-04-10
| | | | | | Previously, a failed malloc/realloc would throw an exception, which was not caught. I think it's better to keep the parser free from NLR (exception throwing), hence this patch.
* py: Clean up includes.xbe2014-03-17
| | | | Remove unnecessary includes. Add includes that improve portability.
* py: Implement bit-shift and not operations for mpz.Damien George2014-03-01
| | | | | | Implement not, shl and shr in mpz library. Add function to create mpzs on the stack, used for memory efficiency when rhs is a small int. Factor out code to parse base-prefix of number into a dedicated function.
* parse: Refactor parse node encoding to support full range of small ints.Paul Sokolovsky2014-02-22
| | | | | Based on suggestion by @dpgeorge at https://github.com/micropython/micropython/pull/313
* parse: Note that fact that parser's small ints are different than VM small int.Paul Sokolovsky2014-02-21
| | | | | | Specifically, VM's small ints are 31 bit, while parser's only 28. There's already MP_OBJ_FITS_SMALL_INT(), so, for clarity, rename MP_FIT_SMALL_INT() to MP_PARSE_FITS_SMALL_INT().
* Implement proper exception type hierarchy.Damien George2014-02-15
| | | | | | | | | | | | | | Each built-in exception is now a type, with base type BaseException. C exceptions are created by passing a pointer to the exception type to make an instance of. When raising an exception from the VM, an instance is created automatically if an exception type is raised (as opposed to an exception instance). Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper. Handling of parse error changed to match new exceptions. mp_const_type renamed to mp_type_type for consistency.
* Replace global "static" -> "STATIC", to allow "analysis builds". Part 2.Paul Sokolovsky2014-02-12
|
* py: Fix bug with LOAD_METHOD; fix int->machine_int_t for small int.Damien George2014-01-29
| | | | | | | | LOAD_METHOD bug was: emitbc did not correctly calculate the amount of stack usage for a LOAD_METHOD operation. small int bug was: int was being used to pass small ints, when it should have been machine_int_t.
* Implement mp_parse_node_free; print properly repr(string).Damien George2014-01-25
|
* Add parse_node_free_struct() and use it to free parse tree after compilation.Paul Sokolovsky2014-01-25
| | | | TODO: Check lexer/parse/compile error path for leaks too.
generated by cgit v1.2.3 (git 2.39.1) at 2025-06-04 00:03:03 +0000