summaryrefslogtreecommitdiffstatshomepage
path: root/py/emitglue.c
Commit message (Collapse)AuthorAge
* py/emitglue: Fix clear cache builtin warning on Clang for AArch32.Alessandro Gatti2024-12-10
| | | | | | | | | This commit fixes a warning occurring on Clang when calling `__builtin___clear_cache` with non-void pointers for its start and end memory area locations. The code now uses a char pointer for the end location, and it still builds without warnings on GCC. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
* py/asmrv32: Add RISC-V RV32IMC native code emitter.Alessandro Gatti2024-06-21
| | | | | | | This adds a native code generation backend for RISC-V RV32I CPUs, currently limited to the I, M, and C instruction sets. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
* py/emitglue: Make mp_emit_glue_assign_native's fun_data arg a const ptr.Damien George2024-02-22
| | | | | | | It will only ever be read from, and in some cases (eg on esp8266) can actually be in ROM. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Remove n_pos_args from DEBUG_printf.Damien George2024-02-20
| | | | | | This argument was renamed in 39bf055d23be4b0f761af115773c3db1074fc2dd. Signed-off-by: Damien George <damien@micropython.org>
* py/objfun: Split viper fun type out to separate mp_type_fun_viper type.Damien George2024-02-20
| | | | | | | | Viper functions are quite different to native functions and benefit from being a separate type. For example, viper functions don't have a bytecode- style prelude, and don't support generators or default arguments. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Include fun_data_len in mp_raw_code_t only when saving.Damien George2024-02-16
| | | | | | | Reduces the size of mp_raw_code_t in the case when MICROPY_DEBUG_PRINTERS is enabled. Signed-off-by: Damien George <damien@micropython.org>
* tools/mpy-tool.py: Skip generating frozen mp_raw_code_t when possible.Damien George2024-02-16
| | | | | | | This reduces frozen code size by using the bytecode directly as the `mp_proto_fun_t`. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Introduce mp_proto_fun_t as a more general mp_raw_code_t.Damien George2024-02-16
| | | | | | | | | | Allows bytecode itself to be used instead of an mp_raw_code_t in the simple and common cases of a bytecode function without any children. This can be used to further reduce frozen code size, and has the potential to optimise other areas like importing. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Simplify mp_raw_code_t's kind and scope_flags members.Damien George2024-02-16
| | | | | | | | | To simplify their access and reduce code size. The `scope_flags` member is only ever used to determine if a function is a generator or not, so make it reflect that fact as a bool type. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Reorder and resize members of mp_raw_code_t.Damien George2024-02-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The mp_raw_code_t struct has been reordered and some members resized. The `n_pos_args` member is renamed to `asm_n_pos_args`, and `type_sig` renamed to `asm_type_sig` to indicate that these are used only for the inline-asm emitters. These two members are also grouped together in the struct. The justifications for resizing the members are: - `fun_data_len` can be 32-bits without issue - `n_children` is already limited to 16-bits by `mp_emit_common_t::ct_cur_child` - `scope_flags` is already limited to 16-bits by `scope_t::scope_flags` - `prelude_offset` is already limited to 16-bits by the argument to `mp_emit_glue_assign_native()` - it's reasonable to limit the maximim number of inline-asm arguments to 12 (24 bits for `asm_type_sig` divided by 2) This change helps to reduce frozen code size (and in some cases RAM usage) in the following cases: - 64-bit targets - builds with MICROPY_PY_SYS_SETTRACE enabled - builds with MICROPY_EMIT_MACHINE_CODE enabled but MICROPY_EMIT_INLINE_ASM disabled With this change, unix 64-bit builds are -4080 bytes in size. Bare-metal ports like rp2 are unchanged (because mp_raw_code_t is still 32 bytes on those 32-bit targets). Signed-off-by: Damien George <damien@micropython.org>
* py: Change ifdef DEBUG_PRINT to if DEBUG_PRINT.Ihor Nehrutsa2023-09-29
| | | | Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
* py/emitnative: Access qstr values using indirection table qstr_table.Damien George2022-05-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the native emitter to access qstr values using the qstr indirection table qstr_table, but only when generating native code that will be saved to a .mpy file. This makes the resulting native code fully static, ie it does not require any fix-ups or rewriting when it is imported. The performance of native code is more or less unchanged. Benchmark results on PYBv1.0 (using --via-mpy and --emit native) are: N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 407.16 -> 411.85 : +4.69 = +1.152% (+/-0.01%) bm_fannkuch.py 100.89 -> 101.20 : +0.31 = +0.307% (+/-0.01%) bm_fft.py 3521.17 -> 3441.72 : -79.45 = -2.256% (+/-0.00%) bm_float.py 6707.29 -> 6644.83 : -62.46 = -0.931% (+/-0.00%) bm_hexiom.py 55.91 -> 55.41 : -0.50 = -0.894% (+/-0.00%) bm_nqueens.py 5343.54 -> 5326.17 : -17.37 = -0.325% (+/-0.00%) bm_pidigits.py 603.89 -> 632.79 : +28.90 = +4.786% (+/-0.33%) core_qstr.py 64.18 -> 64.09 : -0.09 = -0.140% (+/-0.01%) core_yield_from.py 313.61 -> 311.11 : -2.50 = -0.797% (+/-0.03%) misc_aes.py 654.29 -> 659.75 : +5.46 = +0.834% (+/-0.02%) misc_mandel.py 4205.10 -> 4272.08 : +66.98 = +1.593% (+/-0.01%) misc_pystone.py 3077.79 -> 3128.39 : +50.60 = +1.644% (+/-0.01%) misc_raytrace.py 388.45 -> 393.71 : +5.26 = +1.354% (+/-0.01%) viper_call0.py 576.83 -> 566.76 : -10.07 = -1.746% (+/-0.05%) viper_call1a.py 550.39 -> 540.12 : -10.27 = -1.866% (+/-0.11%) viper_call1b.py 438.32 -> 432.09 : -6.23 = -1.421% (+/-0.11%) viper_call1c.py 442.96 -> 436.11 : -6.85 = -1.546% (+/-0.08%) viper_call2a.py 536.31 -> 527.37 : -8.94 = -1.667% (+/-0.04%) viper_call2b.py 378.99 -> 377.50 : -1.49 = -0.393% (+/-0.08%) Signed-off-by: Damien George <damien@micropython.org>
* py: Rework bytecode and .mpy file format to be mostly static data.Damien George2022-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
* py/emitglue: Always flush caches when assigning native ARM code.Damien George2021-06-05
| | | | | | | | | | | | | | | Prior to this commit, cache flushing for ARM native code was done only in the assembler code asm_thumb_end_pass()/asm_arm_end_pass(), at the last pass of the assembler. But this misses flushing the cache when loading native code from an .mpy file, ie in persistentcode.c. The change here makes sure the cache is always flushed/cleaned/invalidated when assigning native code on ARM architectures. This problem was found running tests/micropython/import_mpy_native_gc.py on the mimxrt port. Signed-off-by: Damien George <damien@micropython.org>
* py/showbc: Pass in an mp_print_t struct to all bytecode-print functions.Damien George2020-09-11
| | | | | | So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
* all: Reformat C and Python source code with tools/codeformat.py.Damien George2020-02-28
| | | | This is run with uncrustify 0.70.1, and black 19.10b0.
* py: Integrate sys.settrace feature into the VM and runtime.Milan Rossa2019-08-30
| | | | | | | This commit adds support for sys.settrace, allowing to install Python handlers to trace execution of Python code. The interface follows CPython as closely as possible. The feature is disabled by default and can be enabled via MICROPY_PY_SYS_SETTRACE.
* py: Define EMIT_MACHINE_CODE as EMIT_NATIVE || EMIT_INLINE_ASM.Jun Wu2019-06-28
| | | | | The combination MICROPY_EMIT_NATIVE || MICROPY_EMIT_INLINE_ASM is used in many places, so define a new macro for it.
* py: Add support to save native, viper and asm code to .mpy files.Damien George2019-03-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds support for saving and loading .mpy files that contain native code (native, viper and inline-asm). A lot of the ground work was already done for this in the form of removing pointers from generated native code. The changes here are mainly to link in qstr values to the native code, and change the format of .mpy files to contain native code blocks (possibly mixed with bytecode). A top-level summary: - @micropython.native, @micropython.viper and @micropython.asm_thumb/ asm_xtensa are now allowed in .py files when compiling to .mpy, and they work transparently to the user. - Entire .py files can be compiled to native via mpy-cross -X emit=native and for the most part the generated .mpy files should work the same as their bytecode version. - The .mpy file format is changed to 1) specify in the header if the file contains native code and if so the architecture (eg x86, ARMV7M, Xtensa); 2) for each function block the kind of code is specified (bytecode, native, viper, asm). - When native code is loaded from a .mpy file the native code must be modified (in place) to link qstr values in, just like bytecode (see py/persistentcode.c:arch_link_qstr() function). In addition, this now defines a public, native ABI for dynamically loadable native code generated by other languages, like C.
* py/emitglue: Remove union in mp_raw_code_t to combine bytecode & native.Damien George2019-03-08
|
* py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.Damien George2019-02-12
| | | | | | | | | | | | | | | | | | | | | These macros could in principle be (inline) functions so it makes sense to have them lower case, to match the other C API functions. The remaining macros that are upper case are: - MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR - MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE - MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE - MP_OBJ_FUN_MAKE_SIG - MP_DECLARE_CONST_xxx - MP_DEFINE_CONST_xxx These must remain macros because they are used when defining const data (at least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have MP_OBJ_SMALL_INT_VALUE also a macro). For those macros that have been made lower case, compatibility macros are provided for the old names so that users do not need to change their code immediately.
* py/emitnative: Implement yield and yield-from in native emitter.Damien George2018-10-01
| | | | | | | | | | | This commit adds first class support for yield and yield-from in the native emitter, including send and throw support, and yields enclosed in exception handlers (which requires pulling down the NLR stack before yielding, then rebuilding it when resuming). This has been fully tested and is working on unix x86 and x86-64, and stm32. Also basic tests have been done with the esp8266 port. Performance of existing native code is unchanged.
* py: Make viper functions have the same entry signature as native.Damien George2018-09-15
| | | | | | | | | | | | | | | This commit makes viper functions have the same signature as native functions, at the level of the emitter/assembler. This means that viper functions can now be wrapped in the same uPy object as native functions. Viper functions are now responsible for parsing their arguments (before it was done by the runtime), and this makes calling them more efficient (in most cases) because the viper entry code can be custom generated to suit the signature of the function. This change also opens the way forward for viper functions to take arbitrary numbers of arguments, and for them to handle globals correctly, among other things.
* py: Fix compiling with debug enabled and make more use of DEBUG_printf.Damien George2018-08-02
| | | | | | | DEBUG_printf and MICROPY_DEBUG_PRINTER is now used instead of normal printf, and a fault is fixed in mp_obj_class_lookup with debugging enabled; see issue #3999. Debugging can now be enabled on all ports including when nan-boxing is used.
* py/objgenerator: Eliminate need for mp_obj_gen_wrap wrapper instances.Damien George2018-07-02
| | | | | | | For generating functions there is no need to wrap the bytecode function in a generator wrapper instance. Instead the type of the bytecode function can be changed to mp_type_gen_wrap. This reduces code size and saves a block of GC heap RAM for each generator.
* py/emitglue: When assigning bytecode only pass bytecode len if needed.Damien George2018-02-14
| | | | | Most embedded targets will have this bit of the code disabled, saving a small amount of code space.
* py: Add verbose debug compile-time flag MICROPY_DEBUG_VERBOSE.Stefan Naumann2017-08-15
| | | | It enables all the DEBUG_printf outputs in the py/ source code.
* all: Use the name MicroPython consistently in commentsAlexander Steffen2017-07-31
| | | | | There were several different spellings of MicroPython present in comments, when there should be only one.
* py/emitglue: Refactor to remove assert(0), to improve coverage.Damien George2016-12-21
|
* py: Allow inline-assembler emitter to be generic.Damien George2016-12-09
| | | | | This patch refactors some code so that it is easier to integrate new inline assemblers for different architectures other than ARM Thumb.
* py: Factor persistent code load/save funcs into persistentcode.[ch].Damien George2016-11-16
|
* esp8266: Enable importing of precompiled .mpy files.Damien George2016-10-12
|
* py/emitglue.c: provide mp_raw_code_load_file for any unix architectureChris Packham2016-09-04
| | | | Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
* py/emitglue: Use more compact mp_raise_ValueError function.Damien George2016-08-26
|
* py/emitglue: Fix build on AArch64 (ARMv8, etc.) related to loading .mpy files.Paul Sokolovsky2016-05-12
| | | | Actual loading of .mpy files isn't tested.
* py: Add ability to have frozen persistent bytecode from .mpy files.Damien George2016-04-13
| | | | | | | The config variable MICROPY_MODULE_FROZEN is now made of two separate parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY. This allows to have none, either or both of frozen strings and frozen mpy files (aka frozen bytecode).
* py/emitglue: Make mp_raw_code_t* arguments constant pointers.Damien George2016-04-13
|
* py/emitglue: Move typedef of mp_raw_code_t from .c to .h file.Damien George2016-04-13
| | | | It's needed by frozen bytecode.
* py/emitglue: Get persistent bytecode working on Linux ARM platform.Markus Fix2016-03-07
|
* py: Add MICROPY_DYNAMIC_COMPILER option to config compiler at runtime.Damien George2016-02-25
| | | | | | | | | | | | | | This new compile-time option allows to make the bytecode compiler configurable at runtime by setting the fields in the mp_dynamic_compiler structure. By using this feature, the compiler can generate bytecode that targets any MicroPython runtime/VM, regardless of the host and target compile-time settings. Options so far that fall under this dynamic setting are: - maximum number of bits that a small int can hold; - whether caching of lookups is used in the bytecode; - whether to use unicode strings or not (lexer behaviour differs, and therefore generated string constants differ).
* py/inlineasm: Add ability to specify return type of asm_thumb funcs.Damien George2016-01-27
| | | | | | | | | | Supported return types are: object, bool, int, uint. For example: @micropython.asm_thumb def foo(r0, r1) -> uint: add(r0, r0, r1)
* py/emitglue: Use mp_obj_is_float instead of MP_OBJ_IS_TYPE.Damien George2016-01-13
|
* py/emitglue: Add more feature flags to .mpy persistent bytecode output.Damien George2016-01-08
| | | | | Need to record in .mpy file whether unicode is enabled, and how many bits are in a small int.
* py/emitglue: Add include of unistd.h for read function.Damien George2015-12-18
|
* py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.Damien George2015-11-29
| | | | | | | | | This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
* py: Change qstr_* functions to use size_t as the type for str len arg.Damien George2015-11-29
|
* py: Change mp_print_strn_t func type to use size_t for the str length.Damien George2015-11-29
|
* py/emitglue: Implement persistent saving and loading of const objects.Damien George2015-11-23
|
* py/emitglue: Add feature-flag header to .mpy to detect bytecode compat.Damien George2015-11-23
| | | | | Loading .mpy files will now check to make sure that the target VM can support the bytecode.
* py/emitglue: Host definition of mp_verbose_flag.Paul Sokolovsky2015-11-21
| | | | | This may not seem like the ideal place, but is actually the only place in py/ where it gets referenced, so is just right.
generated by cgit v1.2.3 (git 2.39.1) at 2025-05-05 01:56:59 +0000