summaryrefslogtreecommitdiffstatshomepage
path: root/py
diff options
context:
space:
mode:
authorDamien George <damien@micropython.org>2021-10-22 22:22:47 +1100
committerDamien George <damien@micropython.org>2022-02-24 18:08:43 +1100
commitf2040bfc7ee033e48acef9f289790f3b4e6b74e5 (patch)
tree53402caf1b0e7321bf772278a94f5a87a9e7bf0d /py
parent64bfaae7ab33e628f28ca3b53b10893fb047b48e (diff)
downloadmicropython-f2040bfc7ee033e48acef9f289790f3b4e6b74e5.tar.gz
micropython-f2040bfc7ee033e48acef9f289790f3b4e6b74e5.zip
py: Rework bytecode and .mpy file format to be mostly static data.
Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
Diffstat (limited to 'py')
-rw-r--r--py/asmbase.c3
-rw-r--r--py/asmbase.h2
-rw-r--r--py/bc.c69
-rw-r--r--py/bc.h84
-rw-r--r--py/builtinevex.c2
-rw-r--r--py/builtinimport.c28
-rw-r--r--py/compile.c171
-rw-r--r--py/compile.h3
-rw-r--r--py/dynruntime.h10
-rw-r--r--py/emit.h50
-rw-r--r--py/emitbc.c156
-rw-r--r--py/emitcommon.c11
-rw-r--r--py/emitglue.c61
-rw-r--r--py/emitglue.h26
-rw-r--r--py/emitnative.c265
-rw-r--r--py/emitnx86.c2
-rw-r--r--py/frozenmod.c2
-rw-r--r--py/mpconfig.h8
-rw-r--r--py/nativeglue.c6
-rw-r--r--py/nativeglue.h12
-rw-r--r--py/obj.h4
-rw-r--r--py/objfun.c50
-rw-r--r--py/objfun.h10
-rw-r--r--py/objgenerator.c18
-rw-r--r--py/objmodule.c9
-rw-r--r--py/persistentcode.c554
-rw-r--r--py/persistentcode.h12
-rw-r--r--py/profile.c47
-rw-r--r--py/profile.h4
-rw-r--r--py/qstr.h1
-rw-r--r--py/scope.c3
-rw-r--r--py/scope.h3
-rw-r--r--py/showbc.c73
-rw-r--r--py/vm.c73
34 files changed, 900 insertions, 932 deletions
diff --git a/py/asmbase.c b/py/asmbase.c
index 344e03e7a7..4a3fd089cb 100644
--- a/py/asmbase.c
+++ b/py/asmbase.c
@@ -61,7 +61,8 @@ void mp_asm_base_start_pass(mp_asm_base_t *as, int pass) {
// all functions must go through this one to emit bytes
// if as->pass < MP_ASM_PASS_EMIT, then this function just counts the number
// of bytes needed and returns NULL, and callers should not store any data
-uint8_t *mp_asm_base_get_cur_to_write_bytes(mp_asm_base_t *as, size_t num_bytes_to_write) {
+uint8_t *mp_asm_base_get_cur_to_write_bytes(void *as_in, size_t num_bytes_to_write) {
+ mp_asm_base_t *as = as_in;
uint8_t *c = NULL;
if (as->pass == MP_ASM_PASS_EMIT) {
assert(as->code_offset + num_bytes_to_write <= as->code_size);
diff --git a/py/asmbase.h b/py/asmbase.h
index 24c3af8679..960be7685f 100644
--- a/py/asmbase.h
+++ b/py/asmbase.h
@@ -45,7 +45,7 @@ typedef struct _mp_asm_base_t {
void mp_asm_base_init(mp_asm_base_t *as, size_t max_num_labels);
void mp_asm_base_deinit(mp_asm_base_t *as, bool free_code);
void mp_asm_base_start_pass(mp_asm_base_t *as, int pass);
-uint8_t *mp_asm_base_get_cur_to_write_bytes(mp_asm_base_t *as, size_t num_bytes_to_write);
+uint8_t *mp_asm_base_get_cur_to_write_bytes(void *as, size_t num_bytes_to_write);
void mp_asm_base_label_assign(mp_asm_base_t *as, size_t label);
void mp_asm_base_align(mp_asm_base_t *as, unsigned int align);
void mp_asm_base_data(mp_asm_base_t *as, unsigned int bytesize, uintptr_t val);
diff --git a/py/bc.c b/py/bc.c
index 69b6739e8f..b98df39e28 100644
--- a/py/bc.c
+++ b/py/bc.c
@@ -29,9 +29,9 @@
#include <string.h>
#include <assert.h>
-#include "py/runtime.h"
#include "py/bc0.h"
#include "py/bc.h"
+#include "py/objfun.h"
#if MICROPY_DEBUG_VERBOSE // print debugging info
#define DEBUG_PRINT (1)
@@ -40,7 +40,23 @@
#define DEBUG_printf(...) (void)0
#endif
-#if !MICROPY_PERSISTENT_CODE
+void mp_encode_uint(void *env, mp_encode_uint_allocator_t allocator, mp_uint_t val) {
+ // We store each 7 bits in a separate byte, and that's how many bytes needed
+ byte buf[MP_ENCODE_UINT_MAX_BYTES];
+ byte *p = buf + sizeof(buf);
+ // We encode in little-ending order, but store in big-endian, to help decoding
+ do {
+ *--p = val & 0x7f;
+ val >>= 7;
+ } while (val != 0);
+ byte *c = allocator(env, buf + sizeof(buf) - p);
+ if (c != NULL) {
+ while (p != buf + sizeof(buf) - 1) {
+ *c++ = *p++ | 0x80;
+ }
+ *c = *p;
+ }
+}
mp_uint_t mp_decode_uint(const byte **ptr) {
mp_uint_t unum = 0;
@@ -72,8 +88,6 @@ const byte *mp_decode_uint_skip(const byte *ptr) {
return ptr;
}
-#endif
-
STATIC NORETURN void fun_pos_args_mismatch(mp_obj_fun_bc_t *f, size_t expected, size_t given) {
#if MICROPY_ERROR_REPORTING <= MICROPY_ERROR_REPORTING_TERSE
// generic message, used also for other argument issues
@@ -107,8 +121,8 @@ STATIC void dump_args(const mp_obj_t *a, size_t sz) {
// On entry code_state should be allocated somewhere (stack/heap) and
// contain the following valid entries:
// - code_state->fun_bc should contain a pointer to the function object
-// - code_state->ip should contain the offset in bytes from the pointer
-// code_state->fun_bc->bytecode to the entry n_state (0 for bytecode, non-zero for native)
+// - code_state->ip should contain a pointer to the beginning of the prelude
+// - code_state->n_state should be the number of objects in the local state
void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw, const mp_obj_t *args) {
// This function is pretty complicated. It's main aim is to be efficient in speed and RAM
// usage for the common case of positional only args.
@@ -116,9 +130,6 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
// get the function object that we want to set up (could be bytecode or native code)
mp_obj_fun_bc_t *self = code_state->fun_bc;
- // ip comes in as an offset into bytecode, so turn it into a true pointer
- code_state->ip = self->bytecode + (size_t)code_state->ip;
-
#if MICROPY_STACKLESS
code_state->prev = NULL;
#endif
@@ -134,6 +145,7 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
// Decode prelude
size_t n_state_unused, n_exc_stack_unused, scope_flags, n_pos_args, n_kwonly_args, n_def_pos_args;
MP_BC_PRELUDE_SIG_DECODE_INTO(code_state->ip, n_state_unused, n_exc_stack_unused, scope_flags, n_pos_args, n_kwonly_args, n_def_pos_args);
+ MP_BC_PRELUDE_SIZE_DECODE(code_state->ip);
(void)n_state_unused;
(void)n_exc_stack_unused;
@@ -194,14 +206,20 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
*var_pos_kw_args = dict;
}
- // get pointer to arg_names array
- const mp_obj_t *arg_names = (const mp_obj_t *)self->const_table;
-
for (size_t i = 0; i < n_kw; i++) {
// the keys in kwargs are expected to be qstr objects
mp_obj_t wanted_arg_name = kwargs[2 * i];
+
+ // get pointer to arg_names array
+ const uint8_t *arg_names = code_state->ip;
+ arg_names = mp_decode_uint_skip(arg_names);
+
for (size_t j = 0; j < n_pos_args + n_kwonly_args; j++) {
- if (wanted_arg_name == arg_names[j]) {
+ qstr arg_qstr = mp_decode_uint(&arg_names);
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ arg_qstr = self->context->constants.qstr_table[arg_qstr];
+ #endif
+ if (wanted_arg_name == MP_OBJ_NEW_QSTR(arg_qstr)) {
if (code_state->state[n_state - 1 - j] != MP_OBJ_NULL) {
mp_raise_msg_varg(&mp_type_TypeError,
MP_ERROR_TEXT("function got multiple values for argument '%q'"), MP_OBJ_QSTR_VALUE(wanted_arg_name));
@@ -248,17 +266,25 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
// Check that all mandatory keyword args are specified
// Fill in default kw args if we have them
+ const uint8_t *arg_names = mp_decode_uint_skip(code_state->ip);
+ for (size_t i = 0; i < n_pos_args; i++) {
+ arg_names = mp_decode_uint_skip(arg_names);
+ }
for (size_t i = 0; i < n_kwonly_args; i++) {
+ qstr arg_qstr = mp_decode_uint(&arg_names);
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ arg_qstr = self->context->constants.qstr_table[arg_qstr];
+ #endif
if (code_state->state[n_state - 1 - n_pos_args - i] == MP_OBJ_NULL) {
mp_map_elem_t *elem = NULL;
if ((scope_flags & MP_SCOPE_FLAG_DEFKWARGS) != 0) {
- elem = mp_map_lookup(&((mp_obj_dict_t *)MP_OBJ_TO_PTR(self->extra_args[n_def_pos_args]))->map, arg_names[n_pos_args + i], MP_MAP_LOOKUP);
+ elem = mp_map_lookup(&((mp_obj_dict_t *)MP_OBJ_TO_PTR(self->extra_args[n_def_pos_args]))->map, MP_OBJ_NEW_QSTR(arg_qstr), MP_MAP_LOOKUP);
}
if (elem != NULL) {
code_state->state[n_state - 1 - n_pos_args - i] = elem->value;
} else {
mp_raise_msg_varg(&mp_type_TypeError,
- MP_ERROR_TEXT("function missing required keyword argument '%q'"), MP_OBJ_QSTR_VALUE(arg_names[n_pos_args + i]));
+ MP_ERROR_TEXT("function missing required keyword argument '%q'"), arg_qstr);
}
}
}
@@ -273,12 +299,8 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
}
}
- // read the size part of the prelude
- const byte *ip = code_state->ip;
- MP_BC_PRELUDE_SIZE_DECODE(ip);
-
- // jump over code info (source file and line-number mapping)
- ip += n_info;
+ // jump over code info (source file, argument names and line-number mapping)
+ const uint8_t *ip = code_state->ip + n_info;
// bytecode prelude: initialise closed over variables
for (; n_cell; --n_cell) {
@@ -287,11 +309,6 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
mp_obj_new_cell(code_state->state[n_state - 1 - local_num]);
}
- #if !MICROPY_PERSISTENT_CODE
- // so bytecode is aligned
- ip = MP_ALIGN(ip, sizeof(mp_uint_t));
- #endif
-
// now that we skipped over the prelude, set the ip for the VM
code_state->ip = ip;
diff --git a/py/bc.h b/py/bc.h
index ef5afeae16..7d761e30e6 100644
--- a/py/bc.h
+++ b/py/bc.h
@@ -28,7 +28,6 @@
#define MICROPY_INCLUDED_PY_BC_H
#include "py/runtime.h"
-#include "py/objfun.h"
// bytecode layout:
//
@@ -50,7 +49,9 @@
//
// source info section:
// simple_name : var qstr
-// source_file : var qstr
+// argname0 : var qstr
+// ... : var qstr
+// argnameN : var qstr N = num_pos_args + num_kwonly_args - 1
// <line number info>
//
// closure section:
@@ -58,19 +59,16 @@
// ... : byte
// local_numN : byte N = n_cells-1
//
-// <word alignment padding> only needed if bytecode contains pointers
-//
// <bytecode>
//
//
// constant table layout:
//
-// argname0 : obj (qstr)
-// ... : obj (qstr)
-// argnameN : obj (qstr) N = num_pos_args + num_kwonly_args
// const0 : obj
// constN : obj
+#define MP_ENCODE_UINT_MAX_BYTES ((MP_BYTES_PER_OBJ_WORD * 8 + 6) / 7)
+
#define MP_BC_PRELUDE_SIG_ENCODE(S, E, scope, out_byte, out_env) \
do { \
/*// Get values to store in prelude */ \
@@ -182,9 +180,9 @@ typedef struct _mp_bytecode_prelude_t {
uint n_pos_args;
uint n_kwonly_args;
uint n_def_pos_args;
- qstr qstr_block_name;
- qstr qstr_source_file;
+ qstr qstr_block_name_idx;
const byte *line_info;
+ const byte *line_info_top;
const byte *opcodes;
} mp_bytecode_prelude_t;
@@ -198,12 +196,46 @@ typedef struct _mp_exc_stack_t {
mp_obj_base_t *prev_exc;
} mp_exc_stack_t;
+// Constants associated with a module, to interface bytecode with runtime.
+typedef struct _mp_module_constants_t {
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ qstr_short_t *qstr_table;
+ #else
+ qstr source_file;
+ #endif
+ mp_obj_t *obj_table;
+} mp_module_constants_t;
+
+// State associated with a module.
+typedef struct _mp_module_context_t {
+ mp_obj_module_t module;
+ mp_module_constants_t constants;
+} mp_module_context_t;
+
+// Outer level struct defining a compiled module.
+typedef struct _mp_compiled_module_t {
+ const mp_module_context_t *context;
+ const struct _mp_raw_code_t *rc;
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ bool has_native;
+ size_t n_qstr;
+ size_t n_obj;
+ #endif
+} mp_compiled_module_t;
+
+// Outer level struct defining a frozen module.
+typedef struct _mp_frozen_module_t {
+ const mp_module_constants_t constants;
+ const struct _mp_raw_code_t *rc;
+} mp_frozen_module_t;
+
+// State for an executing function.
typedef struct _mp_code_state_t {
// The fun_bc entry points to the underlying function object that is being executed.
// It is needed to access the start of bytecode and the const_table.
// It is also needed to prevent the GC from reclaiming the bytecode during execution,
// because the ip pointer below will always point to the interior of the bytecode.
- mp_obj_fun_bc_t *fun_bc;
+ struct _mp_obj_fun_bc_t *fun_bc;
const byte *ip;
mp_obj_t *sp;
uint16_t n_state;
@@ -222,6 +254,10 @@ typedef struct _mp_code_state_t {
// mp_exc_stack_t exc_state[0];
} mp_code_state_t;
+// Allocator may return NULL, in which case data is not stored (can be used to compute size).
+typedef uint8_t *(*mp_encode_uint_allocator_t)(void *env, size_t nbytes);
+
+void mp_encode_uint(void *env, mp_encode_uint_allocator_t allocator, mp_uint_t val);
mp_uint_t mp_decode_uint(const byte **ptr);
mp_uint_t mp_decode_uint_value(const byte *ptr);
const byte *mp_decode_uint_skip(const byte *ptr);
@@ -229,10 +265,10 @@ const byte *mp_decode_uint_skip(const byte *ptr);
mp_vm_return_kind_t mp_execute_bytecode(mp_code_state_t *code_state, volatile mp_obj_t inject_exc);
mp_code_state_t *mp_obj_fun_bc_prepare_codestate(mp_obj_t func, size_t n_args, size_t n_kw, const mp_obj_t *args);
void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw, const mp_obj_t *args);
-void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *code, mp_uint_t len, const mp_uint_t *const_table);
-void mp_bytecode_print2(const mp_print_t *print, const byte *code, size_t len, const mp_uint_t *const_table);
+void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *code, mp_uint_t len, const mp_module_constants_t *cm);
+void mp_bytecode_print2(const mp_print_t *print, const byte *code, size_t len, const mp_module_constants_t *cm);
const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip);
-#define mp_bytecode_print_inst(print, code, const_table) mp_bytecode_print2(print, code, 1, const_table)
+#define mp_bytecode_print_inst(print, code, x_table) mp_bytecode_print2(print, code, 1, x_table)
// Helper macros to access pointer with least significant bits holding flags
#define MP_TAGPTR_PTR(x) ((void *)((uintptr_t)(x) & ~((uintptr_t)3)))
@@ -246,10 +282,26 @@ uint mp_opcode_format(const byte *ip, size_t *opcode_size, bool count_var_uint);
#endif
-static inline size_t mp_bytecode_get_source_line(const byte *line_info, size_t bc_offset) {
+static inline void mp_module_context_alloc_tables(mp_module_context_t *context, size_t n_qstr, size_t n_obj) {
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ size_t nq = (n_qstr * sizeof(qstr_short_t) + sizeof(mp_uint_t) - 1) / sizeof(mp_uint_t);
+ size_t no = n_obj;
+ mp_uint_t *mem = m_new(mp_uint_t, nq + no);
+ context->constants.qstr_table = (void *)(mem);
+ context->constants.obj_table = (void *)(mem + nq);
+ #else
+ if (n_obj == 0) {
+ context->constants.obj_table = NULL;
+ } else {
+ context->constants.obj_table = m_new(mp_obj_t, n_obj);
+ }
+ #endif
+}
+
+static inline size_t mp_bytecode_get_source_line(const byte *line_info, const byte *line_info_top, size_t bc_offset) {
size_t source_line = 1;
- size_t c;
- while ((c = *line_info)) {
+ while (line_info < line_info_top) {
+ size_t c = *line_info;
size_t b, l;
if ((c & 0x80) == 0) {
// 0b0LLBBBBB encoding
diff --git a/py/builtinevex.c b/py/builtinevex.c
index 800a20223a..4e1c6a66d1 100644
--- a/py/builtinevex.c
+++ b/py/builtinevex.c
@@ -54,7 +54,7 @@ STATIC mp_obj_t code_execute(mp_obj_code_t *self, mp_obj_dict_t *globals, mp_obj
// the correct one
if (mp_obj_is_type(self->module_fun, &mp_type_fun_bc)) {
mp_obj_fun_bc_t *fun_bc = MP_OBJ_TO_PTR(self->module_fun);
- fun_bc->globals = globals;
+ ((mp_module_context_t *)fun_bc->context)->module.globals = globals;
}
// execute code
diff --git a/py/builtinimport.c b/py/builtinimport.c
index 3e336633d9..094959f97d 100644
--- a/py/builtinimport.c
+++ b/py/builtinimport.c
@@ -146,28 +146,28 @@ STATIC mp_import_stat_t stat_top_level_dir_or_file(qstr mod_name, vstr_t *dest)
}
#if MICROPY_MODULE_FROZEN_STR || MICROPY_ENABLE_COMPILER
-STATIC void do_load_from_lexer(mp_obj_t module_obj, mp_lexer_t *lex) {
+STATIC void do_load_from_lexer(mp_module_context_t *context, mp_lexer_t *lex) {
#if MICROPY_PY___FILE__
qstr source_name = lex->source_name;
- mp_store_attr(module_obj, MP_QSTR___file__, MP_OBJ_NEW_QSTR(source_name));
+ mp_store_attr(MP_OBJ_FROM_PTR(&context->module), MP_QSTR___file__, MP_OBJ_NEW_QSTR(source_name));
#endif
// parse, compile and execute the module in its context
- mp_obj_dict_t *mod_globals = mp_obj_module_get_globals(module_obj);
+ mp_obj_dict_t *mod_globals = context->module.globals;
mp_parse_compile_execute(lex, MP_PARSE_FILE_INPUT, mod_globals, mod_globals);
}
#endif
#if (MICROPY_HAS_FILE_READER && MICROPY_PERSISTENT_CODE_LOAD) || MICROPY_MODULE_FROZEN_MPY
-STATIC void do_execute_raw_code(mp_obj_t module_obj, mp_raw_code_t *raw_code, const char *source_name) {
+STATIC void do_execute_raw_code(mp_module_context_t *context, const mp_raw_code_t *rc, const mp_module_context_t *mc, const char *source_name) {
(void)source_name;
#if MICROPY_PY___FILE__
- mp_store_attr(module_obj, MP_QSTR___file__, MP_OBJ_NEW_QSTR(qstr_from_str(source_name)));
+ mp_store_attr(MP_OBJ_FROM_PTR(&context->module), MP_QSTR___file__, MP_OBJ_NEW_QSTR(qstr_from_str(source_name)));
#endif
// execute the module in its context
- mp_obj_dict_t *mod_globals = mp_obj_module_get_globals(module_obj);
+ mp_obj_dict_t *mod_globals = context->module.globals;
// save context
mp_obj_dict_t *volatile old_globals = mp_globals_get();
@@ -179,7 +179,7 @@ STATIC void do_execute_raw_code(mp_obj_t module_obj, mp_raw_code_t *raw_code, co
nlr_buf_t nlr;
if (nlr_push(&nlr) == 0) {
- mp_obj_t module_fun = mp_make_function_from_raw_code(raw_code, MP_OBJ_NULL, MP_OBJ_NULL);
+ mp_obj_t module_fun = mp_make_function_from_raw_code(rc, mc, NULL);
mp_call_function_0(module_fun);
// finish nlr block, restore context
@@ -195,7 +195,7 @@ STATIC void do_execute_raw_code(mp_obj_t module_obj, mp_raw_code_t *raw_code, co
}
#endif
-STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
+STATIC void do_load(mp_module_context_t *module_obj, vstr_t *file) {
#if MICROPY_MODULE_FROZEN || MICROPY_ENABLE_COMPILER || (MICROPY_PERSISTENT_CODE_LOAD && MICROPY_HAS_FILE_READER)
const char *file_str = vstr_null_terminated_str(file);
#endif
@@ -222,7 +222,9 @@ STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
// its data) in the list of frozen files, execute it.
#if MICROPY_MODULE_FROZEN_MPY
if (frozen_type == MP_FROZEN_MPY) {
- do_execute_raw_code(module_obj, modref, file_str + frozen_path_prefix_len);
+ const mp_frozen_module_t *frozen = modref;
+ module_obj->constants = frozen->constants;
+ do_execute_raw_code(module_obj, frozen->rc, module_obj, file_str + frozen_path_prefix_len);
return;
}
#endif
@@ -234,8 +236,8 @@ STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
// the correct format and, if so, load and execute the file.
#if MICROPY_HAS_FILE_READER && MICROPY_PERSISTENT_CODE_LOAD
if (file_str[file->len - 3] == 'm') {
- mp_raw_code_t *raw_code = mp_raw_code_load_file(file_str);
- do_execute_raw_code(module_obj, raw_code, file_str);
+ mp_compiled_module_t cm = mp_raw_code_load_file(file_str, module_obj);
+ do_execute_raw_code(module_obj, cm.rc, cm.context, file_str);
return;
}
#endif
@@ -434,7 +436,7 @@ STATIC mp_obj_t process_import_at_level(qstr full_mod_name, qstr level_mod_name,
size_t orig_path_len = path->len;
vstr_add_str(path, PATH_SEP_CHAR "__init__.py");
if (stat_file_py_or_mpy(path) == MP_IMPORT_STAT_FILE) {
- do_load(module_obj, path);
+ do_load(MP_OBJ_TO_PTR(module_obj), path);
} else {
// No-op. Nothing to load.
// mp_warning("%s is imported as namespace package", vstr_str(&path));
@@ -443,7 +445,7 @@ STATIC mp_obj_t process_import_at_level(qstr full_mod_name, qstr level_mod_name,
path->len = orig_path_len;
} else { // MP_IMPORT_STAT_FILE
// File -- execute "path.(m)py".
- do_load(module_obj, path);
+ do_load(MP_OBJ_TO_PTR(module_obj), path);
// Note: This should be the last component in the import path. If
// there are remaining components then it's an ImportError
// because the current path(the module that was just loaded) is
diff --git a/py/compile.c b/py/compile.c
index 8ebcc2289f..92736e22e5 100644
--- a/py/compile.c
+++ b/py/compile.c
@@ -35,6 +35,7 @@
#include "py/compile.h"
#include "py/runtime.h"
#include "py/asmbase.h"
+#include "py/nativeglue.h"
#include "py/persistentcode.h"
#if MICROPY_ENABLE_COMPILER
@@ -88,7 +89,7 @@ typedef enum {
#if MICROPY_EMIT_NATIVE && MICROPY_DYNAMIC_COMPILER
#define NATIVE_EMITTER(f) emit_native_table[mp_dynamic_compiler.native_arch]->emit_##f
-#define NATIVE_EMITTER_TABLE emit_native_table[mp_dynamic_compiler.native_arch]
+#define NATIVE_EMITTER_TABLE (emit_native_table[mp_dynamic_compiler.native_arch])
STATIC const emit_method_table_t *emit_native_table[] = {
NULL,
@@ -121,7 +122,7 @@ STATIC const emit_method_table_t *emit_native_table[] = {
#else
#error "unknown native emitter"
#endif
-#define NATIVE_EMITTER_TABLE &NATIVE_EMITTER(method_table)
+#define NATIVE_EMITTER_TABLE (&NATIVE_EMITTER(method_table))
#endif
#if MICROPY_EMIT_INLINE_ASM && MICROPY_DYNAMIC_COMPILER
@@ -162,8 +163,6 @@ STATIC const emit_inline_asm_method_table_t *emit_asm_table[] = {
// elements in this struct are ordered to make it compact
typedef struct _compiler_t {
- qstr source_file;
-
uint8_t is_repl;
uint8_t pass; // holds enum type pass_kind_t
uint8_t have_star;
@@ -185,7 +184,10 @@ typedef struct _compiler_t {
scope_t *scope_head;
scope_t *scope_cur;
+ mp_emit_common_t emit_common;
+
emit_t *emit; // current emitter
+ emit_t *emit_bc;
#if NEED_METHOD_TABLE
const emit_method_table_t *emit_method_table; // current emit method table
#endif
@@ -196,6 +198,72 @@ typedef struct _compiler_t {
#endif
} compiler_t;
+/******************************************************************************/
+// mp_emit_common_t helper functions
+// These are defined here so they can be inlined, to reduce code size.
+
+STATIC void mp_emit_common_init(mp_emit_common_t *emit, qstr source_file) {
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ mp_map_init(&emit->qstr_map, 1);
+
+ // add the source file as the first entry in the qstr table
+ mp_map_elem_t *elem = mp_map_lookup(&emit->qstr_map, MP_OBJ_NEW_QSTR(source_file), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND);
+ elem->value = MP_OBJ_NEW_SMALL_INT(0);
+ #endif
+}
+
+STATIC void mp_emit_common_start_pass(mp_emit_common_t *emit, pass_kind_t pass) {
+ emit->pass = pass;
+ if (pass == MP_PASS_STACK_SIZE) {
+ emit->ct_cur_obj_base = emit->ct_cur_obj;
+ } else if (pass > MP_PASS_STACK_SIZE) {
+ emit->ct_cur_obj = emit->ct_cur_obj_base;
+ }
+ if (pass == MP_PASS_EMIT) {
+ if (emit->ct_cur_child == 0) {
+ emit->children = NULL;
+ } else {
+ emit->children = m_new0(mp_raw_code_t *, emit->ct_cur_child);
+ }
+ }
+ emit->ct_cur_child = 0;
+}
+
+STATIC void mp_emit_common_finalise(mp_emit_common_t *emit, bool has_native_code) {
+ emit->ct_cur_obj += has_native_code; // allocate an additional slot for &mp_fun_table
+ emit->const_table = m_new0(mp_uint_t, emit->ct_cur_obj);
+ emit->ct_cur_obj = has_native_code; // reserve slot 0 for &mp_fun_table
+ #if MICROPY_EMIT_NATIVE
+ if (has_native_code) {
+ // store mp_fun_table pointer at the start of the constant table
+ emit->const_table[0] = (mp_uint_t)(uintptr_t)&mp_fun_table;
+ }
+ #endif
+}
+
+STATIC void mp_emit_common_populate_module_context(mp_emit_common_t *emit, qstr source_file, mp_module_context_t *context) {
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ size_t qstr_map_used = emit->qstr_map.used;
+ mp_module_context_alloc_tables(context, qstr_map_used, emit->ct_cur_obj);
+ for (size_t i = 0; i < emit->qstr_map.alloc; ++i) {
+ if (mp_map_slot_is_filled(&emit->qstr_map, i)) {
+ size_t idx = MP_OBJ_SMALL_INT_VALUE(emit->qstr_map.table[i].value);
+ qstr qst = MP_OBJ_QSTR_VALUE(emit->qstr_map.table[i].key);
+ context->constants.qstr_table[idx] = qst;
+ }
+ }
+ #else
+ mp_module_context_alloc_tables(context, 0, emit->ct_cur_obj);
+ context->constants.source_file = source_file;
+ #endif
+
+ if (emit->ct_cur_obj > 0) {
+ memcpy(context->constants.obj_table, emit->const_table, emit->ct_cur_obj * sizeof(mp_uint_t));
+ }
+}
+
+/******************************************************************************/
+
STATIC void compile_error_set_line(compiler_t *comp, mp_parse_node_t pn) {
// if the line of the error is unknown then try to update it from the pn
if (comp->compile_error_line == 0 && MP_PARSE_NODE_IS_STRUCT(pn)) {
@@ -246,7 +314,7 @@ STATIC void compile_decrease_except_level(compiler_t *comp) {
}
STATIC scope_t *scope_new_and_link(compiler_t *comp, scope_kind_t kind, mp_parse_node_t pn, uint emit_options) {
- scope_t *scope = scope_new(kind, pn, comp->source_file, emit_options);
+ scope_t *scope = scope_new(kind, pn, emit_options);
scope->parent = comp->scope_cur;
scope->next = NULL;
if (comp->scope_head == NULL) {
@@ -290,7 +358,8 @@ STATIC void compile_generic_all_nodes(compiler_t *comp, mp_parse_node_struct_t *
STATIC void compile_load_id(compiler_t *comp, qstr qst) {
if (comp->pass == MP_PASS_SCOPE) {
mp_emit_common_get_id_for_load(comp->scope_cur, qst);
- } else {
+ }
+ {
#if NEED_METHOD_TABLE
mp_emit_common_id_op(comp->emit, &comp->emit_method_table->load_id, comp->scope_cur, qst);
#else
@@ -302,7 +371,8 @@ STATIC void compile_load_id(compiler_t *comp, qstr qst) {
STATIC void compile_store_id(compiler_t *comp, qstr qst) {
if (comp->pass == MP_PASS_SCOPE) {
mp_emit_common_get_id_for_modification(comp->scope_cur, qst);
- } else {
+ }
+ {
#if NEED_METHOD_TABLE
mp_emit_common_id_op(comp->emit, &comp->emit_method_table->store_id, comp->scope_cur, qst);
#else
@@ -314,7 +384,8 @@ STATIC void compile_store_id(compiler_t *comp, qstr qst) {
STATIC void compile_delete_id(compiler_t *comp, qstr qst) {
if (comp->pass == MP_PASS_SCOPE) {
mp_emit_common_get_id_for_modification(comp->scope_cur, qst);
- } else {
+ }
+ {
#if NEED_METHOD_TABLE
mp_emit_common_id_op(comp->emit, &comp->emit_method_table->delete_id, comp->scope_cur, qst);
#else
@@ -819,7 +890,7 @@ STATIC bool compile_built_in_decorator(compiler_t *comp, size_t name_len, mp_par
compile_syntax_error(comp, name_nodes[1], MP_ERROR_TEXT("invalid micropython decorator"));
}
- #if MICROPY_DYNAMIC_COMPILER
+ #if MICROPY_EMIT_NATIVE && MICROPY_DYNAMIC_COMPILER
if (*emit_options == MP_EMIT_OPT_NATIVE_PYTHON || *emit_options == MP_EMIT_OPT_VIPER) {
if (emit_native_table[mp_dynamic_compiler.native_arch] == NULL) {
compile_syntax_error(comp, name_nodes[1], MP_ERROR_TEXT("invalid arch"));
@@ -2082,6 +2153,7 @@ STATIC void compile_lambdef(compiler_t *comp, mp_parse_node_struct_t *pns) {
STATIC void compile_namedexpr_helper(compiler_t *comp, mp_parse_node_t pn_name, mp_parse_node_t pn_expr) {
if (!MP_PARSE_NODE_IS_ID(pn_name)) {
compile_syntax_error(comp, (mp_parse_node_t)pn_name, MP_ERROR_TEXT("can't assign to expression"));
+ return; // because pn_name is not a valid qstr (in compile_store_id below)
}
compile_node(comp, pn_expr);
EMIT(dup_top);
@@ -2814,6 +2886,7 @@ STATIC void compile_scope_func_lambda_param(compiler_t *comp, mp_parse_node_t pn
// comes before a star, so counts as a positional parameter
comp->scope_cur->num_pos_args += 1;
}
+ mp_emit_common_use_qstr(&comp->emit_common, param_name);
} else {
assert(MP_PARSE_NODE_IS_STRUCT(pn));
pns = (mp_parse_node_struct_t *)pn;
@@ -2827,6 +2900,7 @@ STATIC void compile_scope_func_lambda_param(compiler_t *comp, mp_parse_node_t pn
// comes before a star, so counts as a positional parameter
comp->scope_cur->num_pos_args += 1;
}
+ mp_emit_common_use_qstr(&comp->emit_common, param_name);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns) == pn_star) {
if (comp->have_star) {
// more than one star
@@ -2976,6 +3050,7 @@ STATIC void compile_scope(compiler_t *comp, scope_t *scope, pass_kind_t pass) {
comp->pass = pass;
comp->scope_cur = scope;
comp->next_label = 0;
+ mp_emit_common_start_pass(&comp->emit_common, pass);
EMIT_ARG(start_pass, pass, scope);
reserve_labels_for_native(comp, 6); // used by native's start_pass
@@ -2984,6 +3059,14 @@ STATIC void compile_scope(compiler_t *comp, scope_t *scope, pass_kind_t pass) {
// they will be computed in this first pass
scope->stack_size = 0;
scope->exc_stack_size = 0;
+
+ #if MICROPY_EMIT_NATIVE
+ if (scope->emit_options == MP_EMIT_OPT_NATIVE_PYTHON || scope->emit_options == MP_EMIT_OPT_VIPER) {
+ // allow native code to perfom basic tasks during the pass scope
+ // note: the first argument passed here is mp_emit_common_t, not the native emitter context
+ NATIVE_EMITTER_TABLE->start_pass((void *)&comp->emit_common, comp->pass, scope);
+ }
+ #endif
}
// compile
@@ -3064,6 +3147,7 @@ STATIC void compile_scope(compiler_t *comp, scope_t *scope, pass_kind_t pass) {
if (comp->pass == MP_PASS_SCOPE) {
scope_find_or_add_id(comp->scope_cur, qstr_arg, ID_INFO_KIND_LOCAL);
scope->num_pos_args = 1;
+ mp_emit_common_use_qstr(&comp->emit_common, MP_QSTR__star_);
}
// Set the source line number for the start of the comprehension
@@ -3296,9 +3380,10 @@ STATIC void compile_scope_inline_asm(compiler_t *comp, scope_t *scope, pass_kind
f, mp_asm_base_get_code_size((mp_asm_base_t *)comp->emit_inline_asm),
NULL,
#if MICROPY_PERSISTENT_CODE_SAVE
- 0, 0, 0, 0, NULL,
+ 0,
+ 0, 0, NULL,
#endif
- comp->scope_cur->num_pos_args, 0, type_sig);
+ 0, comp->scope_cur->num_pos_args, type_sig);
}
}
@@ -3310,7 +3395,7 @@ STATIC void compile_scope_inline_asm(compiler_t *comp, scope_t *scope, pass_kind
}
#endif
-STATIC void scope_compute_things(scope_t *scope) {
+STATIC void scope_compute_things(scope_t *scope, mp_emit_common_t *emit_common) {
// in MicroPython we put the *x parameter after all other parameters (except **y)
if (scope->scope_flags & MP_SCOPE_FLAG_VARARGS) {
id_info_t *id_param = NULL;
@@ -3399,6 +3484,7 @@ STATIC void scope_compute_things(scope_t *scope) {
}
scope->num_pos_args += num_free; // free vars are counted as params for passing them into the function
scope->num_locals += num_free;
+ mp_emit_common_use_qstr(emit_common, MP_QSTR__star_);
}
}
}
@@ -3406,15 +3492,15 @@ STATIC void scope_compute_things(scope_t *scope) {
#if !MICROPY_PERSISTENT_CODE_SAVE
STATIC
#endif
-mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl) {
+mp_compiled_module_t mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl, mp_module_context_t *context) {
// put compiler state on the stack, it's relatively small
compiler_t comp_state = {0};
compiler_t *comp = &comp_state;
- comp->source_file = source_file;
comp->is_repl = is_repl;
comp->break_label = INVALID_LABEL;
comp->continue_label = INVALID_LABEL;
+ mp_emit_common_init(&comp->emit_common, source_file);
// create the module scope
#if MICROPY_EMIT_NATIVE
@@ -3425,10 +3511,11 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
scope_t *module_scope = scope_new_and_link(comp, SCOPE_MODULE, parse_tree->root, emit_opt);
// create standard emitter; it's used at least for MP_PASS_SCOPE
- emit_t *emit_bc = emit_bc_new();
+ emit_t *emit_bc = emit_bc_new(&comp->emit_common);
- // compile pass 1
+ // compile MP_PASS_SCOPE
comp->emit = emit_bc;
+ comp->emit_bc = emit_bc;
#if MICROPY_EMIT_NATIVE
comp->emit_method_table = &emit_bc_method_table;
#endif
@@ -3458,14 +3545,24 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
}
// compute some things related to scope and identifiers
+ bool has_native_code = false;
for (scope_t *s = comp->scope_head; s != NULL && comp->compile_error == MP_OBJ_NULL; s = s->next) {
- scope_compute_things(s);
+ #if MICROPY_EMIT_NATIVE
+ if (s->emit_options == MP_EMIT_OPT_NATIVE_PYTHON || s->emit_options == MP_EMIT_OPT_VIPER) {
+ has_native_code = true;
+ }
+ #endif
+
+ scope_compute_things(s, &comp->emit_common);
}
// set max number of labels now that it's calculated
emit_bc_set_max_num_labels(emit_bc, max_num_labels);
- // compile pass 2 and 3
+ // finalise and allocate the constant table
+ mp_emit_common_finalise(&comp->emit_common, has_native_code);
+
+ // compile MP_PASS_STACK_SIZE, MP_PASS_CODE_SIZE, MP_PASS_EMIT
#if MICROPY_EMIT_NATIVE
emit_t *emit_native = NULL;
#endif
@@ -3505,7 +3602,7 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
case MP_EMIT_OPT_NATIVE_PYTHON:
case MP_EMIT_OPT_VIPER:
if (emit_native == NULL) {
- emit_native = NATIVE_EMITTER(new)(&comp->compile_error, &comp->next_label, max_num_labels);
+ emit_native = NATIVE_EMITTER(new)(&comp->emit_common, &comp->compile_error, &comp->next_label, max_num_labels);
}
comp->emit_method_table = NATIVE_EMITTER_TABLE;
comp->emit = emit_native;
@@ -3540,10 +3637,33 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
// number for the start of this scope
compile_error_set_line(comp, comp->scope_cur->pn);
// add a traceback to the exception using relevant source info
- mp_obj_exception_add_traceback(comp->compile_error, comp->source_file,
+ mp_obj_exception_add_traceback(comp->compile_error, source_file,
comp->compile_error_line, comp->scope_cur->simple_name);
}
+ // construct the global qstr/const table for this module
+ mp_compiled_module_t cm;
+ cm.rc = module_scope->raw_code;
+ cm.context = context;
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ cm.has_native = has_native_code;
+ cm.n_qstr = comp->emit_common.qstr_map.used;
+ cm.n_obj = comp->emit_common.ct_cur_obj;
+ #endif
+ if (comp->compile_error == MP_OBJ_NULL) {
+ mp_emit_common_populate_module_context(&comp->emit_common, source_file, context);
+
+ #if MICROPY_DEBUG_PRINTERS
+ // now that the module context is valid, the raw codes can be printed
+ if (mp_verbose_flag >= 2) {
+ for (scope_t *s = comp->scope_head; s != NULL; s = s->next) {
+ mp_raw_code_t *rc = s->raw_code;
+ mp_bytecode_print(&mp_plat_print, rc, rc->fun_data, rc->fun_data_len, &cm.context->constants);
+ }
+ }
+ #endif
+ }
+
// free the emitters
emit_bc_free(emit_bc);
@@ -3562,7 +3682,6 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
mp_parse_tree_clear(parse_tree);
// free the scopes
- mp_raw_code_t *outer_raw_code = module_scope->raw_code;
for (scope_t *s = module_scope; s;) {
scope_t *next = s->next;
scope_free(s);
@@ -3571,15 +3690,17 @@ mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_f
if (comp->compile_error != MP_OBJ_NULL) {
nlr_raise(comp->compile_error);
- } else {
- return outer_raw_code;
}
+
+ return cm;
}
mp_obj_t mp_compile(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl) {
- mp_raw_code_t *rc = mp_compile_to_raw_code(parse_tree, source_file, is_repl);
+ mp_module_context_t *context = m_new_obj(mp_module_context_t);
+ context->module.globals = mp_globals_get();
+ mp_compiled_module_t cm = mp_compile_to_raw_code(parse_tree, source_file, is_repl, context);
// return function that executes the outer module
- return mp_make_function_from_raw_code(rc, MP_OBJ_NULL, MP_OBJ_NULL);
+ return mp_make_function_from_raw_code(cm.rc, cm.context, NULL);
}
#endif // MICROPY_ENABLE_COMPILER
diff --git a/py/compile.h b/py/compile.h
index 1ad1f5e9cd..ae87bf2a04 100644
--- a/py/compile.h
+++ b/py/compile.h
@@ -32,11 +32,12 @@
// the compiler will raise an exception if an error occurred
// the compiler will clear the parse tree before it returns
+// mp_globals_get() will be used for the context
mp_obj_t mp_compile(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl);
#if MICROPY_PERSISTENT_CODE_SAVE
// this has the same semantics as mp_compile
-mp_raw_code_t *mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl);
+mp_compiled_module_t mp_compile_to_raw_code(mp_parse_tree_t *parse_tree, qstr source_file, bool is_repl, mp_module_context_t *globals);
#endif
// this is implemented in runtime.c
diff --git a/py/dynruntime.h b/py/dynruntime.h
index fdb91ed37e..b9dca60d13 100644
--- a/py/dynruntime.h
+++ b/py/dynruntime.h
@@ -30,6 +30,7 @@
// MicroPython runtime API defined in py/obj.h and py/runtime.h.
#include "py/nativeglue.h"
+#include "py/objfun.h"
#include "py/objstr.h"
#include "py/objtype.h"
@@ -177,8 +178,8 @@ static inline mp_obj_t mp_obj_len_dyn(mp_obj_t o) {
#define mp_unary_op(op, obj) (mp_fun_table.unary_op((op), (obj)))
#define mp_binary_op(op, lhs, rhs) (mp_fun_table.binary_op((op), (lhs), (rhs)))
-#define mp_make_function_from_raw_code(rc, def_args, def_kw_args) \
- (mp_fun_table.make_function_from_raw_code((rc), (def_args), (def_kw_args)))
+#define mp_make_function_from_raw_code(rc, context, def_args) \
+ (mp_fun_table.make_function_from_raw_code((rc), (context), (def_args)))
#define mp_call_function_n_kw(fun, n_args, n_kw, args) \
(mp_fun_table.call_function_n_kw((fun), (n_args) | ((n_kw) << 8), args))
@@ -187,11 +188,10 @@ static inline mp_obj_t mp_obj_len_dyn(mp_obj_t o) {
(mp_fun_table.arg_check_num_sig((n_args), (n_kw), MP_OBJ_FUN_MAKE_SIG((n_args_min), (n_args_max), (takes_kw))))
#define MP_DYNRUNTIME_INIT_ENTRY \
- mp_obj_t old_globals = mp_fun_table.swap_globals(self->globals); \
+ mp_obj_t old_globals = mp_fun_table.swap_globals(self->context->module.globals); \
mp_raw_code_t rc; \
rc.kind = MP_CODE_NATIVE_VIPER; \
rc.scope_flags = 0; \
- rc.const_table = (void *)self->const_table; \
(void)rc;
#define MP_DYNRUNTIME_INIT_EXIT \
@@ -199,7 +199,7 @@ static inline mp_obj_t mp_obj_len_dyn(mp_obj_t o) {
return mp_const_none;
#define MP_DYNRUNTIME_MAKE_FUNCTION(f) \
- (mp_make_function_from_raw_code((rc.fun_data = (f), &rc), MP_OBJ_NULL, MP_OBJ_NULL))
+ (mp_make_function_from_raw_code((rc.fun_data = (f), &rc), self->context, NULL))
#define mp_import_name(name, fromlist, level) \
(mp_fun_table.import_name((name), (fromlist), (level)))
diff --git a/py/emit.h b/py/emit.h
index 13bd3e9b2e..6f3593a0e8 100644
--- a/py/emit.h
+++ b/py/emit.h
@@ -92,6 +92,18 @@ typedef enum {
typedef struct _emit_t emit_t;
+typedef struct _mp_emit_common_t {
+ pass_kind_t pass;
+ uint16_t ct_cur_obj_base;
+ uint16_t ct_cur_obj;
+ uint16_t ct_cur_child;
+ mp_uint_t *const_table;
+ mp_raw_code_t **children;
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ mp_map_t qstr_map;
+ #endif
+} mp_emit_common_t;
+
typedef struct _mp_emit_method_table_id_ops_t {
void (*local)(emit_t *emit, qstr qst, mp_uint_t local_num, int kind);
void (*global)(emit_t *emit, qstr qst, int kind);
@@ -99,7 +111,7 @@ typedef struct _mp_emit_method_table_id_ops_t {
typedef struct _emit_method_table_t {
#if MICROPY_DYNAMIC_COMPILER
- emit_t *(*emit_new)(mp_obj_t * error_slot, uint *label_slot, mp_uint_t max_num_labels);
+ emit_t *(*emit_new)(mp_emit_common_t * emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
void (*emit_free)(emit_t *emit);
#endif
@@ -161,6 +173,28 @@ typedef struct _emit_method_table_t {
void (*end_except_handler)(emit_t *emit);
} emit_method_table_t;
+#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+qstr_short_t mp_emit_common_use_qstr(mp_emit_common_t *emit, qstr qst);
+#else
+static inline qstr_short_t mp_emit_common_use_qstr(mp_emit_common_t *emit, qstr qst) {
+ return qst;
+}
+#endif
+
+static inline size_t mp_emit_common_alloc_const_obj(mp_emit_common_t *emit, mp_obj_t obj) {
+ if (emit->pass == MP_PASS_EMIT) {
+ emit->const_table[emit->ct_cur_obj] = (mp_uint_t)obj;
+ }
+ return emit->ct_cur_obj++;
+}
+
+static inline size_t mp_emit_common_alloc_const_child(mp_emit_common_t *emit, mp_raw_code_t *rc) {
+ if (emit->pass == MP_PASS_EMIT) {
+ emit->children[emit->ct_cur_child] = rc;
+ }
+ return emit->ct_cur_child++;
+}
+
static inline void mp_emit_common_get_id_for_load(scope_t *scope, qstr qst) {
scope_find_or_add_id(scope, qst, ID_INFO_KIND_GLOBAL_IMPLICIT);
}
@@ -180,13 +214,13 @@ extern const mp_emit_method_table_id_ops_t mp_emit_bc_method_table_load_id_ops;
extern const mp_emit_method_table_id_ops_t mp_emit_bc_method_table_store_id_ops;
extern const mp_emit_method_table_id_ops_t mp_emit_bc_method_table_delete_id_ops;
-emit_t *emit_bc_new(void);
-emit_t *emit_native_x64_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
-emit_t *emit_native_x86_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
-emit_t *emit_native_thumb_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
-emit_t *emit_native_arm_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
-emit_t *emit_native_xtensa_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
-emit_t *emit_native_xtensawin_new(mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_bc_new(mp_emit_common_t *emit_common);
+emit_t *emit_native_x64_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_native_x86_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_native_thumb_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_native_arm_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_native_xtensa_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
+emit_t *emit_native_xtensawin_new(mp_emit_common_t *emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels);
void emit_bc_set_max_num_labels(emit_t *emit, mp_uint_t max_num_labels);
diff --git a/py/emitbc.c b/py/emitbc.c
index ca74046033..c04701ca78 100644
--- a/py/emitbc.c
+++ b/py/emitbc.c
@@ -36,8 +36,7 @@
#if MICROPY_ENABLE_COMPILER
-#define BYTES_FOR_INT ((MP_BYTES_PER_OBJ_WORD * 8 + 6) / 7)
-#define DUMMY_DATA_SIZE (BYTES_FOR_INT)
+#define DUMMY_DATA_SIZE (MP_ENCODE_UINT_MAX_BYTES)
struct _emit_t {
// Accessed as mp_obj_t, so must be aligned as such, and we rely on the
@@ -50,6 +49,7 @@ struct _emit_t {
int stack_size;
+ mp_emit_common_t *emit_common;
scope_t *scope;
mp_uint_t last_source_line_offset;
@@ -66,17 +66,11 @@ struct _emit_t {
size_t n_info;
size_t n_cell;
-
- #if MICROPY_PERSISTENT_CODE
- uint16_t ct_cur_obj;
- uint16_t ct_num_obj;
- uint16_t ct_cur_raw_code;
- #endif
- mp_uint_t *const_table;
};
-emit_t *emit_bc_new(void) {
+emit_t *emit_bc_new(mp_emit_common_t *emit_common) {
emit_t *emit = m_new0(emit_t, 1);
+ emit->emit_common = emit_common;
return emit;
}
@@ -90,26 +84,9 @@ void emit_bc_free(emit_t *emit) {
m_del_obj(emit_t, emit);
}
-typedef byte *(*emit_allocator_t)(emit_t *emit, int nbytes);
-
-STATIC void emit_write_uint(emit_t *emit, emit_allocator_t allocator, mp_uint_t val) {
- // We store each 7 bits in a separate byte, and that's how many bytes needed
- byte buf[BYTES_FOR_INT];
- byte *p = buf + sizeof(buf);
- // We encode in little-ending order, but store in big-endian, to help decoding
- do {
- *--p = val & 0x7f;
- val >>= 7;
- } while (val != 0);
- byte *c = allocator(emit, buf + sizeof(buf) - p);
- while (p != buf + sizeof(buf) - 1) {
- *c++ = *p++ | 0x80;
- }
- *c = *p;
-}
-
// all functions must go through this one to emit code info
-STATIC byte *emit_get_cur_to_write_code_info(emit_t *emit, int num_bytes_to_write) {
+STATIC uint8_t *emit_get_cur_to_write_code_info(void *emit_in, size_t num_bytes_to_write) {
+ emit_t *emit = emit_in;
if (emit->pass < MP_PASS_EMIT) {
emit->code_info_offset += num_bytes_to_write;
return emit->dummy_data;
@@ -126,14 +103,7 @@ STATIC void emit_write_code_info_byte(emit_t *emit, byte val) {
}
STATIC void emit_write_code_info_qstr(emit_t *emit, qstr qst) {
- #if MICROPY_PERSISTENT_CODE
- assert((qst >> 16) == 0);
- byte *c = emit_get_cur_to_write_code_info(emit, 2);
- c[0] = qst;
- c[1] = qst >> 8;
- #else
- emit_write_uint(emit, emit_get_cur_to_write_code_info, qst);
- #endif
+ mp_encode_uint(emit, emit_get_cur_to_write_code_info, mp_emit_common_use_qstr(emit->emit_common, qst));
}
#if MICROPY_ENABLE_SOURCE_LINE
@@ -166,7 +136,8 @@ STATIC void emit_write_code_info_bytes_lines(emit_t *emit, mp_uint_t bytes_to_sk
#endif
// all functions must go through this one to emit byte code
-STATIC byte *emit_get_cur_to_write_bytecode(emit_t *emit, int num_bytes_to_write) {
+STATIC uint8_t *emit_get_cur_to_write_bytecode(void *emit_in, size_t num_bytes_to_write) {
+ emit_t *emit = emit_in;
if (emit->pass < MP_PASS_EMIT) {
emit->bytecode_offset += num_bytes_to_write;
return emit->dummy_data;
@@ -189,12 +160,12 @@ STATIC void emit_write_bytecode_byte(emit_t *emit, int stack_adj, byte b1) {
c[0] = b1;
}
-// Similar to emit_write_bytecode_uint(), just some extra handling to encode sign
+// Similar to mp_encode_uint(), just some extra handling to encode sign
STATIC void emit_write_bytecode_byte_int(emit_t *emit, int stack_adj, byte b1, mp_int_t num) {
emit_write_bytecode_byte(emit, stack_adj, b1);
// We store each 7 bits in a separate byte, and that's how many bytes needed
- byte buf[BYTES_FOR_INT];
+ byte buf[MP_ENCODE_UINT_MAX_BYTES];
byte *p = buf + sizeof(buf);
// We encode in little-ending order, but store in big-endian, to help decoding
do {
@@ -218,61 +189,25 @@ STATIC void emit_write_bytecode_byte_int(emit_t *emit, int stack_adj, byte b1, m
STATIC void emit_write_bytecode_byte_uint(emit_t *emit, int stack_adj, byte b, mp_uint_t val) {
emit_write_bytecode_byte(emit, stack_adj, b);
- emit_write_uint(emit, emit_get_cur_to_write_bytecode, val);
+ mp_encode_uint(emit, emit_get_cur_to_write_bytecode, val);
}
-#if MICROPY_PERSISTENT_CODE
-STATIC void emit_write_bytecode_byte_const(emit_t *emit, int stack_adj, byte b, mp_uint_t n, mp_uint_t c) {
- if (emit->pass == MP_PASS_EMIT) {
- emit->const_table[n] = c;
- }
+STATIC void emit_write_bytecode_byte_const(emit_t *emit, int stack_adj, byte b, mp_uint_t n) {
emit_write_bytecode_byte_uint(emit, stack_adj, b, n);
}
-#endif
STATIC void emit_write_bytecode_byte_qstr(emit_t *emit, int stack_adj, byte b, qstr qst) {
- #if MICROPY_PERSISTENT_CODE
- assert((qst >> 16) == 0);
- mp_emit_bc_adjust_stack_size(emit, stack_adj);
- byte *c = emit_get_cur_to_write_bytecode(emit, 3);
- c[0] = b;
- c[1] = qst;
- c[2] = qst >> 8;
- #else
- emit_write_bytecode_byte_uint(emit, stack_adj, b, qst);
- #endif
+ emit_write_bytecode_byte_uint(emit, stack_adj, b, mp_emit_common_use_qstr(emit->emit_common, qst));
}
STATIC void emit_write_bytecode_byte_obj(emit_t *emit, int stack_adj, byte b, mp_obj_t obj) {
- #if MICROPY_PERSISTENT_CODE
emit_write_bytecode_byte_const(emit, stack_adj, b,
- emit->scope->num_pos_args + emit->scope->num_kwonly_args
- + emit->ct_cur_obj++, (mp_uint_t)obj);
- #else
- // aligns the pointer so it is friendly to GC
- emit_write_bytecode_byte(emit, stack_adj, b);
- emit->bytecode_offset = (size_t)MP_ALIGN(emit->bytecode_offset, sizeof(mp_obj_t));
- mp_obj_t *c = (mp_obj_t *)emit_get_cur_to_write_bytecode(emit, sizeof(mp_obj_t));
- // Verify thar c is already uint-aligned
- assert(c == MP_ALIGN(c, sizeof(mp_obj_t)));
- *c = obj;
- #endif
+ mp_emit_common_alloc_const_obj(emit->emit_common, obj));
}
-STATIC void emit_write_bytecode_byte_raw_code(emit_t *emit, int stack_adj, byte b, mp_raw_code_t *rc) {
- #if MICROPY_PERSISTENT_CODE
+STATIC void emit_write_bytecode_byte_child(emit_t *emit, int stack_adj, byte b, mp_raw_code_t *rc) {
emit_write_bytecode_byte_const(emit, stack_adj, b,
- emit->scope->num_pos_args + emit->scope->num_kwonly_args
- + emit->ct_num_obj + emit->ct_cur_raw_code++, (mp_uint_t)(uintptr_t)rc);
- #else
- // aligns the pointer so it is friendly to GC
- emit_write_bytecode_byte(emit, stack_adj, b);
- emit->bytecode_offset = (size_t)MP_ALIGN(emit->bytecode_offset, sizeof(void *));
- void **c = (void **)emit_get_cur_to_write_bytecode(emit, sizeof(void *));
- // Verify thar c is already uint-aligned
- assert(c == MP_ALIGN(c, sizeof(void *)));
- *c = rc;
- #endif
+ mp_emit_common_alloc_const_child(emit->emit_common, rc));
#if MICROPY_PY_SYS_SETTRACE
rc->line_of_definition = emit->last_source_line;
#endif
@@ -343,27 +278,19 @@ void mp_emit_bc_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scope) {
}
// Write number of cells and size of the source code info
- if (pass >= MP_PASS_CODE_SIZE) {
- MP_BC_PRELUDE_SIZE_ENCODE(emit->n_info, emit->n_cell, emit_write_code_info_byte, emit);
+ if (emit->pass >= MP_PASS_CODE_SIZE) {
+ size_t n_info = emit->n_info;
+ size_t n_cell = emit->n_cell;
+ MP_BC_PRELUDE_SIZE_ENCODE(n_info, n_cell, emit_write_code_info_byte, emit);
}
emit->n_info = emit->code_info_offset;
- // Write the name and source file of this function.
+ // Write the name of this function.
emit_write_code_info_qstr(emit, scope->simple_name);
- emit_write_code_info_qstr(emit, scope->source_file);
-
- #if MICROPY_PERSISTENT_CODE
- emit->ct_cur_obj = 0;
- emit->ct_cur_raw_code = 0;
- #endif
-
- if (pass == MP_PASS_EMIT) {
- // Write argument names (needed to resolve positional args passed as
- // keywords). We store them as full word-sized objects for efficient access
- // in mp_setup_code_state this is the start of the prelude and is guaranteed
- // to be aligned on a word boundary.
+ // Write argument names, needed to resolve positional args passed as keywords.
+ {
// For a given argument position (indexed by i) we need to find the
// corresponding id_info which is a parameter, as it has the correct
// qstr name to use as the argument name. Note that it's not a simple
@@ -383,7 +310,7 @@ void mp_emit_bc_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scope) {
break;
}
}
- emit->const_table[i] = (mp_uint_t)MP_OBJ_NEW_QSTR(qst);
+ emit_write_code_info_qstr(emit, qst);
}
}
}
@@ -396,8 +323,6 @@ void mp_emit_bc_end_pass(emit_t *emit) {
// check stack is back to zero size
assert(emit->stack_size == 0);
- emit_write_code_info_byte(emit, 0); // end of line number info
-
// Calculate size of source code info section
emit->n_info = emit->code_info_offset - emit->n_info;
@@ -412,39 +337,20 @@ void mp_emit_bc_end_pass(emit_t *emit) {
}
}
- #if MICROPY_PERSISTENT_CODE
- assert(emit->pass <= MP_PASS_STACK_SIZE || (emit->ct_num_obj == emit->ct_cur_obj));
- emit->ct_num_obj = emit->ct_cur_obj;
- #endif
-
if (emit->pass == MP_PASS_CODE_SIZE) {
- #if !MICROPY_PERSISTENT_CODE
- // so bytecode is aligned
- emit->code_info_offset = (size_t)MP_ALIGN(emit->code_info_offset, sizeof(mp_uint_t));
- #endif
-
// calculate size of total code-info + bytecode, in bytes
emit->code_info_size = emit->code_info_offset;
emit->bytecode_size = emit->bytecode_offset;
emit->code_base = m_new0(byte, emit->code_info_size + emit->bytecode_size);
- #if MICROPY_PERSISTENT_CODE
- emit->const_table = m_new0(mp_uint_t,
- emit->scope->num_pos_args + emit->scope->num_kwonly_args
- + emit->ct_cur_obj + emit->ct_cur_raw_code);
- #else
- emit->const_table = m_new0(mp_uint_t,
- emit->scope->num_pos_args + emit->scope->num_kwonly_args);
- #endif
-
} else if (emit->pass == MP_PASS_EMIT) {
mp_emit_glue_assign_bytecode(emit->scope->raw_code, emit->code_base,
#if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
emit->code_info_size + emit->bytecode_size,
#endif
- emit->const_table,
+ emit->emit_common->children,
#if MICROPY_PERSISTENT_CODE_SAVE
- emit->ct_cur_obj, emit->ct_cur_raw_code,
+ emit->emit_common->ct_cur_child,
#endif
emit->scope->scope_flags);
}
@@ -783,21 +689,21 @@ void mp_emit_bc_unpack_ex(emit_t *emit, mp_uint_t n_left, mp_uint_t n_right) {
void mp_emit_bc_make_function(emit_t *emit, scope_t *scope, mp_uint_t n_pos_defaults, mp_uint_t n_kw_defaults) {
if (n_pos_defaults == 0 && n_kw_defaults == 0) {
- emit_write_bytecode_byte_raw_code(emit, 1, MP_BC_MAKE_FUNCTION, scope->raw_code);
+ emit_write_bytecode_byte_child(emit, 1, MP_BC_MAKE_FUNCTION, scope->raw_code);
} else {
- emit_write_bytecode_byte_raw_code(emit, -1, MP_BC_MAKE_FUNCTION_DEFARGS, scope->raw_code);
+ emit_write_bytecode_byte_child(emit, -1, MP_BC_MAKE_FUNCTION_DEFARGS, scope->raw_code);
}
}
void mp_emit_bc_make_closure(emit_t *emit, scope_t *scope, mp_uint_t n_closed_over, mp_uint_t n_pos_defaults, mp_uint_t n_kw_defaults) {
if (n_pos_defaults == 0 && n_kw_defaults == 0) {
int stack_adj = -n_closed_over + 1;
- emit_write_bytecode_byte_raw_code(emit, stack_adj, MP_BC_MAKE_CLOSURE, scope->raw_code);
+ emit_write_bytecode_byte_child(emit, stack_adj, MP_BC_MAKE_CLOSURE, scope->raw_code);
emit_write_bytecode_raw_byte(emit, n_closed_over);
} else {
assert(n_closed_over <= 255);
int stack_adj = -2 - (mp_int_t)n_closed_over + 1;
- emit_write_bytecode_byte_raw_code(emit, stack_adj, MP_BC_MAKE_CLOSURE_DEFARGS, scope->raw_code);
+ emit_write_bytecode_byte_child(emit, stack_adj, MP_BC_MAKE_CLOSURE_DEFARGS, scope->raw_code);
emit_write_bytecode_raw_byte(emit, n_closed_over);
}
}
diff --git a/py/emitcommon.c b/py/emitcommon.c
index 791bf398ab..c1780d2db9 100644
--- a/py/emitcommon.c
+++ b/py/emitcommon.c
@@ -30,6 +30,17 @@
#if MICROPY_ENABLE_COMPILER
+#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+qstr_short_t mp_emit_common_use_qstr(mp_emit_common_t *emit, qstr qst) {
+ mp_map_elem_t *elem = mp_map_lookup(&emit->qstr_map, MP_OBJ_NEW_QSTR(qst), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND);
+ if (elem->value == MP_OBJ_NULL) {
+ assert(emit->pass == MP_PASS_SCOPE);
+ elem->value = MP_OBJ_NEW_SMALL_INT(emit->qstr_map.used - 1);
+ }
+ return MP_OBJ_SMALL_INT_VALUE(elem->value);
+}
+#endif
+
void mp_emit_common_get_id_for_modification(scope_t *scope, qstr qst) {
// name adding/lookup
id_info_t *id = scope_find_or_add_id(scope, qst, ID_INFO_KIND_GLOBAL_IMPLICIT);
diff --git a/py/emitglue.c b/py/emitglue.c
index 09b48682ff..cd902838af 100644
--- a/py/emitglue.c
+++ b/py/emitglue.c
@@ -34,6 +34,7 @@
#include "py/emitglue.h"
#include "py/runtime0.h"
#include "py/bc.h"
+#include "py/objfun.h"
#include "py/profile.h"
#if MICROPY_DEBUG_VERBOSE // print debugging info
@@ -63,20 +64,22 @@ void mp_emit_glue_assign_bytecode(mp_raw_code_t *rc, const byte *code,
#if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
size_t len,
#endif
- const mp_uint_t *const_table,
+ mp_raw_code_t **children,
#if MICROPY_PERSISTENT_CODE_SAVE
- uint16_t n_obj, uint16_t n_raw_code,
+ size_t n_children,
#endif
mp_uint_t scope_flags) {
rc->kind = MP_CODE_BYTECODE;
rc->scope_flags = scope_flags;
rc->fun_data = code;
- rc->const_table = const_table;
- #if MICROPY_PERSISTENT_CODE_SAVE
+ #if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
rc->fun_data_len = len;
- rc->n_obj = n_obj;
- rc->n_raw_code = n_raw_code;
+ #endif
+ rc->children = children;
+
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ rc->n_children = n_children;
#endif
#if MICROPY_PY_SYS_SETTRACE
@@ -85,26 +88,22 @@ void mp_emit_glue_assign_bytecode(mp_raw_code_t *rc, const byte *code,
#endif
#ifdef DEBUG_PRINT
- #if !MICROPY_DEBUG_PRINTERS
+ #if !(MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS)
const size_t len = 0;
#endif
DEBUG_printf("assign byte code: code=%p len=" UINT_FMT " flags=%x\n", code, len, (uint)scope_flags);
#endif
- #if MICROPY_DEBUG_PRINTERS
- if (mp_verbose_flag >= 2) {
- mp_bytecode_print(&mp_plat_print, rc, code, len, const_table);
- }
- #endif
}
#if MICROPY_EMIT_MACHINE_CODE
-void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, void *fun_data, mp_uint_t fun_len, const mp_uint_t *const_table,
+void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, void *fun_data, mp_uint_t fun_len,
+ mp_raw_code_t **children,
#if MICROPY_PERSISTENT_CODE_SAVE
+ size_t n_children,
uint16_t prelude_offset,
- uint16_t n_obj, uint16_t n_raw_code,
uint16_t n_qstr, mp_qstr_link_entry_t *qstr_link,
#endif
- mp_uint_t n_pos_args, mp_uint_t scope_flags, mp_uint_t type_sig) {
+ mp_uint_t scope_flags, mp_uint_t n_pos_args, mp_uint_t type_sig) {
assert(kind == MP_CODE_NATIVE_PY || kind == MP_CODE_NATIVE_VIPER || kind == MP_CODE_NATIVE_ASM);
@@ -135,20 +134,24 @@ void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, void
rc->kind = kind;
rc->scope_flags = scope_flags;
- rc->n_pos_args = n_pos_args;
rc->fun_data = fun_data;
- rc->const_table = const_table;
- rc->type_sig = type_sig;
- #if MICROPY_PERSISTENT_CODE_SAVE
+ #if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
rc->fun_data_len = fun_len;
+ #endif
+ rc->children = children;
+
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ rc->n_children = n_children;
rc->prelude_offset = prelude_offset;
- rc->n_obj = n_obj;
- rc->n_raw_code = n_raw_code;
rc->n_qstr = n_qstr;
rc->qstr_link = qstr_link;
#endif
+ // These two entries are only needed for MP_CODE_NATIVE_ASM.
+ rc->n_pos_args = n_pos_args;
+ rc->type_sig = type_sig;
+
#ifdef DEBUG_PRINT
DEBUG_printf("assign native: kind=%d fun=%p len=" UINT_FMT " n_pos_args=" UINT_FMT " flags=%x\n", kind, fun_data, fun_len, n_pos_args, (uint)scope_flags);
for (mp_uint_t i = 0; i < fun_len; i++) {
@@ -170,15 +173,15 @@ void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, void
}
#endif
-mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_args, mp_obj_t def_kw_args) {
+mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, const mp_module_context_t *context, const mp_obj_t *def_args) {
DEBUG_OP_printf("make_function_from_raw_code %p\n", rc);
assert(rc != NULL);
// def_args must be MP_OBJ_NULL or a tuple
- assert(def_args == MP_OBJ_NULL || mp_obj_is_type(def_args, &mp_type_tuple));
+ assert(def_args == NULL || def_args[0] == MP_OBJ_NULL || mp_obj_is_type(def_args[0], &mp_type_tuple));
// def_kw_args must be MP_OBJ_NULL or a dict
- assert(def_kw_args == MP_OBJ_NULL || mp_obj_is_type(def_kw_args, &mp_type_dict));
+ assert(def_args == NULL || def_args[1] == MP_OBJ_NULL || mp_obj_is_type(def_args[1], &mp_type_dict));
// make the function, depending on the raw code kind
mp_obj_t fun;
@@ -186,7 +189,7 @@ mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_ar
#if MICROPY_EMIT_NATIVE
case MP_CODE_NATIVE_PY:
case MP_CODE_NATIVE_VIPER:
- fun = mp_obj_new_fun_native(def_args, def_kw_args, rc->fun_data, rc->const_table);
+ fun = mp_obj_new_fun_native(def_args, rc->fun_data, context, rc->children);
// Check for a generator function, and if so change the type of the object
if ((rc->scope_flags & MP_SCOPE_FLAG_GENERATOR) != 0) {
((mp_obj_base_t *)MP_OBJ_TO_PTR(fun))->type = &mp_type_native_gen_wrap;
@@ -201,7 +204,7 @@ mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_ar
default:
// rc->kind should always be set and BYTECODE is the only remaining case
assert(rc->kind == MP_CODE_BYTECODE);
- fun = mp_obj_new_fun_bc(def_args, def_kw_args, rc->fun_data, rc->const_table);
+ fun = mp_obj_new_fun_bc(def_args, rc->fun_data, context, rc->children);
// check for generator functions and if so change the type of the object
if ((rc->scope_flags & MP_SCOPE_FLAG_GENERATOR) != 0) {
((mp_obj_base_t *)MP_OBJ_TO_PTR(fun))->type = &mp_type_gen_wrap;
@@ -218,16 +221,16 @@ mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_ar
return fun;
}
-mp_obj_t mp_make_closure_from_raw_code(const mp_raw_code_t *rc, mp_uint_t n_closed_over, const mp_obj_t *args) {
+mp_obj_t mp_make_closure_from_raw_code(const mp_raw_code_t *rc, const mp_module_context_t *context, mp_uint_t n_closed_over, const mp_obj_t *args) {
DEBUG_OP_printf("make_closure_from_raw_code %p " UINT_FMT " %p\n", rc, n_closed_over, args);
// make function object
mp_obj_t ffun;
if (n_closed_over & 0x100) {
// default positional and keyword args given
- ffun = mp_make_function_from_raw_code(rc, args[0], args[1]);
+ ffun = mp_make_function_from_raw_code(rc, context, args);
} else {
// default positional and keyword args not given
- ffun = mp_make_function_from_raw_code(rc, MP_OBJ_NULL, MP_OBJ_NULL);
+ ffun = mp_make_function_from_raw_code(rc, context, NULL);
}
// wrap function in closure object
return mp_obj_new_closure(ffun, n_closed_over & 0xff, args + ((n_closed_over >> 7) & 2));
diff --git a/py/emitglue.h b/py/emitglue.h
index a5411dc2e2..9a22a93218 100644
--- a/py/emitglue.h
+++ b/py/emitglue.h
@@ -54,16 +54,20 @@ typedef struct _mp_qstr_link_entry_t {
uint16_t qst;
} mp_qstr_link_entry_t;
+// compiled bytecode: instance in RAM, referenced by outer scope, usually freed after first (and only) use
+// mpy file: instance in RAM, created when .mpy file is loaded (same comments as above)
+// frozen: instance in ROM
typedef struct _mp_raw_code_t {
mp_uint_t kind : 3; // of type mp_raw_code_kind_t
mp_uint_t scope_flags : 7;
mp_uint_t n_pos_args : 11;
const void *fun_data;
- const mp_uint_t *const_table;
+ #if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
+ size_t fun_data_len; // so mp_raw_code_save and mp_bytecode_print work
+ #endif
+ struct _mp_raw_code_t **children;
#if MICROPY_PERSISTENT_CODE_SAVE
- size_t fun_data_len;
- uint16_t n_obj;
- uint16_t n_raw_code;
+ size_t n_children;
#if MICROPY_PY_SYS_SETTRACE
mp_bytecode_prelude_t prelude;
// line_of_definition is a Python source line where the raw_code was
@@ -89,22 +93,22 @@ void mp_emit_glue_assign_bytecode(mp_raw_code_t *rc, const byte *code,
#if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
size_t len,
#endif
- const mp_uint_t *const_table,
+ mp_raw_code_t **children,
#if MICROPY_PERSISTENT_CODE_SAVE
- uint16_t n_obj, uint16_t n_raw_code,
+ size_t n_children,
#endif
mp_uint_t scope_flags);
void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, void *fun_data, mp_uint_t fun_len,
- const mp_uint_t *const_table,
+ mp_raw_code_t **children,
#if MICROPY_PERSISTENT_CODE_SAVE
+ size_t n_children,
uint16_t prelude_offset,
- uint16_t n_obj, uint16_t n_raw_code,
uint16_t n_qstr, mp_qstr_link_entry_t *qstr_link,
#endif
- mp_uint_t n_pos_args, mp_uint_t scope_flags, mp_uint_t type_sig);
+ mp_uint_t scope_flags, mp_uint_t n_pos_args, mp_uint_t type_sig);
-mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, mp_obj_t def_args, mp_obj_t def_kw_args);
-mp_obj_t mp_make_closure_from_raw_code(const mp_raw_code_t *rc, mp_uint_t n_closed_over, const mp_obj_t *args);
+mp_obj_t mp_make_function_from_raw_code(const mp_raw_code_t *rc, const mp_module_context_t *context, const mp_obj_t *def_args);
+mp_obj_t mp_make_closure_from_raw_code(const mp_raw_code_t *rc, const mp_module_context_t *context, mp_uint_t n_closed_over, const mp_obj_t *args);
#endif // MICROPY_INCLUDED_PY_EMITGLUE_H
diff --git a/py/emitnative.c b/py/emitnative.c
index 6504f37765..ca34e89f64 100644
--- a/py/emitnative.c
+++ b/py/emitnative.c
@@ -48,6 +48,7 @@
#include "py/emit.h"
#include "py/nativeglue.h"
+#include "py/objfun.h"
#include "py/objstr.h"
#if MICROPY_DEBUG_VERBOSE // print debugging info
@@ -92,9 +93,13 @@
#define OFFSETOF_CODE_STATE_FUN_BC (offsetof(mp_code_state_t, fun_bc) / sizeof(uintptr_t))
#define OFFSETOF_CODE_STATE_IP (offsetof(mp_code_state_t, ip) / sizeof(uintptr_t))
#define OFFSETOF_CODE_STATE_SP (offsetof(mp_code_state_t, sp) / sizeof(uintptr_t))
-#define OFFSETOF_OBJ_FUN_BC_GLOBALS (offsetof(mp_obj_fun_bc_t, globals) / sizeof(uintptr_t))
+#define OFFSETOF_CODE_STATE_N_STATE (offsetof(mp_code_state_t, n_state) / sizeof(uintptr_t))
+#define OFFSETOF_OBJ_FUN_BC_CONTEXT (offsetof(mp_obj_fun_bc_t, context) / sizeof(uintptr_t))
+#define OFFSETOF_OBJ_FUN_BC_CHILD_TABLE (offsetof(mp_obj_fun_bc_t, child_table) / sizeof(uintptr_t))
#define OFFSETOF_OBJ_FUN_BC_BYTECODE (offsetof(mp_obj_fun_bc_t, bytecode) / sizeof(uintptr_t))
-#define OFFSETOF_OBJ_FUN_BC_CONST_TABLE (offsetof(mp_obj_fun_bc_t, const_table) / sizeof(uintptr_t))
+#define OFFSETOF_MODULE_CONTEXT_OBJ_TABLE (offsetof(mp_module_context_t, constants.obj_table) / sizeof(uintptr_t))
+#define OFFSETOF_MODULE_CONTEXT_GLOBALS (offsetof(mp_module_context_t, module.globals) / sizeof(uintptr_t))
+#define INDEX_OF_MP_FUN_TABLE_IN_CONST_TABLE (0)
// If not already defined, set parent args to same as child call registers
#ifndef REG_PARENT_RET
@@ -205,6 +210,7 @@ typedef struct _exc_stack_entry_t {
} exc_stack_entry_t;
struct _emit_t {
+ mp_emit_common_t *emit_common;
mp_obj_t *error_slot;
uint *label_slot;
uint exit_label;
@@ -225,18 +231,17 @@ struct _emit_t {
exc_stack_entry_t *exc_stack;
int prelude_offset;
+ #if N_PRELUDE_AS_BYTES_OBJ
+ size_t prelude_const_table_offset;
+ #endif
int start_offset;
int n_state;
uint16_t code_state_start;
uint16_t stack_start;
int stack_size;
+ uint16_t n_info;
uint16_t n_cell;
- uint16_t const_table_cur_obj;
- uint16_t const_table_num_obj;
- uint16_t const_table_cur_raw_code;
- mp_uint_t *const_table;
-
#if MICROPY_PERSISTENT_CODE_SAVE
uint16_t qstr_link_cur;
mp_qstr_link_entry_t *qstr_link;
@@ -255,8 +260,9 @@ STATIC void emit_native_global_exc_entry(emit_t *emit);
STATIC void emit_native_global_exc_exit(emit_t *emit);
STATIC void emit_native_load_const_obj(emit_t *emit, mp_obj_t obj);
-emit_t *EXPORT_FUN(new)(mp_obj_t * error_slot, uint *label_slot, mp_uint_t max_num_labels) {
+emit_t *EXPORT_FUN(new)(mp_emit_common_t * emit_common, mp_obj_t *error_slot, uint *label_slot, mp_uint_t max_num_labels) {
emit_t *emit = m_new0(emit_t, 1);
+ emit->emit_common = emit_common;
emit->error_slot = error_slot;
emit->label_slot = label_slot;
emit->stack_info_alloc = 8;
@@ -340,30 +346,22 @@ STATIC void emit_native_mov_reg_qstr_obj(emit_t *emit, int reg_dest, qstr qst) {
emit_native_mov_state_reg((emit), (local_num), (reg_temp)); \
} while (false)
-#define emit_native_mov_state_imm_fix_u16_via(emit, local_num, imm, reg_temp) \
- do { \
- ASM_MOV_REG_IMM_FIX_U16((emit)->as, (reg_temp), (imm)); \
- emit_native_mov_state_reg((emit), (local_num), (reg_temp)); \
- } while (false)
-
-#define emit_native_mov_state_imm_fix_word_via(emit, local_num, imm, reg_temp) \
- do { \
- ASM_MOV_REG_IMM_FIX_WORD((emit)->as, (reg_temp), (imm)); \
- emit_native_mov_state_reg((emit), (local_num), (reg_temp)); \
- } while (false)
-
STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scope) {
DEBUG_printf("start_pass(pass=%u, scope=%p)\n", pass, scope);
+ if (pass == MP_PASS_SCOPE) {
+ // Note: the first argument passed here is mp_emit_common_t, not the native emitter context
+ #if N_PRELUDE_AS_BYTES_OBJ
+ if (scope->emit_options == MP_EMIT_OPT_NATIVE_PYTHON) {
+ mp_emit_common_alloc_const_obj((mp_emit_common_t *)emit, mp_const_none);
+ }
+ #endif
+ return;
+ }
+
emit->pass = pass;
emit->do_viper_types = scope->emit_options == MP_EMIT_OPT_VIPER;
emit->stack_size = 0;
- #if N_PRELUDE_AS_BYTES_OBJ
- emit->const_table_cur_obj = emit->do_viper_types ? 0 : 1; // reserve first obj for prelude bytes obj
- #else
- emit->const_table_cur_obj = 0;
- #endif
- emit->const_table_cur_raw_code = 0;
#if MICROPY_PERSISTENT_CODE_SAVE
emit->qstr_link_cur = 0;
#endif
@@ -455,8 +453,9 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
#endif
// Load REG_FUN_TABLE with a pointer to mp_fun_table, found in the const_table
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_CONST_TABLE);
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_LOCAL_3, 0);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_FUN_TABLE, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_FUN_TABLE, INDEX_OF_MP_FUN_TABLE_IN_CONST_TABLE);
// Store function object (passed as first arg) to stack if needed
if (NEED_FUN_OBJ(emit)) {
@@ -514,7 +513,7 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
emit->stack_start = SIZEOF_CODE_STATE;
#if N_PRELUDE_AS_BYTES_OBJ
// Load index of prelude bytes object in const_table
- mp_asm_base_data(&emit->as->base, ASM_WORD_SIZE, (uintptr_t)(emit->scope->num_pos_args + emit->scope->num_kwonly_args + 1));
+ mp_asm_base_data(&emit->as->base, ASM_WORD_SIZE, (uintptr_t)emit->prelude_const_table_offset);
#else
mp_asm_base_data(&emit->as->base, ASM_WORD_SIZE, (uintptr_t)emit->prelude_offset);
#endif
@@ -536,8 +535,9 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
// Load REG_FUN_TABLE with a pointer to mp_fun_table, found in the const_table
ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_GENERATOR_STATE, LOCAL_IDX_FUN_OBJ(emit));
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_OBJ_FUN_BC_CONST_TABLE);
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_TEMP0, emit->scope->num_pos_args + emit->scope->num_kwonly_args);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_TEMP0, INDEX_OF_MP_FUN_TABLE_IN_CONST_TABLE);
} else {
// The locals and stack start after the code_state structure
emit->stack_start = emit->code_state_start + SIZEOF_CODE_STATE;
@@ -555,22 +555,33 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
#endif
// Load REG_FUN_TABLE with a pointer to mp_fun_table, found in the const_table
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_CONST_TABLE);
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_LOCAL_3, emit->scope->num_pos_args + emit->scope->num_kwonly_args);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_FUN_TABLE, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_FUN_TABLE, INDEX_OF_MP_FUN_TABLE_IN_CONST_TABLE);
// Set code_state.fun_bc
ASM_MOV_LOCAL_REG(emit->as, LOCAL_IDX_FUN_OBJ(emit), REG_PARENT_ARG_1);
- // Set code_state.ip (offset from start of this function to prelude info)
+ // Set code_state.ip, a pointer to the beginning of the prelude
+ // Need to use some locals for this, so assert that they are available for use
+ MP_STATIC_ASSERT(REG_LOCAL_3 != REG_PARENT_ARG_1);
+ MP_STATIC_ASSERT(REG_LOCAL_3 != REG_PARENT_ARG_2);
+ MP_STATIC_ASSERT(REG_LOCAL_3 != REG_PARENT_ARG_3);
+ MP_STATIC_ASSERT(REG_LOCAL_3 != REG_PARENT_ARG_4);
int code_state_ip_local = emit->code_state_start + OFFSETOF_CODE_STATE_IP;
#if N_PRELUDE_AS_BYTES_OBJ
- // Prelude is a bytes object in const_table; store ip = prelude->data - fun_bc->bytecode
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_LOCAL_3, emit->scope->num_pos_args + emit->scope->num_kwonly_args + 1);
+ // Prelude is a bytes object in const_table[prelude_const_table_offset].
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_LOCAL_3, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_LOCAL_3, emit->prelude_const_table_offset);
ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_LOCAL_3, offsetof(mp_obj_str_t, data) / sizeof(uintptr_t));
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_PARENT_ARG_1, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_BYTECODE);
- ASM_SUB_REG_REG(emit->as, REG_LOCAL_3, REG_PARENT_ARG_1);
- emit_native_mov_state_reg(emit, code_state_ip_local, REG_LOCAL_3);
#else
+ MP_STATIC_ASSERT(REG_LOCAL_2 != REG_PARENT_ARG_1);
+ MP_STATIC_ASSERT(REG_LOCAL_2 != REG_PARENT_ARG_2);
+ MP_STATIC_ASSERT(REG_LOCAL_2 != REG_PARENT_ARG_3);
+ MP_STATIC_ASSERT(REG_LOCAL_2 != REG_PARENT_ARG_4);
+ // Prelude is at the end of the machine code
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_3, REG_PARENT_ARG_1, OFFSETOF_OBJ_FUN_BC_BYTECODE);
if (emit->pass == MP_PASS_CODE_SIZE) {
// Commit to the encoding size based on the value of prelude_offset in this pass.
// By using 32768 as the cut-off it is highly unlikely that prelude_offset will
@@ -579,14 +590,16 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
}
if (emit->prelude_offset_uses_u16_encoding) {
assert(emit->prelude_offset <= 65535);
- emit_native_mov_state_imm_fix_u16_via(emit, code_state_ip_local, emit->prelude_offset, REG_PARENT_ARG_1);
+ ASM_MOV_REG_IMM_FIX_U16((emit)->as, REG_LOCAL_2, emit->prelude_offset);
} else {
- emit_native_mov_state_imm_fix_word_via(emit, code_state_ip_local, emit->prelude_offset, REG_PARENT_ARG_1);
+ ASM_MOV_REG_IMM_FIX_WORD((emit)->as, REG_LOCAL_2, emit->prelude_offset);
}
+ ASM_ADD_REG_REG(emit->as, REG_LOCAL_3, REG_LOCAL_2);
#endif
+ emit_native_mov_state_reg(emit, code_state_ip_local, REG_LOCAL_3);
// Set code_state.n_state (only works on little endian targets due to n_state being uint16_t)
- emit_native_mov_state_imm_via(emit, emit->code_state_start + offsetof(mp_code_state_t, n_state) / sizeof(uintptr_t), emit->n_state, REG_ARG_1);
+ emit_native_mov_state_imm_via(emit, emit->code_state_start + OFFSETOF_CODE_STATE_N_STATE, emit->n_state, REG_ARG_1);
// Put address of code_state into first arg
ASM_MOV_REG_LOCAL_ADDR(emit->as, REG_ARG_1, emit->code_state_start);
@@ -628,30 +641,17 @@ STATIC void emit_native_start_pass(emit_t *emit, pass_kind_t pass, scope_t *scop
emit->local_vtype[id->local_num] = VTYPE_PYOBJ;
}
}
-
- if (pass == MP_PASS_EMIT) {
- // write argument names as qstr objects
- // see comment in corresponding part of emitbc.c about the logic here
- for (int i = 0; i < scope->num_pos_args + scope->num_kwonly_args; i++) {
- qstr qst = MP_QSTR__star_;
- for (int j = 0; j < scope->id_info_len; ++j) {
- id_info_t *id = &scope->id_info[j];
- if ((id->flags & ID_FLAG_IS_PARAM) && id->local_num == i) {
- qst = id->qst;
- break;
- }
- }
- emit->const_table[i] = (mp_uint_t)MP_OBJ_NEW_QSTR(qst);
- }
- }
}
-
}
static inline void emit_native_write_code_info_byte(emit_t *emit, byte val) {
mp_asm_base_data(&emit->as->base, 1, val);
}
+static inline void emit_native_write_code_info_qstr(emit_t *emit, qstr qst) {
+ mp_encode_uint(&emit->as->base, mp_asm_base_get_cur_to_write_bytes, mp_emit_common_use_qstr(emit->emit_common, qst));
+}
+
STATIC void emit_native_end_pass(emit_t *emit) {
emit_native_global_exc_exit(emit);
@@ -662,21 +662,25 @@ STATIC void emit_native_end_pass(emit_t *emit) {
size_t n_exc_stack = 0; // exc-stack not needed for native code
MP_BC_PRELUDE_SIG_ENCODE(n_state, n_exc_stack, emit->scope, emit_native_write_code_info_byte, emit);
- #if MICROPY_PERSISTENT_CODE
- size_t n_info = 4;
- #else
- size_t n_info = 1;
- #endif
- MP_BC_PRELUDE_SIZE_ENCODE(n_info, emit->n_cell, emit_native_write_code_info_byte, emit);
-
- #if MICROPY_PERSISTENT_CODE
- mp_asm_base_data(&emit->as->base, 1, emit->scope->simple_name);
- mp_asm_base_data(&emit->as->base, 1, emit->scope->simple_name >> 8);
- mp_asm_base_data(&emit->as->base, 1, emit->scope->source_file);
- mp_asm_base_data(&emit->as->base, 1, emit->scope->source_file >> 8);
- #else
- mp_asm_base_data(&emit->as->base, 1, 1);
- #endif
+ size_t n_info = emit->n_info;
+ size_t n_cell = emit->n_cell;
+ MP_BC_PRELUDE_SIZE_ENCODE(n_info, n_cell, emit_native_write_code_info_byte, emit);
+
+ // bytecode prelude: source info (function and argument qstrs)
+ size_t info_start = mp_asm_base_get_code_pos(&emit->as->base);
+ emit_native_write_code_info_qstr(emit, emit->scope->simple_name);
+ for (int i = 0; i < emit->scope->num_pos_args + emit->scope->num_kwonly_args; i++) {
+ qstr qst = MP_QSTR__star_;
+ for (int j = 0; j < emit->scope->id_info_len; ++j) {
+ id_info_t *id = &emit->scope->id_info[j];
+ if ((id->flags & ID_FLAG_IS_PARAM) && id->local_num == i) {
+ qst = id->qst;
+ break;
+ }
+ }
+ emit_native_write_code_info_qstr(emit, qst);
+ }
+ emit->n_info = mp_asm_base_get_code_pos(&emit->as->base) - info_start;
// bytecode prelude: initialise closed over variables
size_t cell_start = mp_asm_base_get_code_pos(&emit->as->base);
@@ -690,13 +694,14 @@ STATIC void emit_native_end_pass(emit_t *emit) {
emit->n_cell = mp_asm_base_get_code_pos(&emit->as->base) - cell_start;
#if N_PRELUDE_AS_BYTES_OBJ
- // Prelude bytes object is after qstr arg names and mp_fun_table
- size_t table_off = emit->scope->num_pos_args + emit->scope->num_kwonly_args + 1;
+ // Create the prelude as a bytes object, and store it in the constant table
+ mp_obj_t prelude = mp_const_none;
if (emit->pass == MP_PASS_EMIT) {
void *buf = emit->as->base.code_base + emit->prelude_offset;
size_t n = emit->as->base.code_offset - emit->prelude_offset;
- emit->const_table[table_off] = (uintptr_t)mp_obj_new_bytes(buf, n);
+ prelude = mp_obj_new_bytes(buf, n);
}
+ emit->prelude_const_table_offset = mp_emit_common_alloc_const_obj(emit->emit_common, prelude);
#endif
}
@@ -706,31 +711,15 @@ STATIC void emit_native_end_pass(emit_t *emit) {
assert(emit->stack_size == 0);
assert(emit->exc_stack_size == 0);
- // Deal with const table accounting
- assert(emit->pass <= MP_PASS_STACK_SIZE || (emit->const_table_num_obj == emit->const_table_cur_obj));
- emit->const_table_num_obj = emit->const_table_cur_obj;
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ // Allocate qstr_link table if needed
if (emit->pass == MP_PASS_CODE_SIZE) {
- size_t const_table_alloc = 1 + emit->const_table_num_obj + emit->const_table_cur_raw_code;
- size_t nqstr = 0;
- if (!emit->do_viper_types) {
- // Add room for qstr names of arguments
- nqstr = emit->scope->num_pos_args + emit->scope->num_kwonly_args;
- const_table_alloc += nqstr;
- }
- emit->const_table = m_new(mp_uint_t, const_table_alloc);
- #if !MICROPY_DYNAMIC_COMPILER
- // Store mp_fun_table pointer just after qstrs
- // (but in dynamic-compiler mode eliminate dependency on mp_fun_table)
- emit->const_table[nqstr] = (mp_uint_t)(uintptr_t)&mp_fun_table;
- #endif
-
- #if MICROPY_PERSISTENT_CODE_SAVE
size_t qstr_link_alloc = emit->qstr_link_cur;
if (qstr_link_alloc > 0) {
emit->qstr_link = m_new(mp_qstr_link_entry_t, qstr_link_alloc);
}
- #endif
}
+ #endif
if (emit->pass == MP_PASS_EMIT) {
void *f = mp_asm_base_get_code(&emit->as->base);
@@ -738,13 +727,14 @@ STATIC void emit_native_end_pass(emit_t *emit) {
mp_emit_glue_assign_native(emit->scope->raw_code,
emit->do_viper_types ? MP_CODE_NATIVE_VIPER : MP_CODE_NATIVE_PY,
- f, f_len, emit->const_table,
+ f, f_len,
+ emit->emit_common->children,
#if MICROPY_PERSISTENT_CODE_SAVE
+ emit->emit_common->ct_cur_child,
emit->prelude_offset,
- emit->const_table_cur_obj, emit->const_table_cur_raw_code,
emit->qstr_link_cur, emit->qstr_link,
#endif
- emit->scope->num_pos_args, emit->scope->scope_flags, 0);
+ emit->scope->scope_flags, 0, 0);
}
}
@@ -1137,29 +1127,19 @@ STATIC exc_stack_entry_t *emit_native_pop_exc_stack(emit_t *emit) {
return e;
}
-STATIC void emit_load_reg_with_ptr(emit_t *emit, int reg, mp_uint_t ptr, size_t table_off) {
- if (!emit->do_viper_types) {
- // Skip qstr names of arguments
- table_off += emit->scope->num_pos_args + emit->scope->num_kwonly_args;
- }
- if (emit->pass == MP_PASS_EMIT) {
- emit->const_table[table_off] = ptr;
- }
+STATIC void emit_load_reg_with_object(emit_t *emit, int reg, mp_obj_t obj) {
+ size_t table_off = mp_emit_common_alloc_const_obj(emit->emit_common, obj);
emit_native_mov_reg_state(emit, REG_TEMP0, LOCAL_IDX_FUN_OBJ(emit));
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_OBJ_FUN_BC_CONST_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
ASM_LOAD_REG_REG_OFFSET(emit->as, reg, REG_TEMP0, table_off);
}
-STATIC void emit_load_reg_with_object(emit_t *emit, int reg, mp_obj_t obj) {
- // First entry is for mp_fun_table
- size_t table_off = 1 + emit->const_table_cur_obj++;
- emit_load_reg_with_ptr(emit, reg, (mp_uint_t)obj, table_off);
-}
-
-STATIC void emit_load_reg_with_raw_code(emit_t *emit, int reg, mp_raw_code_t *rc) {
- // First entry is for mp_fun_table, then constant objects
- size_t table_off = 1 + emit->const_table_num_obj + emit->const_table_cur_raw_code++;
- emit_load_reg_with_ptr(emit, reg, (mp_uint_t)rc, table_off);
+STATIC void emit_load_reg_with_child(emit_t *emit, int reg, mp_raw_code_t *rc) {
+ size_t table_off = mp_emit_common_alloc_const_child(emit->emit_common, rc);
+ emit_native_mov_reg_state(emit, REG_TEMP0, LOCAL_IDX_FUN_OBJ(emit));
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_TEMP0, REG_TEMP0, OFFSETOF_OBJ_FUN_BC_CHILD_TABLE);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, reg, REG_TEMP0, table_off);
}
STATIC void emit_native_label_assign(emit_t *emit, mp_uint_t l) {
@@ -1203,7 +1183,8 @@ STATIC void emit_native_global_exc_entry(emit_t *emit) {
if (!(emit->scope->scope_flags & MP_SCOPE_FLAG_GENERATOR)) {
// Set new globals
emit_native_mov_reg_state(emit, REG_ARG_1, LOCAL_IDX_FUN_OBJ(emit));
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_ARG_1, REG_ARG_1, OFFSETOF_OBJ_FUN_BC_GLOBALS);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_ARG_1, REG_ARG_1, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_ARG_1, REG_ARG_1, OFFSETOF_MODULE_CONTEXT_GLOBALS);
emit_call(emit, MP_F_NATIVE_SWAP_GLOBALS);
// Save old globals (or NULL if globals didn't change)
@@ -1254,7 +1235,8 @@ STATIC void emit_native_global_exc_entry(emit_t *emit) {
#if N_NLR_SETJMP
// Reload REG_FUN_TABLE, since it may be clobbered by longjmp
emit_native_mov_reg_state(emit, REG_LOCAL_1, LOCAL_IDX_FUN_OBJ(emit));
- ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_1, REG_LOCAL_1, offsetof(mp_obj_fun_bc_t, const_table) / sizeof(uintptr_t));
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_1, REG_LOCAL_1, OFFSETOF_OBJ_FUN_BC_CONTEXT);
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_LOCAL_1, REG_LOCAL_1, OFFSETOF_MODULE_CONTEXT_OBJ_TABLE);
ASM_LOAD_REG_REG_OFFSET(emit->as, REG_FUN_TABLE, REG_LOCAL_1, emit->scope->num_pos_args + emit->scope->num_kwonly_args);
#endif
ASM_MOV_REG_LOCAL(emit->as, REG_LOCAL_1, LOCAL_IDX_EXC_HANDLER_PC(emit));
@@ -1385,11 +1367,7 @@ STATIC void emit_native_import(emit_t *emit, qstr qst, int kind) {
STATIC void emit_native_load_const_tok(emit_t *emit, mp_token_kind_t tok) {
DEBUG_printf("load_const_tok(tok=%u)\n", tok);
if (tok == MP_TOKEN_ELLIPSIS) {
- #if MICROPY_PERSISTENT_CODE_SAVE
emit_native_load_const_obj(emit, MP_OBJ_FROM_PTR(&mp_const_ellipsis_obj));
- #else
- emit_post_push_imm(emit, VTYPE_PYOBJ, (mp_uint_t)MP_OBJ_FROM_PTR(&mp_const_ellipsis_obj));
- #endif
} else {
emit_native_pre(emit);
if (tok == MP_TOKEN_KW_NONE) {
@@ -2682,33 +2660,46 @@ STATIC void emit_native_unpack_ex(emit_t *emit, mp_uint_t n_left, mp_uint_t n_ri
STATIC void emit_native_make_function(emit_t *emit, scope_t *scope, mp_uint_t n_pos_defaults, mp_uint_t n_kw_defaults) {
// call runtime, with type info for args, or don't support dict/default params, or only support Python objects for them
emit_native_pre(emit);
+ emit_native_mov_reg_state(emit, REG_ARG_2, LOCAL_IDX_FUN_OBJ(emit));
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_ARG_2, REG_ARG_2, OFFSETOF_OBJ_FUN_BC_CONTEXT);
if (n_pos_defaults == 0 && n_kw_defaults == 0) {
need_reg_all(emit);
- ASM_MOV_REG_IMM(emit->as, REG_ARG_2, (mp_uint_t)MP_OBJ_NULL);
- ASM_MOV_REG_IMM(emit->as, REG_ARG_3, (mp_uint_t)MP_OBJ_NULL);
+ ASM_MOV_REG_IMM(emit->as, REG_ARG_3, 0);
} else {
- vtype_kind_t vtype_def_tuple, vtype_def_dict;
- emit_pre_pop_reg_reg(emit, &vtype_def_dict, REG_ARG_3, &vtype_def_tuple, REG_ARG_2);
- assert(vtype_def_tuple == VTYPE_PYOBJ);
- assert(vtype_def_dict == VTYPE_PYOBJ);
+ emit_get_stack_pointer_to_reg_for_pop(emit, REG_ARG_3, 2);
need_reg_all(emit);
}
- emit_load_reg_with_raw_code(emit, REG_ARG_1, scope->raw_code);
+ emit_load_reg_with_child(emit, REG_ARG_1, scope->raw_code);
ASM_CALL_IND(emit->as, MP_F_MAKE_FUNCTION_FROM_RAW_CODE);
emit_post_push_reg(emit, VTYPE_PYOBJ, REG_RET);
}
STATIC void emit_native_make_closure(emit_t *emit, scope_t *scope, mp_uint_t n_closed_over, mp_uint_t n_pos_defaults, mp_uint_t n_kw_defaults) {
+ // make function
emit_native_pre(emit);
+ emit_native_mov_reg_state(emit, REG_ARG_2, LOCAL_IDX_FUN_OBJ(emit));
+ ASM_LOAD_REG_REG_OFFSET(emit->as, REG_ARG_2, REG_ARG_2, OFFSETOF_OBJ_FUN_BC_CONTEXT);
if (n_pos_defaults == 0 && n_kw_defaults == 0) {
- emit_get_stack_pointer_to_reg_for_pop(emit, REG_ARG_3, n_closed_over);
- ASM_MOV_REG_IMM(emit->as, REG_ARG_2, n_closed_over);
+ need_reg_all(emit);
+ ASM_MOV_REG_IMM(emit->as, REG_ARG_3, 0);
} else {
- emit_get_stack_pointer_to_reg_for_pop(emit, REG_ARG_3, n_closed_over + 2);
- ASM_MOV_REG_IMM(emit->as, REG_ARG_2, 0x100 | n_closed_over);
+ emit_get_stack_pointer_to_reg_for_pop(emit, REG_ARG_3, 2 + n_closed_over);
+ adjust_stack(emit, 2 + n_closed_over);
+ need_reg_all(emit);
+ }
+ emit_load_reg_with_child(emit, REG_ARG_1, scope->raw_code);
+ ASM_CALL_IND(emit->as, MP_F_MAKE_FUNCTION_FROM_RAW_CODE);
+
+ // make closure
+ #if REG_ARG_1 != REG_RET
+ ASM_MOV_REG_REG(emit->as, REG_ARG_1, REG_RET);
+ #endif
+ ASM_MOV_REG_IMM(emit->as, REG_ARG_2, n_closed_over);
+ emit_get_stack_pointer_to_reg_for_pop(emit, REG_ARG_3, n_closed_over);
+ if (n_pos_defaults != 0 || n_kw_defaults != 0) {
+ adjust_stack(emit, -2);
}
- emit_load_reg_with_raw_code(emit, REG_ARG_1, scope->raw_code);
- ASM_CALL_IND(emit->as, MP_F_MAKE_CLOSURE_FROM_RAW_CODE);
+ ASM_CALL_IND(emit->as, MP_F_NEW_CLOSURE);
emit_post_push_reg(emit, VTYPE_PYOBJ, REG_RET);
}
diff --git a/py/emitnx86.c b/py/emitnx86.c
index f0553f0682..d990ef5a9e 100644
--- a/py/emitnx86.c
+++ b/py/emitnx86.c
@@ -56,7 +56,7 @@ STATIC byte mp_f_n_args[MP_F_NUMBER_OF] = {
[MP_F_UNPACK_EX] = 3,
[MP_F_DELETE_NAME] = 1,
[MP_F_DELETE_GLOBAL] = 1,
- [MP_F_MAKE_CLOSURE_FROM_RAW_CODE] = 3,
+ [MP_F_NEW_CLOSURE] = 3,
[MP_F_ARG_CHECK_NUM_SIG] = 3,
[MP_F_SETUP_CODE_STATE] = 4,
[MP_F_SMALL_INT_FLOOR_DIVIDE] = 2,
diff --git a/py/frozenmod.c b/py/frozenmod.c
index 6cb68d1ec0..61c2f20aa1 100644
--- a/py/frozenmod.c
+++ b/py/frozenmod.c
@@ -57,7 +57,7 @@ extern const char mp_frozen_str_content[];
#include "py/emitglue.h"
-extern const mp_raw_code_t *const mp_frozen_mpy_content[];
+extern const mp_frozen_module_t *const mp_frozen_mpy_content[];
#endif // MICROPY_MODULE_FROZEN_MPY
diff --git a/py/mpconfig.h b/py/mpconfig.h
index 47779a67de..5edff69dfd 100644
--- a/py/mpconfig.h
+++ b/py/mpconfig.h
@@ -328,6 +328,14 @@
#define MICROPY_PERSISTENT_CODE (MICROPY_PERSISTENT_CODE_LOAD || MICROPY_PERSISTENT_CODE_SAVE || MICROPY_MODULE_FROZEN_MPY)
#endif
+// Whether bytecode uses a qstr_table to map internal qstr indices in the bytecode
+// to global qstr values in the runtime (behaviour when feature is enabled), or
+// just stores global qstr values directly in the bytecode. This must be enabled
+// if MICROPY_PERSISTENT_CODE is enabled.
+#ifndef MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+#define MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE (MICROPY_PERSISTENT_CODE)
+#endif
+
// Whether to emit x64 native code
#ifndef MICROPY_EMIT_X64
#define MICROPY_EMIT_X64 (0)
diff --git a/py/nativeglue.c b/py/nativeglue.c
index 30e5b40061..aed6fecddf 100644
--- a/py/nativeglue.c
+++ b/py/nativeglue.c
@@ -300,7 +300,7 @@ const mp_fun_table_t mp_fun_table = {
mp_unpack_ex,
mp_delete_name,
mp_delete_global,
- mp_make_closure_from_raw_code,
+ mp_obj_new_closure,
mp_arg_check_num_sig,
mp_setup_code_state,
mp_small_int_floor_divide,
@@ -344,4 +344,8 @@ const mp_fun_table_t mp_fun_table = {
&mp_stream_write_obj,
};
+#elif MICROPY_EMIT_NATIVE && MICROPY_DYNAMIC_COMPILER
+
+const int mp_fun_table;
+
#endif // MICROPY_EMIT_NATIVE
diff --git a/py/nativeglue.h b/py/nativeglue.h
index 9d9a97b9e7..49ce665f2d 100644
--- a/py/nativeglue.h
+++ b/py/nativeglue.h
@@ -75,7 +75,7 @@ typedef enum {
MP_F_UNPACK_EX,
MP_F_DELETE_NAME,
MP_F_DELETE_GLOBAL,
- MP_F_MAKE_CLOSURE_FROM_RAW_CODE,
+ MP_F_NEW_CLOSURE,
MP_F_ARG_CHECK_NUM_SIG,
MP_F_SETUP_CODE_STATE,
MP_F_SMALL_INT_FLOOR_DIVIDE,
@@ -112,7 +112,7 @@ typedef struct _mp_fun_table_t {
void (*set_store)(mp_obj_t self_in, mp_obj_t item);
mp_obj_t (*list_append)(mp_obj_t self_in, mp_obj_t arg);
mp_obj_t (*dict_store)(mp_obj_t self_in, mp_obj_t key, mp_obj_t value);
- mp_obj_t (*make_function_from_raw_code)(const mp_raw_code_t *rc, mp_obj_t def_args, mp_obj_t def_kw_args);
+ mp_obj_t (*make_function_from_raw_code)(const mp_raw_code_t *rc, const mp_module_context_t *cm, const mp_obj_t *def_args);
mp_obj_t (*call_function_n_kw)(mp_obj_t fun_in, size_t n_args_kw, const mp_obj_t *args);
mp_obj_t (*call_method_n_kw)(size_t n_args, size_t n_kw, const mp_obj_t *args);
mp_obj_t (*call_method_n_kw_var)(bool have_self, size_t n_args_n_kw, const mp_obj_t *args);
@@ -129,7 +129,7 @@ typedef struct _mp_fun_table_t {
void (*unpack_ex)(mp_obj_t seq, size_t num, mp_obj_t *items);
void (*delete_name)(qstr qst);
void (*delete_global)(qstr qst);
- mp_obj_t (*make_closure_from_raw_code)(const mp_raw_code_t *rc, mp_uint_t n_closed_over, const mp_obj_t *args);
+ mp_obj_t (*new_closure)(mp_obj_t fun, size_t n_closed_over, const mp_obj_t *closed);
void (*arg_check_num_sig)(size_t n_args, size_t n_kw, uint32_t sig);
void (*setup_code_state)(mp_code_state_t *code_state, size_t n_args, size_t n_kw, const mp_obj_t *args);
mp_int_t (*small_int_floor_divide)(mp_int_t num, mp_int_t denom);
@@ -172,6 +172,12 @@ typedef struct _mp_fun_table_t {
const mp_obj_fun_builtin_var_t *stream_write_obj;
} mp_fun_table_t;
+#if (MICROPY_EMIT_NATIVE && !MICROPY_DYNAMIC_COMPILER) || MICROPY_ENABLE_DYNRUNTIME
extern const mp_fun_table_t mp_fun_table;
+#elif MICROPY_EMIT_NATIVE && MICROPY_DYNAMIC_COMPILER
+// In dynamic-compiler mode eliminate dependency on entries in mp_fun_table.
+// This only needs to be an independent pointer, content doesn't matter.
+extern const int mp_fun_table;
+#endif
#endif // MICROPY_INCLUDED_PY_NATIVEGLUE_H
diff --git a/py/obj.h b/py/obj.h
index b52ee0e2f4..08a35ee6f8 100644
--- a/py/obj.h
+++ b/py/obj.h
@@ -781,9 +781,6 @@ mp_obj_t mp_obj_new_exception_msg_varg(const mp_obj_type_t *exc_type, mp_rom_err
#ifdef va_start
mp_obj_t mp_obj_new_exception_msg_vlist(const mp_obj_type_t *exc_type, mp_rom_error_text_t fmt, va_list arg); // same fmt restrictions as above
#endif
-mp_obj_t mp_obj_new_fun_bc(mp_obj_t def_args, mp_obj_t def_kw_args, const byte *code, const mp_uint_t *const_table);
-mp_obj_t mp_obj_new_fun_native(mp_obj_t def_args_in, mp_obj_t def_kw_args, const void *fun_data, const mp_uint_t *const_table);
-mp_obj_t mp_obj_new_fun_asm(size_t n_args, const void *fun_data, mp_uint_t type_sig);
mp_obj_t mp_obj_new_gen_wrap(mp_obj_t fun);
mp_obj_t mp_obj_new_closure(mp_obj_t fun, size_t n_closed, const mp_obj_t *closed);
mp_obj_t mp_obj_new_tuple(size_t n, const mp_obj_t *items);
@@ -992,7 +989,6 @@ typedef struct _mp_obj_fun_builtin_var_t {
} mp_obj_fun_builtin_var_t;
qstr mp_obj_fun_get_name(mp_const_obj_t fun);
-qstr mp_obj_code_get_name(const byte *code_info);
mp_obj_t mp_identity(mp_obj_t self);
MP_DECLARE_CONST_FUN_OBJ_1(mp_identity_obj);
diff --git a/py/objfun.c b/py/objfun.c
index 85f531c88b..3542cc0e3f 100644
--- a/py/objfun.c
+++ b/py/objfun.c
@@ -143,13 +143,13 @@ const mp_obj_type_t mp_type_fun_builtin_var = {
/******************************************************************************/
/* byte code functions */
-qstr mp_obj_code_get_name(const byte *code_info) {
+STATIC qstr mp_obj_code_get_name(const mp_obj_fun_bc_t *fun, const byte *code_info) {
MP_BC_PRELUDE_SIZE_DECODE(code_info);
- #if MICROPY_PERSISTENT_CODE
- return code_info[0] | (code_info[1] << 8);
- #else
- return mp_decode_uint_value(code_info);
+ mp_uint_t name = mp_decode_uint_value(code_info);
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ name = fun->context->constants.qstr_table[name];
#endif
+ return name;
}
#if MICROPY_EMIT_NATIVE
@@ -167,7 +167,7 @@ qstr mp_obj_fun_get_name(mp_const_obj_t fun_in) {
const byte *bc = fun->bytecode;
MP_BC_PRELUDE_SIG_DECODE(bc);
- return mp_obj_code_get_name(bc);
+ return mp_obj_code_get_name(fun, bc);
}
#if MICROPY_CPYTHON_COMPAT
@@ -209,7 +209,7 @@ STATIC void dump_args(const mp_obj_t *a, size_t sz) {
#define INIT_CODESTATE(code_state, _fun_bc, _n_state, n_args, n_kw, args) \
code_state->fun_bc = _fun_bc; \
- code_state->ip = 0; \
+ code_state->ip = _fun_bc->bytecode; \
code_state->n_state = _n_state; \
mp_setup_code_state(code_state, n_args, n_kw, args); \
code_state->old_globals = mp_globals_get();
@@ -240,7 +240,7 @@ mp_code_state_t *mp_obj_fun_bc_prepare_codestate(mp_obj_t self_in, size_t n_args
INIT_CODESTATE(code_state, self, n_state, n_args, n_kw, args);
// execute the byte code with the correct globals context
- mp_globals_set(self->globals);
+ mp_globals_set(self->context->module.globals);
return code_state;
}
@@ -285,7 +285,7 @@ STATIC mp_obj_t fun_bc_call(mp_obj_t self_in, size_t n_args, size_t n_kw, const
INIT_CODESTATE(code_state, self, n_state, n_args, n_kw, args);
// execute the byte code with the correct globals context
- mp_globals_set(self->globals);
+ mp_globals_set(self->context->module.globals);
mp_vm_return_kind_t vm_return_kind = mp_execute_bytecode(code_state, MP_OBJ_NULL);
mp_globals_set(code_state->old_globals);
@@ -358,7 +358,7 @@ void mp_obj_fun_bc_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
}
if (attr == MP_QSTR___globals__) {
mp_obj_fun_bc_t *self = MP_OBJ_TO_PTR(self_in);
- dest[0] = MP_OBJ_FROM_PTR(self->globals);
+ dest[0] = MP_OBJ_FROM_PTR(self->context->module.globals);
}
}
#endif
@@ -377,25 +377,29 @@ const mp_obj_type_t mp_type_fun_bc = {
#endif
};
-mp_obj_t mp_obj_new_fun_bc(mp_obj_t def_args_in, mp_obj_t def_kw_args, const byte *code, const mp_uint_t *const_table) {
+mp_obj_t mp_obj_new_fun_bc(const mp_obj_t *def_args, const byte *code, const mp_module_context_t *context, struct _mp_raw_code_t *const *child_table) {
size_t n_def_args = 0;
size_t n_extra_args = 0;
- mp_obj_tuple_t *def_args = MP_OBJ_TO_PTR(def_args_in);
- if (def_args_in != MP_OBJ_NULL) {
- assert(mp_obj_is_type(def_args_in, &mp_type_tuple));
- n_def_args = def_args->len;
- n_extra_args = def_args->len;
+ mp_obj_tuple_t *def_pos_args = NULL;
+ mp_obj_t def_kw_args = MP_OBJ_NULL;
+ if (def_args != NULL && def_args[0] != MP_OBJ_NULL) {
+ assert(mp_obj_is_type(def_args[0], &mp_type_tuple));
+ def_pos_args = MP_OBJ_TO_PTR(def_args[0]);
+ n_def_args = def_pos_args->len;
+ n_extra_args = def_pos_args->len;
}
- if (def_kw_args != MP_OBJ_NULL) {
+ if (def_args != NULL && def_args[1] != MP_OBJ_NULL) {
+ assert(mp_obj_is_type(def_args[1], &mp_type_dict));
+ def_kw_args = def_args[1];
n_extra_args += 1;
}
mp_obj_fun_bc_t *o = m_new_obj_var(mp_obj_fun_bc_t, mp_obj_t, n_extra_args);
o->base.type = &mp_type_fun_bc;
- o->globals = mp_globals_get();
o->bytecode = code;
- o->const_table = const_table;
- if (def_args != NULL) {
- memcpy(o->extra_args, def_args->items, n_def_args * sizeof(mp_obj_t));
+ o->context = context;
+ o->child_table = child_table;
+ if (def_pos_args != NULL) {
+ memcpy(o->extra_args, def_pos_args->items, n_def_args * sizeof(mp_obj_t));
}
if (def_kw_args != MP_OBJ_NULL) {
o->extra_args[n_def_args] = def_kw_args;
@@ -423,8 +427,8 @@ STATIC const mp_obj_type_t mp_type_fun_native = {
.unary_op = mp_generic_unary_op,
};
-mp_obj_t mp_obj_new_fun_native(mp_obj_t def_args_in, mp_obj_t def_kw_args, const void *fun_data, const mp_uint_t *const_table) {
- mp_obj_fun_bc_t *o = MP_OBJ_TO_PTR(mp_obj_new_fun_bc(def_args_in, def_kw_args, (const byte *)fun_data, const_table));
+mp_obj_t mp_obj_new_fun_native(const mp_obj_t *def_args, const void *fun_data, const mp_module_context_t *mc, struct _mp_raw_code_t *const *child_table) {
+ mp_obj_fun_bc_t *o = MP_OBJ_TO_PTR(mp_obj_new_fun_bc(def_args, (const byte *)fun_data, mc, child_table));
o->base.type = &mp_type_fun_native;
return MP_OBJ_FROM_PTR(o);
}
diff --git a/py/objfun.h b/py/objfun.h
index 771bf31a95..9de15b8841 100644
--- a/py/objfun.h
+++ b/py/objfun.h
@@ -26,13 +26,14 @@
#ifndef MICROPY_INCLUDED_PY_OBJFUN_H
#define MICROPY_INCLUDED_PY_OBJFUN_H
+#include "py/bc.h"
#include "py/obj.h"
typedef struct _mp_obj_fun_bc_t {
mp_obj_base_t base;
- mp_obj_dict_t *globals; // the context within which this function was defined
- const byte *bytecode; // bytecode for the function
- const mp_uint_t *const_table; // constant table
+ const mp_module_context_t *context; // context within which this function was defined
+ struct _mp_raw_code_t *const *child_table; // table of children
+ const byte *bytecode; // bytecode for the function
#if MICROPY_PY_SYS_SETTRACE
const struct _mp_raw_code_t *rc;
#endif
@@ -42,6 +43,9 @@ typedef struct _mp_obj_fun_bc_t {
mp_obj_t extra_args[];
} mp_obj_fun_bc_t;
+mp_obj_t mp_obj_new_fun_bc(const mp_obj_t *def_args, const byte *code, const mp_module_context_t *cm, struct _mp_raw_code_t *const *raw_code_table);
+mp_obj_t mp_obj_new_fun_native(const mp_obj_t *def_args, const void *fun_data, const mp_module_context_t *cm, struct _mp_raw_code_t *const *raw_code_table);
+mp_obj_t mp_obj_new_fun_asm(size_t n_args, const void *fun_data, mp_uint_t type_sig);
void mp_obj_fun_bc_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest);
#endif // MICROPY_INCLUDED_PY_OBJFUN_H
diff --git a/py/objgenerator.c b/py/objgenerator.c
index 784310092e..cbe79e66a5 100644
--- a/py/objgenerator.c
+++ b/py/objgenerator.c
@@ -65,7 +65,7 @@ STATIC mp_obj_t gen_wrap_call(mp_obj_t self_in, size_t n_args, size_t n_kw, cons
o->pend_exc = mp_const_none;
o->code_state.fun_bc = self_fun;
- o->code_state.ip = 0;
+ o->code_state.ip = self_fun->bytecode;
o->code_state.n_state = n_state;
mp_setup_code_state(&o->code_state, n_args, n_kw, args);
return MP_OBJ_FROM_PTR(o);
@@ -91,14 +91,18 @@ STATIC mp_obj_t native_gen_wrap_call(mp_obj_t self_in, size_t n_args, size_t n_k
// The state for a native generating function is held in the same struct as a bytecode function
mp_obj_fun_bc_t *self_fun = MP_OBJ_TO_PTR(self_in);
- // Determine start of prelude, and extract n_state from it
+ // Determine start of prelude.
uintptr_t prelude_offset = ((uintptr_t *)self_fun->bytecode)[0];
#if MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ
// Prelude is in bytes object in const_table, at index prelude_offset
- mp_obj_str_t *prelude_bytes = MP_OBJ_TO_PTR(self_fun->const_table[prelude_offset]);
- prelude_offset = (const byte *)prelude_bytes->data - self_fun->bytecode;
+ mp_obj_str_t *prelude_bytes = MP_OBJ_TO_PTR(self_fun->context->constants.obj_table[prelude_offset]);
+ const uint8_t *prelude_ptr = prelude_bytes->data;
+ #else
+ const uint8_t *prelude_ptr = self_fun->bytecode + prelude_offset;
#endif
- const uint8_t *ip = self_fun->bytecode + prelude_offset;
+
+ // Extract n_state from the prelude.
+ const uint8_t *ip = prelude_ptr;
size_t n_state, n_exc_stack_unused, scope_flags, n_pos_args, n_kwonly_args, n_def_args;
MP_BC_PRELUDE_SIG_DECODE_INTO(ip, n_state, n_exc_stack_unused, scope_flags, n_pos_args, n_kwonly_args, n_def_args);
size_t n_exc_stack = 0;
@@ -111,7 +115,7 @@ STATIC mp_obj_t native_gen_wrap_call(mp_obj_t self_in, size_t n_args, size_t n_k
// Parse the input arguments and set up the code state
o->pend_exc = mp_const_none;
o->code_state.fun_bc = self_fun;
- o->code_state.ip = (const byte *)prelude_offset;
+ o->code_state.ip = prelude_ptr;
o->code_state.n_state = n_state;
mp_setup_code_state(&o->code_state, n_args, n_kw, args);
@@ -184,7 +188,7 @@ mp_vm_return_kind_t mp_obj_gen_resume(mp_obj_t self_in, mp_obj_t send_value, mp_
// Set up the correct globals context for the generator and execute it
self->code_state.old_globals = mp_globals_get();
- mp_globals_set(self->code_state.fun_bc->globals);
+ mp_globals_set(self->code_state.fun_bc->context->module.globals);
mp_vm_return_kind_t ret_kind;
diff --git a/py/objmodule.c b/py/objmodule.c
index 9be4bad92c..4b04f7ca9c 100644
--- a/py/objmodule.c
+++ b/py/objmodule.c
@@ -112,6 +112,7 @@ const mp_obj_type_t mp_type_module = {
.attr = module_attr,
};
+#include "py/bc.h"
mp_obj_t mp_obj_new_module(qstr module_name) {
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map;
mp_map_elem_t *el = mp_map_lookup(mp_loaded_modules_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND);
@@ -122,12 +123,12 @@ mp_obj_t mp_obj_new_module(qstr module_name) {
}
// create new module object
- mp_obj_module_t *o = m_new_obj(mp_obj_module_t);
- o->base.type = &mp_type_module;
- o->globals = MP_OBJ_TO_PTR(mp_obj_new_dict(MICROPY_MODULE_DICT_SIZE));
+ mp_module_context_t *o = m_new_obj(mp_module_context_t);
+ o->module.base.type = &mp_type_module;
+ o->module.globals = MP_OBJ_TO_PTR(mp_obj_new_dict(MICROPY_MODULE_DICT_SIZE));
// store __name__ entry in the module
- mp_obj_dict_store(MP_OBJ_FROM_PTR(o->globals), MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(module_name));
+ mp_obj_dict_store(MP_OBJ_FROM_PTR(o->module.globals), MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(module_name));
// store the new module into the slot in the global dict holding all modules
el->value = MP_OBJ_FROM_PTR(o);
diff --git a/py/persistentcode.c b/py/persistentcode.c
index ac523990c1..b473f18308 100644
--- a/py/persistentcode.c
+++ b/py/persistentcode.c
@@ -63,57 +63,6 @@ STATIC int mp_small_int_bits(void) {
}
#endif
-#define QSTR_WINDOW_SIZE (32)
-
-typedef struct _qstr_window_t {
- uint16_t idx; // indexes the head of the window
- uint16_t window[QSTR_WINDOW_SIZE];
-} qstr_window_t;
-
-// Push a qstr to the head of the window, and the tail qstr is overwritten
-STATIC void qstr_window_push(qstr_window_t *qw, qstr qst) {
- qw->idx = (qw->idx + 1) % QSTR_WINDOW_SIZE;
- qw->window[qw->idx] = qst;
-}
-
-// Pull an existing qstr from within the window to the head of the window
-STATIC qstr qstr_window_pull(qstr_window_t *qw, size_t idx) {
- qstr qst = qw->window[idx];
- if (idx > qw->idx) {
- memmove(&qw->window[idx], &qw->window[idx + 1], (QSTR_WINDOW_SIZE - idx - 1) * sizeof(uint16_t));
- qw->window[QSTR_WINDOW_SIZE - 1] = qw->window[0];
- idx = 0;
- }
- memmove(&qw->window[idx], &qw->window[idx + 1], (qw->idx - idx) * sizeof(uint16_t));
- qw->window[qw->idx] = qst;
- return qst;
-}
-
-#if MICROPY_PERSISTENT_CODE_LOAD
-
-// Access a qstr at the given index, relative to the head of the window (0=head)
-STATIC qstr qstr_window_access(qstr_window_t *qw, size_t idx) {
- return qstr_window_pull(qw, (qw->idx + QSTR_WINDOW_SIZE - idx) % QSTR_WINDOW_SIZE);
-}
-
-#endif
-
-#if MICROPY_PERSISTENT_CODE_SAVE
-
-// Insert a qstr at the head of the window, either by pulling an existing one or pushing a new one
-STATIC size_t qstr_window_insert(qstr_window_t *qw, qstr qst) {
- for (size_t idx = 0; idx < QSTR_WINDOW_SIZE; ++idx) {
- if (qw->window[idx] == qst) {
- qstr_window_pull(qw, idx);
- return (qw->idx + QSTR_WINDOW_SIZE - idx) % QSTR_WINDOW_SIZE;
- }
- }
- qstr_window_push(qw, qst);
- return QSTR_WINDOW_SIZE;
-}
-
-#endif
-
typedef struct _bytecode_prelude_t {
uint n_state;
uint n_exc_stack;
@@ -124,23 +73,6 @@ typedef struct _bytecode_prelude_t {
uint code_info_size;
} bytecode_prelude_t;
-// ip will point to start of opcodes
-// return value will point to simple_name, source_file qstrs
-STATIC byte *extract_prelude(const byte **ip, bytecode_prelude_t *prelude) {
- MP_BC_PRELUDE_SIG_DECODE(*ip);
- prelude->n_state = n_state;
- prelude->n_exc_stack = n_exc_stack;
- prelude->scope_flags = scope_flags;
- prelude->n_pos_args = n_pos_args;
- prelude->n_kwonly_args = n_kwonly_args;
- prelude->n_def_pos_args = n_def_pos_args;
- MP_BC_PRELUDE_SIZE_DECODE(*ip);
- byte *ip_info = (byte *)*ip;
- *ip += n_info;
- *ip += n_cell;
- return ip_info;
-}
-
#endif // MICROPY_PERSISTENT_CODE_LOAD || MICROPY_PERSISTENT_CODE_SAVE
#if MICROPY_PERSISTENT_CODE_LOAD
@@ -148,13 +80,14 @@ STATIC byte *extract_prelude(const byte **ip, bytecode_prelude_t *prelude) {
#include "py/parsenum.h"
STATIC int read_byte(mp_reader_t *reader);
-STATIC size_t read_uint(mp_reader_t *reader, byte **out);
+STATIC size_t read_uint(mp_reader_t *reader);
#if MICROPY_EMIT_MACHINE_CODE
typedef struct _reloc_info_t {
mp_reader_t *reader;
- mp_uint_t *const_table;
+ uint8_t *rodata;
+ uint8_t *bss;
} reloc_info_t;
#if MICROPY_EMIT_THUMB
@@ -197,13 +130,13 @@ void mp_native_relocate(void *ri_in, uint8_t *text, uintptr_t reloc_text) {
while ((op = read_byte(ri->reader)) != 0xff) {
if (op & 1) {
// Point to new location to make adjustments
- size_t addr = read_uint(ri->reader, NULL);
+ size_t addr = read_uint(ri->reader);
if ((addr & 1) == 0) {
// Point to somewhere in text
addr_to_adjust = &((uintptr_t *)text)[addr >> 1];
} else {
// Point to somewhere in rodata
- addr_to_adjust = &((uintptr_t *)ri->const_table[1])[addr >> 1];
+ addr_to_adjust = &((uintptr_t *)ri->rodata)[addr >> 1];
}
}
op >>= 1;
@@ -212,15 +145,18 @@ void mp_native_relocate(void *ri_in, uint8_t *text, uintptr_t reloc_text) {
if (op <= 5) {
if (op & 1) {
// Read in number of adjustments to make
- n = read_uint(ri->reader, NULL);
+ n = read_uint(ri->reader);
}
op >>= 1;
if (op == 0) {
// Destination is text
dest = reloc_text;
+ } else if (op == 1) {
+ // Destination is rodata
+ dest = (uintptr_t)ri->rodata;
} else {
- // Destination is rodata (op=1) or bss (op=1 if no rodata, else op=2)
- dest = ri->const_table[op];
+ // Destination is bss
+ dest = (uintptr_t)ri->bss;
}
} else if (op == 6) {
// Destination is mp_fun_table itself
@@ -247,14 +183,10 @@ STATIC void read_bytes(mp_reader_t *reader, byte *buf, size_t len) {
}
}
-STATIC size_t read_uint(mp_reader_t *reader, byte **out) {
+STATIC size_t read_uint(mp_reader_t *reader) {
size_t unum = 0;
for (;;) {
byte b = reader->readbyte(reader->data);
- if (out != NULL) {
- **out = b;
- ++*out;
- }
unum = (unum << 7) | (b & 0x7f);
if ((b & 0x80) == 0) {
break;
@@ -263,35 +195,41 @@ STATIC size_t read_uint(mp_reader_t *reader, byte **out) {
return unum;
}
-STATIC qstr load_qstr(mp_reader_t *reader, qstr_window_t *qw) {
- size_t len = read_uint(reader, NULL);
- if (len == 0) {
- // static qstr
- return read_byte(reader);
- }
+STATIC qstr load_qstr(mp_reader_t *reader) {
+ size_t len = read_uint(reader);
if (len & 1) {
- // qstr in window
- return qstr_window_access(qw, len >> 1);
+ // static qstr
+ return len >> 1;
}
len >>= 1;
char *str = m_new(char, len);
read_bytes(reader, (byte *)str, len);
+ read_byte(reader); // read and discard null terminator
qstr qst = qstr_from_strn(str, len);
m_del(char, str, len);
- qstr_window_push(qw, qst);
return qst;
}
STATIC mp_obj_t load_obj(mp_reader_t *reader) {
byte obj_type = read_byte(reader);
+ #if MICROPY_EMIT_MACHINE_CODE
+ if (obj_type == 't') {
+ return MP_OBJ_FROM_PTR(&mp_fun_table);
+ } else
+ #endif
if (obj_type == 'e') {
return MP_OBJ_FROM_PTR(&mp_const_ellipsis_obj);
} else {
- size_t len = read_uint(reader, NULL);
+ size_t len = read_uint(reader);
+ if (len == 0 && obj_type == 'b') {
+ read_byte(reader); // skip null terminator
+ return mp_const_empty_bytes;
+ }
vstr_t vstr;
vstr_init_len(&vstr, len);
read_bytes(reader, (byte *)vstr.buf, len);
if (obj_type == 's' || obj_type == 'b') {
+ read_byte(reader); // skip null terminator
return mp_obj_new_str_from_vstr(obj_type == 's' ? &mp_type_str : &mp_type_bytes, &vstr);
} else if (obj_type == 'i') {
return mp_parse_num_integer(vstr.buf, vstr.len, 10, NULL);
@@ -302,58 +240,12 @@ STATIC mp_obj_t load_obj(mp_reader_t *reader) {
}
}
-STATIC void load_prelude_qstrs(mp_reader_t *reader, qstr_window_t *qw, byte *ip) {
- qstr simple_name = load_qstr(reader, qw);
- ip[0] = simple_name;
- ip[1] = simple_name >> 8;
- qstr source_file = load_qstr(reader, qw);
- ip[2] = source_file;
- ip[3] = source_file >> 8;
-}
-
-STATIC void load_prelude(mp_reader_t *reader, qstr_window_t *qw, byte **ip, bytecode_prelude_t *prelude) {
- // Read in the prelude header
- byte *ip_read = *ip;
- read_uint(reader, &ip_read); // read in n_state/etc (is effectively a var-uint)
- read_uint(reader, &ip_read); // read in n_info/n_cell (is effectively a var-uint)
-
- // Prelude header has been read into *ip, now decode and extract values from it
- extract_prelude((const byte **)ip, prelude);
-
- // Load qstrs in prelude
- load_prelude_qstrs(reader, qw, ip_read);
- ip_read += 4;
-
- // Read remaining code info
- read_bytes(reader, ip_read, *ip - ip_read);
-}
-
-STATIC void load_bytecode(mp_reader_t *reader, qstr_window_t *qw, byte *ip, byte *ip_top) {
- while (ip < ip_top) {
- *ip = read_byte(reader);
- size_t sz;
- uint f = mp_opcode_format(ip, &sz, false);
- ++ip;
- --sz;
- if (f == MP_BC_FORMAT_QSTR) {
- qstr qst = load_qstr(reader, qw);
- *ip++ = qst;
- *ip++ = qst >> 8;
- sz -= 2;
- } else if (f == MP_BC_FORMAT_VAR_UINT) {
- while ((*ip++ = read_byte(reader)) & 0x80) {
- }
- }
- read_bytes(reader, ip, sz);
- ip += sz;
- }
-}
-
-STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader, qstr_window_t *qw) {
+STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader) {
// Load function kind and data length
- size_t kind_len = read_uint(reader, NULL);
+ size_t kind_len = read_uint(reader);
int kind = (kind_len & 3) + MP_CODE_BYTECODE;
- size_t fun_data_len = kind_len >> 2;
+ bool has_children = !!(kind_len & 4);
+ size_t fun_data_len = kind_len >> 3;
#if !MICROPY_EMIT_MACHINE_CODE
if (kind != MP_CODE_BYTECODE) {
@@ -362,23 +254,18 @@ STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader, qstr_window_t *qw) {
#endif
uint8_t *fun_data = NULL;
- bytecode_prelude_t prelude = {0};
#if MICROPY_EMIT_MACHINE_CODE
size_t prelude_offset = 0;
- mp_uint_t type_sig = 0;
- size_t n_qstr_link = 0;
+ mp_uint_t native_scope_flags = 0;
+ mp_uint_t native_n_pos_args = 0;
+ mp_uint_t native_type_sig = 0;
#endif
if (kind == MP_CODE_BYTECODE) {
// Allocate memory for the bytecode
fun_data = m_new(uint8_t, fun_data_len);
-
- // Load prelude
- byte *ip = fun_data;
- load_prelude(reader, qw, &ip, &prelude);
-
// Load bytecode
- load_bytecode(reader, qw, ip, fun_data + fun_data_len);
+ read_bytes(reader, fun_data, fun_data_len);
#if MICROPY_EMIT_MACHINE_CODE
} else {
@@ -389,10 +276,10 @@ STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader, qstr_window_t *qw) {
if (kind == MP_CODE_NATIVE_PY || kind == MP_CODE_NATIVE_VIPER) {
// Parse qstr link table and link native code
- n_qstr_link = read_uint(reader, NULL);
+ size_t n_qstr_link = read_uint(reader);
for (size_t i = 0; i < n_qstr_link; ++i) {
- size_t off = read_uint(reader, NULL);
- qstr qst = load_qstr(reader, qw);
+ size_t off = read_uint(reader);
+ qstr qst = load_qstr(reader);
uint8_t *dest = fun_data + (off >> 2);
if ((off & 3) == 0) {
// Generic 16-bit link
@@ -409,113 +296,92 @@ STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader, qstr_window_t *qw) {
}
if (kind == MP_CODE_NATIVE_PY) {
- // Extract prelude for later use
- prelude_offset = read_uint(reader, NULL);
+ // Read prelude offset within fun_data, and extract scope flags.
+ prelude_offset = read_uint(reader);
const byte *ip = fun_data + prelude_offset;
- byte *ip_info = extract_prelude(&ip, &prelude);
- // Load qstrs in prelude
- load_prelude_qstrs(reader, qw, ip_info);
+ MP_BC_PRELUDE_SIG_DECODE(ip);
+ native_scope_flags = scope_flags;
} else {
- // Load basic scope info for viper and asm
- prelude.scope_flags = read_uint(reader, NULL);
- prelude.n_pos_args = 0;
- prelude.n_kwonly_args = 0;
+ // Load basic scope info for viper and asm.
+ native_scope_flags = read_uint(reader);
if (kind == MP_CODE_NATIVE_ASM) {
- prelude.n_pos_args = read_uint(reader, NULL);
- type_sig = read_uint(reader, NULL);
+ native_n_pos_args = read_uint(reader);
+ native_type_sig = read_uint(reader);
}
}
#endif
}
- size_t n_obj = 0;
- size_t n_raw_code = 0;
- mp_uint_t *const_table = NULL;
+ size_t n_children = 0;
+ mp_raw_code_t **children = NULL;
- if (kind != MP_CODE_NATIVE_ASM) {
- // Load constant table for bytecode, native and viper
-
- // Number of entries in constant table
- n_obj = read_uint(reader, NULL);
- n_raw_code = read_uint(reader, NULL);
-
- // Allocate constant table
- size_t n_alloc = prelude.n_pos_args + prelude.n_kwonly_args + n_obj + n_raw_code;
- #if MICROPY_EMIT_MACHINE_CODE
- if (kind != MP_CODE_BYTECODE) {
- ++n_alloc; // additional entry for mp_fun_table
- if (prelude.scope_flags & MP_SCOPE_FLAG_VIPERRODATA) {
- ++n_alloc; // additional entry for rodata
- }
- if (prelude.scope_flags & MP_SCOPE_FLAG_VIPERBSS) {
- ++n_alloc; // additional entry for BSS
- }
+ #if MICROPY_EMIT_MACHINE_CODE
+ // Load optional BSS/rodata for viper.
+ uint8_t *rodata = NULL;
+ uint8_t *bss = NULL;
+ if (kind == MP_CODE_NATIVE_VIPER) {
+ size_t rodata_size = 0;
+ if (native_scope_flags & MP_SCOPE_FLAG_VIPERRODATA) {
+ rodata_size = read_uint(reader);
}
- #endif
- const_table = m_new(mp_uint_t, n_alloc);
- mp_uint_t *ct = const_table;
-
- // Load function argument names (initial entries in const_table)
- // (viper has n_pos_args=n_kwonly_args=0 so doesn't load any qstrs here)
- for (size_t i = 0; i < prelude.n_pos_args + prelude.n_kwonly_args; ++i) {
- *ct++ = (mp_uint_t)MP_OBJ_NEW_QSTR(load_qstr(reader, qw));
+ size_t bss_size = 0;
+ if (native_scope_flags & MP_SCOPE_FLAG_VIPERBSS) {
+ bss_size = read_uint(reader);
}
- #if MICROPY_EMIT_MACHINE_CODE
- if (kind != MP_CODE_BYTECODE) {
- // Populate mp_fun_table entry
- *ct++ = (mp_uint_t)(uintptr_t)&mp_fun_table;
-
- // Allocate and load rodata if needed
- if (prelude.scope_flags & MP_SCOPE_FLAG_VIPERRODATA) {
- size_t size = read_uint(reader, NULL);
- uint8_t *rodata = m_new(uint8_t, size);
- read_bytes(reader, rodata, size);
- *ct++ = (uintptr_t)rodata;
+ if (rodata_size + bss_size != 0) {
+ bss_size = (uintptr_t)MP_ALIGN(bss_size, sizeof(uintptr_t));
+ uint8_t *data = m_new0(uint8_t, bss_size + rodata_size);
+ bss = data;
+ rodata = bss + bss_size;
+ if (native_scope_flags & MP_SCOPE_FLAG_VIPERRODATA) {
+ read_bytes(reader, rodata, rodata_size);
}
- // Allocate BSS if needed
- if (prelude.scope_flags & MP_SCOPE_FLAG_VIPERBSS) {
- size_t size = read_uint(reader, NULL);
- uint8_t *bss = m_new0(uint8_t, size);
- *ct++ = (uintptr_t)bss;
- }
+ // Viper code with BSS/rodata should not have any children.
+ // Reuse the children pointer to reference the BSS/rodata
+ // memory so that it is not reclaimed by the GC.
+ assert(!has_children);
+ children = (void *)data;
}
- #endif
+ }
+ #endif
- // Load constant objects and raw code children
- for (size_t i = 0; i < n_obj; ++i) {
- *ct++ = (mp_uint_t)load_obj(reader);
- }
- for (size_t i = 0; i < n_raw_code; ++i) {
- *ct++ = (mp_uint_t)(uintptr_t)load_raw_code(reader, qw);
+ // Load children if any.
+ if (has_children) {
+ n_children = read_uint(reader);
+ children = m_new(mp_raw_code_t *, n_children);
+ for (size_t i = 0; i < n_children; ++i) {
+ children[i] = load_raw_code(reader);
}
}
// Create raw_code and return it
mp_raw_code_t *rc = mp_emit_glue_new_raw_code();
if (kind == MP_CODE_BYTECODE) {
+ const byte *ip = fun_data;
+ MP_BC_PRELUDE_SIG_DECODE(ip);
// Assign bytecode to raw code object
mp_emit_glue_assign_bytecode(rc, fun_data,
#if MICROPY_PERSISTENT_CODE_SAVE || MICROPY_DEBUG_PRINTERS
fun_data_len,
#endif
- const_table,
+ children,
#if MICROPY_PERSISTENT_CODE_SAVE
- n_obj, n_raw_code,
+ n_children,
#endif
- prelude.scope_flags);
+ scope_flags);
#if MICROPY_EMIT_MACHINE_CODE
} else {
// Relocate and commit code to executable address space
- reloc_info_t ri = {reader, const_table};
+ reloc_info_t ri = {reader, rodata, bss};
#if defined(MP_PLAT_COMMIT_EXEC)
- void *opt_ri = (prelude.scope_flags & MP_SCOPE_FLAG_VIPERRELOC) ? &ri : NULL;
+ void *opt_ri = (native_scope_flags & MP_SCOPE_FLAG_VIPERRELOC) ? &ri : NULL;
fun_data = MP_PLAT_COMMIT_EXEC(fun_data, fun_data_len, opt_ri);
#else
- if (prelude.scope_flags & MP_SCOPE_FLAG_VIPERRELOC) {
+ if (native_scope_flags & MP_SCOPE_FLAG_VIPERRELOC) {
#if MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE
// If native code needs relocations then it's not guaranteed that a pointer to
// the head of `buf` (containing the machine code) will be retained for the GC
@@ -534,26 +400,27 @@ STATIC mp_raw_code_t *load_raw_code(mp_reader_t *reader, qstr_window_t *qw) {
// Assign native code to raw code object
mp_emit_glue_assign_native(rc, kind,
- fun_data, fun_data_len, const_table,
+ fun_data, fun_data_len,
+ children,
#if MICROPY_PERSISTENT_CODE_SAVE
+ n_children,
prelude_offset,
- n_obj, n_raw_code,
- n_qstr_link, NULL,
+ 0, NULL,
#endif
- prelude.n_pos_args, prelude.scope_flags, type_sig);
+ native_scope_flags, native_n_pos_args, native_type_sig
+ );
#endif
}
return rc;
}
-mp_raw_code_t *mp_raw_code_load(mp_reader_t *reader) {
+mp_compiled_module_t mp_raw_code_load(mp_reader_t *reader, mp_module_context_t *context) {
byte header[4];
read_bytes(reader, header, sizeof(header));
if (header[0] != 'M'
|| header[1] != MPY_VERSION
|| MPY_FEATURE_DECODE_FLAGS(header[2]) != MPY_FEATURE_FLAGS
- || header[3] > mp_small_int_bits()
- || read_uint(reader, NULL) > QSTR_WINDOW_SIZE) {
+ || header[3] > mp_small_int_bits()) {
mp_raise_ValueError(MP_ERROR_TEXT("incompatible .mpy file"));
}
if (MPY_FEATURE_DECODE_ARCH(header[2]) != MP_NATIVE_ARCH_NONE) {
@@ -562,25 +429,49 @@ mp_raw_code_t *mp_raw_code_load(mp_reader_t *reader) {
mp_raise_ValueError(MP_ERROR_TEXT("incompatible .mpy arch"));
}
}
- qstr_window_t qw;
- qw.idx = 0;
- mp_raw_code_t *rc = load_raw_code(reader, &qw);
+
+ size_t n_qstr = read_uint(reader);
+ size_t n_obj = read_uint(reader);
+ mp_module_context_alloc_tables(context, n_qstr, n_obj);
+
+ // Load qstrs.
+ for (size_t i = 0; i < n_qstr; ++i) {
+ context->constants.qstr_table[i] = load_qstr(reader);
+ }
+
+ // Load constant objects.
+ for (size_t i = 0; i < n_obj; ++i) {
+ context->constants.obj_table[i] = load_obj(reader);
+ }
+
+ // Load top-level module.
+ mp_compiled_module_t cm2;
+ cm2.rc = load_raw_code(reader);
+ cm2.context = context;
+
+ #if MICROPY_PERSISTENT_CODE_SAVE
+ cm2.has_native = MPY_FEATURE_DECODE_ARCH(header[2]) != MP_NATIVE_ARCH_NONE;
+ cm2.n_qstr = n_qstr;
+ cm2.n_obj = n_obj;
+ #endif
+
reader->close(reader->data);
- return rc;
+
+ return cm2;
}
-mp_raw_code_t *mp_raw_code_load_mem(const byte *buf, size_t len) {
+mp_compiled_module_t mp_raw_code_load_mem(const byte *buf, size_t len, mp_module_context_t *context) {
mp_reader_t reader;
mp_reader_new_mem(&reader, buf, len, 0);
- return mp_raw_code_load(&reader);
+ return mp_raw_code_load(&reader, context);
}
#if MICROPY_HAS_FILE_READER
-mp_raw_code_t *mp_raw_code_load_file(const char *filename) {
+mp_compiled_module_t mp_raw_code_load_file(const char *filename, mp_module_context_t *context) {
mp_reader_t reader;
mp_reader_new_file(&reader, filename);
- return mp_raw_code_load(&reader);
+ return mp_raw_code_load(&reader, context);
}
#endif // MICROPY_HAS_FILE_READER
@@ -607,26 +498,25 @@ STATIC void mp_print_uint(mp_print_t *print, size_t n) {
print->print_strn(print->data, (char *)p, buf + sizeof(buf) - p);
}
-STATIC void save_qstr(mp_print_t *print, qstr_window_t *qw, qstr qst) {
+STATIC void save_qstr(mp_print_t *print, qstr qst) {
if (qst <= QSTR_LAST_STATIC) {
// encode static qstr
- byte buf[2] = {0, qst & 0xff};
- mp_print_bytes(print, buf, 2);
- return;
- }
- size_t idx = qstr_window_insert(qw, qst);
- if (idx < QSTR_WINDOW_SIZE) {
- // qstr found in window, encode index to it
- mp_print_uint(print, idx << 1 | 1);
+ mp_print_uint(print, qst << 1 | 1);
return;
}
size_t len;
const byte *str = qstr_data(qst, &len);
mp_print_uint(print, len << 1);
- mp_print_bytes(print, str, len);
+ mp_print_bytes(print, str, len + 1); // +1 to store null terminator
}
STATIC void save_obj(mp_print_t *print, mp_obj_t o) {
+ #if MICROPY_EMIT_MACHINE_CODE
+ if (o == MP_OBJ_FROM_PTR(&mp_fun_table)) {
+ byte obj_type = 't';
+ mp_print_bytes(print, &obj_type, 1);
+ } else
+ #endif
if (mp_obj_is_str_or_bytes(o)) {
byte obj_type;
if (mp_obj_is_str(o)) {
@@ -638,7 +528,7 @@ STATIC void save_obj(mp_print_t *print, mp_obj_t o) {
const char *str = mp_obj_str_get_data(o, &len);
mp_print_bytes(print, &obj_type, 1);
mp_print_uint(print, len);
- mp_print_bytes(print, (const byte *)str, len);
+ mp_print_bytes(print, (const byte *)str, len + 1); // +1 to store null terminator
} else if (MP_OBJ_TO_PTR(o) == &mp_const_ellipsis_obj) {
byte obj_type = 'e';
mp_print_bytes(print, &obj_type, 1);
@@ -667,136 +557,45 @@ STATIC void save_obj(mp_print_t *print, mp_obj_t o) {
}
}
-STATIC void save_prelude_qstrs(mp_print_t *print, qstr_window_t *qw, const byte *ip) {
- save_qstr(print, qw, ip[0] | (ip[1] << 8)); // simple_name
- save_qstr(print, qw, ip[2] | (ip[3] << 8)); // source_file
-}
-
-STATIC void save_bytecode(mp_print_t *print, qstr_window_t *qw, const byte *ip, const byte *ip_top) {
- while (ip < ip_top) {
- size_t sz;
- uint f = mp_opcode_format(ip, &sz, true);
- if (f == MP_BC_FORMAT_QSTR) {
- mp_print_bytes(print, ip, 1);
- qstr qst = ip[1] | (ip[2] << 8);
- save_qstr(print, qw, qst);
- ip += 3;
- sz -= 3;
- }
- mp_print_bytes(print, ip, sz);
- ip += sz;
- }
-}
-
-STATIC void save_raw_code(mp_print_t *print, mp_raw_code_t *rc, qstr_window_t *qstr_window) {
+STATIC void save_raw_code(mp_print_t *print, const mp_raw_code_t *rc) {
// Save function kind and data length
- mp_print_uint(print, (rc->fun_data_len << 2) | (rc->kind - MP_CODE_BYTECODE));
+ mp_print_uint(print, (rc->fun_data_len << 3) | ((rc->n_children != 0) << 2) | (rc->kind - MP_CODE_BYTECODE));
- bytecode_prelude_t prelude;
+ // Save function code.
+ mp_print_bytes(print, rc->fun_data, rc->fun_data_len);
- if (rc->kind == MP_CODE_BYTECODE) {
- // Extract prelude
- const byte *ip = rc->fun_data;
- const byte *ip_info = extract_prelude(&ip, &prelude);
-
- // Save prelude
- mp_print_bytes(print, rc->fun_data, ip_info - (const byte *)rc->fun_data);
- save_prelude_qstrs(print, qstr_window, ip_info);
- ip_info += 4;
- mp_print_bytes(print, ip_info, ip - ip_info);
-
- // Save bytecode
- const byte *ip_top = (const byte *)rc->fun_data + rc->fun_data_len;
- save_bytecode(print, qstr_window, ip, ip_top);
#if MICROPY_EMIT_MACHINE_CODE
- } else {
- // Save native code
- mp_print_bytes(print, rc->fun_data, rc->fun_data_len);
-
- if (rc->kind == MP_CODE_NATIVE_PY || rc->kind == MP_CODE_NATIVE_VIPER) {
- // Save qstr link table for native code
- mp_print_uint(print, rc->n_qstr);
- for (size_t i = 0; i < rc->n_qstr; ++i) {
- mp_print_uint(print, rc->qstr_link[i].off);
- save_qstr(print, qstr_window, rc->qstr_link[i].qst);
- }
+ if (rc->kind == MP_CODE_NATIVE_PY || rc->kind == MP_CODE_NATIVE_VIPER) {
+ // Save qstr link table for native code
+ mp_print_uint(print, rc->n_qstr);
+ for (size_t i = 0; i < rc->n_qstr; ++i) {
+ mp_print_uint(print, rc->qstr_link[i].off);
+ save_qstr(print, rc->qstr_link[i].qst);
}
-
- if (rc->kind == MP_CODE_NATIVE_PY) {
- // Save prelude size
- mp_print_uint(print, rc->prelude_offset);
-
- // Extract prelude and save qstrs in prelude
- const byte *ip = (const byte *)rc->fun_data + rc->prelude_offset;
- const byte *ip_info = extract_prelude(&ip, &prelude);
- save_prelude_qstrs(print, qstr_window, ip_info);
- } else {
- // Save basic scope info for viper and asm
- mp_print_uint(print, rc->scope_flags & MP_SCOPE_FLAG_ALL_SIG);
- prelude.n_pos_args = 0;
- prelude.n_kwonly_args = 0;
- if (rc->kind == MP_CODE_NATIVE_ASM) {
- mp_print_uint(print, rc->n_pos_args);
- mp_print_uint(print, rc->type_sig);
- }
- }
- #endif
}
- if (rc->kind != MP_CODE_NATIVE_ASM) {
- // Save constant table for bytecode, native and viper
-
- // Number of entries in constant table
- mp_print_uint(print, rc->n_obj);
- mp_print_uint(print, rc->n_raw_code);
-
- const mp_uint_t *const_table = rc->const_table;
-
- // Save function argument names (initial entries in const_table)
- // (viper has n_pos_args=n_kwonly_args=0 so doesn't save any qstrs here)
- for (size_t i = 0; i < prelude.n_pos_args + prelude.n_kwonly_args; ++i) {
- mp_obj_t o = (mp_obj_t)*const_table++;
- save_qstr(print, qstr_window, MP_OBJ_QSTR_VALUE(o));
- }
-
- if (rc->kind != MP_CODE_BYTECODE) {
- // Skip saving mp_fun_table entry
- ++const_table;
- }
-
- // Save constant objects and raw code children
- for (size_t i = 0; i < rc->n_obj; ++i) {
- save_obj(print, (mp_obj_t)*const_table++);
+ if (rc->kind == MP_CODE_NATIVE_PY) {
+ // Save prelude size
+ mp_print_uint(print, rc->prelude_offset);
+ } else if (rc->kind == MP_CODE_NATIVE_VIPER || rc->kind == MP_CODE_NATIVE_ASM) {
+ // Save basic scope info for viper and asm
+ mp_print_uint(print, rc->scope_flags & MP_SCOPE_FLAG_ALL_SIG);
+ if (rc->kind == MP_CODE_NATIVE_ASM) {
+ mp_print_uint(print, rc->n_pos_args);
+ mp_print_uint(print, rc->type_sig);
}
- for (size_t i = 0; i < rc->n_raw_code; ++i) {
- save_raw_code(print, (mp_raw_code_t *)(uintptr_t)*const_table++, qstr_window);
- }
- }
-}
-
-STATIC bool mp_raw_code_has_native(mp_raw_code_t *rc) {
- if (rc->kind != MP_CODE_BYTECODE) {
- return true;
}
+ #endif
- const byte *ip = rc->fun_data;
- bytecode_prelude_t prelude;
- extract_prelude(&ip, &prelude);
-
- const mp_uint_t *const_table = rc->const_table
- + prelude.n_pos_args + prelude.n_kwonly_args
- + rc->n_obj;
-
- for (size_t i = 0; i < rc->n_raw_code; ++i) {
- if (mp_raw_code_has_native((mp_raw_code_t *)(uintptr_t)*const_table++)) {
- return true;
+ if (rc->n_children) {
+ mp_print_uint(print, rc->n_children);
+ for (size_t i = 0; i < rc->n_children; ++i) {
+ save_raw_code(print, rc->children[i]);
}
}
-
- return false;
}
-void mp_raw_code_save(mp_raw_code_t *rc, mp_print_t *print) {
+void mp_raw_code_save(mp_compiled_module_t *cm, mp_print_t *print) {
// header contains:
// byte 'M'
// byte version
@@ -813,16 +612,27 @@ void mp_raw_code_save(mp_raw_code_t *rc, mp_print_t *print) {
mp_small_int_bits(),
#endif
};
- if (mp_raw_code_has_native(rc)) {
+ if (cm->has_native) {
header[2] |= MPY_FEATURE_ENCODE_ARCH(MPY_FEATURE_ARCH_DYNAMIC);
}
mp_print_bytes(print, header, sizeof(header));
- mp_print_uint(print, QSTR_WINDOW_SIZE);
- qstr_window_t qw;
- qw.idx = 0;
- memset(qw.window, 0, sizeof(qw.window));
- save_raw_code(print, rc, &qw);
+ // Number of entries in constant table.
+ mp_print_uint(print, cm->n_qstr);
+ mp_print_uint(print, cm->n_obj);
+
+ // Save qstrs.
+ for (size_t i = 0; i < cm->n_qstr; ++i) {
+ save_qstr(print, cm->context->constants.qstr_table[i]);
+ }
+
+ // Save constant objects.
+ for (size_t i = 0; i < cm->n_obj; ++i) {
+ save_obj(print, (mp_obj_t)cm->context->constants.obj_table[i]);
+ }
+
+ // Save outer raw code, which will save all its child raw codes.
+ save_raw_code(print, cm->rc);
}
#if MICROPY_PERSISTENT_CODE_SAVE_FILE
@@ -839,12 +649,12 @@ STATIC void fd_print_strn(void *env, const char *str, size_t len) {
(void)ret;
}
-void mp_raw_code_save_file(mp_raw_code_t *rc, const char *filename) {
+void mp_raw_code_save_file(mp_compiled_module_t *cm, const char *filename) {
MP_THREAD_GIL_EXIT();
int fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC, 0644);
MP_THREAD_GIL_ENTER();
mp_print_t fd_print = {(void *)(intptr_t)fd, fd_print_strn};
- mp_raw_code_save(rc, &fd_print);
+ mp_raw_code_save(cm, &fd_print);
MP_THREAD_GIL_EXIT();
close(fd);
MP_THREAD_GIL_ENTER();
diff --git a/py/persistentcode.h b/py/persistentcode.h
index 94fc3bf2d3..1991ba26ff 100644
--- a/py/persistentcode.h
+++ b/py/persistentcode.h
@@ -31,7 +31,7 @@
#include "py/emitglue.h"
// The current version of .mpy files
-#define MPY_VERSION 5
+#define MPY_VERSION 6
// Macros to encode/decode flags to/from the feature byte
#define MPY_FEATURE_ENCODE_FLAGS(flags) (flags)
@@ -102,12 +102,12 @@ enum {
MP_NATIVE_ARCH_XTENSAWIN,
};
-mp_raw_code_t *mp_raw_code_load(mp_reader_t *reader);
-mp_raw_code_t *mp_raw_code_load_mem(const byte *buf, size_t len);
-mp_raw_code_t *mp_raw_code_load_file(const char *filename);
+mp_compiled_module_t mp_raw_code_load(mp_reader_t *reader, mp_module_context_t *ctx);
+mp_compiled_module_t mp_raw_code_load_mem(const byte *buf, size_t len, mp_module_context_t *ctx);
+mp_compiled_module_t mp_raw_code_load_file(const char *filename, mp_module_context_t *ctx);
-void mp_raw_code_save(mp_raw_code_t *rc, mp_print_t *print);
-void mp_raw_code_save_file(mp_raw_code_t *rc, const char *filename);
+void mp_raw_code_save(mp_compiled_module_t *cm, mp_print_t *print);
+void mp_raw_code_save_file(mp_compiled_module_t *cm, const char *filename);
void mp_native_relocate(void *reloc, uint8_t *text, uintptr_t reloc_text);
diff --git a/py/profile.c b/py/profile.c
index d0ac99e074..4e23e9eac4 100644
--- a/py/profile.c
+++ b/py/profile.c
@@ -27,14 +27,16 @@
#include "py/profile.h"
#include "py/bc0.h"
#include "py/gc.h"
+#include "py/objfun.h"
#if MICROPY_PY_SYS_SETTRACE
#define prof_trace_cb MP_STATE_THREAD(prof_trace_callback)
+#define QSTR_MAP(context, idx) (context->constants.qstr_table[idx])
STATIC uint mp_prof_bytecode_lineno(const mp_raw_code_t *rc, size_t bc) {
const mp_bytecode_prelude_t *prelude = &rc->prelude;
- return mp_bytecode_get_source_line(prelude->line_info, bc);
+ return mp_bytecode_get_source_line(prelude->line_info, prelude->line_info_top, bc);
}
void mp_prof_extract_prelude(const byte *bytecode, mp_bytecode_prelude_t *prelude) {
@@ -50,13 +52,14 @@ void mp_prof_extract_prelude(const byte *bytecode, mp_bytecode_prelude_t *prelud
MP_BC_PRELUDE_SIZE_DECODE(ip);
- prelude->line_info = ip + 4;
+ prelude->line_info_top = ip + n_info;
prelude->opcodes = ip + n_info + n_cell;
- qstr block_name = ip[0] | (ip[1] << 8);
- qstr source_file = ip[2] | (ip[3] << 8);
- prelude->qstr_block_name = block_name;
- prelude->qstr_source_file = source_file;
+ prelude->qstr_block_name_idx = mp_decode_uint_value(ip);
+ for (size_t i = 0; i < 1 + n_pos_args + n_kwonly_args; ++i) {
+ ip = mp_decode_uint_skip(ip);
+ }
+ prelude->line_info = ip;
}
/******************************************************************************/
@@ -69,22 +72,19 @@ STATIC void code_print(const mp_print_t *print, mp_obj_t o_in, mp_print_kind_t k
const mp_bytecode_prelude_t *prelude = &rc->prelude;
mp_printf(print,
"<code object %q at 0x%p, file \"%q\", line %d>",
- prelude->qstr_block_name,
+ QSTR_MAP(o->context, prelude->qstr_block_name_idx),
o,
- prelude->qstr_source_file,
+ QSTR_MAP(o->context, 0),
rc->line_of_definition
);
}
-STATIC mp_obj_tuple_t *code_consts(const mp_raw_code_t *rc) {
- const mp_bytecode_prelude_t *prelude = &rc->prelude;
- int start = prelude->n_pos_args + prelude->n_kwonly_args + rc->n_obj;
- int stop = prelude->n_pos_args + prelude->n_kwonly_args + rc->n_obj + rc->n_raw_code;
- mp_obj_tuple_t *consts = MP_OBJ_TO_PTR(mp_obj_new_tuple(stop - start + 1, NULL));
+STATIC mp_obj_tuple_t *code_consts(const mp_module_context_t *context, const mp_raw_code_t *rc) {
+ mp_obj_tuple_t *consts = MP_OBJ_TO_PTR(mp_obj_new_tuple(rc->n_children + 1, NULL));
size_t const_no = 0;
- for (int i = start; i < stop; ++i) {
- mp_obj_t code = mp_obj_new_code((const mp_raw_code_t *)MP_OBJ_TO_PTR(rc->const_table[i]));
+ for (size_t i = 0; i < rc->n_children; ++i) {
+ mp_obj_t code = mp_obj_new_code(context, rc->children[i]);
if (code == MP_OBJ_NULL) {
m_malloc_fail(sizeof(mp_obj_code_t));
}
@@ -149,16 +149,16 @@ STATIC void code_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
);
break;
case MP_QSTR_co_consts:
- dest[0] = MP_OBJ_FROM_PTR(code_consts(rc));
+ dest[0] = MP_OBJ_FROM_PTR(code_consts(o->context, rc));
break;
case MP_QSTR_co_filename:
- dest[0] = MP_OBJ_NEW_QSTR(prelude->qstr_source_file);
+ dest[0] = MP_OBJ_NEW_QSTR(QSTR_MAP(o->context, 0));
break;
case MP_QSTR_co_firstlineno:
dest[0] = MP_OBJ_NEW_SMALL_INT(mp_prof_bytecode_lineno(rc, 0));
break;
case MP_QSTR_co_name:
- dest[0] = MP_OBJ_NEW_QSTR(prelude->qstr_block_name);
+ dest[0] = MP_OBJ_NEW_QSTR(QSTR_MAP(o->context, prelude->qstr_block_name_idx));
break;
case MP_QSTR_co_names:
dest[0] = MP_OBJ_FROM_PTR(o->dict_locals);
@@ -180,12 +180,13 @@ const mp_obj_type_t mp_type_settrace_codeobj = {
.attr = code_attr,
};
-mp_obj_t mp_obj_new_code(const mp_raw_code_t *rc) {
+mp_obj_t mp_obj_new_code(const mp_module_context_t *context, const mp_raw_code_t *rc) {
mp_obj_code_t *o = m_new_obj_maybe(mp_obj_code_t);
if (o == NULL) {
return MP_OBJ_NULL;
}
o->base.type = &mp_type_settrace_codeobj;
+ o->context = context;
o->rc = rc;
o->dict_locals = mp_locals_get(); // this is a wrong! how to do this properly?
o->lnotab = MP_OBJ_NULL;
@@ -204,9 +205,9 @@ STATIC void frame_print(const mp_print_t *print, mp_obj_t o_in, mp_print_kind_t
mp_printf(print,
"<frame at 0x%p, file '%q', line %d, code %q>",
frame,
- prelude->qstr_source_file,
+ QSTR_MAP(code->context, 0),
frame->lineno,
- prelude->qstr_block_name
+ QSTR_MAP(code->context, prelude->qstr_block_name_idx)
);
}
@@ -229,7 +230,7 @@ STATIC void frame_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
dest[0] = MP_OBJ_FROM_PTR(o->code);
break;
case MP_QSTR_f_globals:
- dest[0] = MP_OBJ_FROM_PTR(o->code_state->fun_bc->globals);
+ dest[0] = MP_OBJ_FROM_PTR(o->code_state->fun_bc->context->module.globals);
break;
case MP_QSTR_f_lasti:
dest[0] = MP_OBJ_NEW_SMALL_INT(o->lasti);
@@ -258,7 +259,7 @@ mp_obj_t mp_obj_new_frame(const mp_code_state_t *code_state) {
return MP_OBJ_NULL;
}
- mp_obj_code_t *code = o->code = MP_OBJ_TO_PTR(mp_obj_new_code(code_state->fun_bc->rc));
+ mp_obj_code_t *code = o->code = MP_OBJ_TO_PTR(mp_obj_new_code(code_state->fun_bc->context, code_state->fun_bc->rc));
if (code == NULL) {
return MP_OBJ_NULL;
}
diff --git a/py/profile.h b/py/profile.h
index 64e207d04f..7f3f914034 100644
--- a/py/profile.h
+++ b/py/profile.h
@@ -34,7 +34,9 @@
#define mp_prof_is_executing MP_STATE_THREAD(prof_callback_is_executing)
typedef struct _mp_obj_code_t {
+ // TODO this was 4 words
mp_obj_base_t base;
+ const mp_module_context_t *context;
const mp_raw_code_t *rc;
mp_obj_dict_t *dict_locals;
mp_obj_t lnotab;
@@ -53,7 +55,7 @@ typedef struct _mp_obj_frame_t {
void mp_prof_extract_prelude(const byte *bytecode, mp_bytecode_prelude_t *prelude);
-mp_obj_t mp_obj_new_code(const mp_raw_code_t *rc);
+mp_obj_t mp_obj_new_code(const mp_module_context_t *mc, const mp_raw_code_t *rc);
mp_obj_t mp_obj_new_frame(const mp_code_state_t *code_state);
// This is the implementation for the sys.settrace
diff --git a/py/qstr.h b/py/qstr.h
index ded105760c..fa634f90b0 100644
--- a/py/qstr.h
+++ b/py/qstr.h
@@ -46,6 +46,7 @@ enum {
};
typedef size_t qstr;
+typedef uint16_t qstr_short_t;
#if MICROPY_QSTR_BYTES_IN_HASH == 1
typedef uint8_t qstr_hash_t;
diff --git a/py/scope.c b/py/scope.c
index 98e02fb53f..8fc0943289 100644
--- a/py/scope.c
+++ b/py/scope.c
@@ -40,7 +40,7 @@ STATIC const uint8_t scope_simple_name_table[] = {
[SCOPE_GEN_EXPR] = MP_QSTR__lt_genexpr_gt_,
};
-scope_t *scope_new(scope_kind_t kind, mp_parse_node_t pn, qstr source_file, mp_uint_t emit_options) {
+scope_t *scope_new(scope_kind_t kind, mp_parse_node_t pn, mp_uint_t emit_options) {
// Make sure those qstrs indeed fit in an uint8_t.
MP_STATIC_ASSERT(MP_QSTR__lt_module_gt_ <= UINT8_MAX);
MP_STATIC_ASSERT(MP_QSTR__lt_lambda_gt_ <= UINT8_MAX);
@@ -52,7 +52,6 @@ scope_t *scope_new(scope_kind_t kind, mp_parse_node_t pn, qstr source_file, mp_u
scope_t *scope = m_new0(scope_t, 1);
scope->kind = kind;
scope->pn = pn;
- scope->source_file = source_file;
if (kind == SCOPE_FUNCTION || kind == SCOPE_CLASS) {
assert(MP_PARSE_NODE_IS_STRUCT(pn));
scope->simple_name = MP_PARSE_NODE_LEAF_ARG(((mp_parse_node_struct_t *)pn)->nodes[0]);
diff --git a/py/scope.h b/py/scope.h
index edf164c4ad..5006deadea 100644
--- a/py/scope.h
+++ b/py/scope.h
@@ -75,7 +75,6 @@ typedef struct _scope_t {
struct _scope_t *next;
mp_parse_node_t pn;
mp_raw_code_t *raw_code;
- uint16_t source_file; // a qstr
uint16_t simple_name; // a qstr
uint16_t scope_flags; // see runtime0.h
uint16_t emit_options; // see emitglue.h
@@ -90,7 +89,7 @@ typedef struct _scope_t {
id_info_t *id_info;
} scope_t;
-scope_t *scope_new(scope_kind_t kind, mp_parse_node_t pn, qstr source_file, mp_uint_t emit_options);
+scope_t *scope_new(scope_kind_t kind, mp_parse_node_t pn, mp_uint_t emit_options);
void scope_free(scope_t *scope);
id_info_t *scope_find_or_add_id(scope_t *scope, qstr qstr, id_info_kind_t kind);
id_info_t *scope_find(scope_t *scope, qstr qstr);
diff --git a/py/showbc.c b/py/showbc.c
index da8077eccd..1300ac1fa3 100644
--- a/py/showbc.c
+++ b/py/showbc.c
@@ -41,43 +41,32 @@
#define DECODE_ULABEL do { unum = (ip[0] | (ip[1] << 8)); ip += 2; } while (0)
#define DECODE_SLABEL do { unum = (ip[0] | (ip[1] << 8)) - 0x8000; ip += 2; } while (0)
-#if MICROPY_PERSISTENT_CODE
+#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
#define DECODE_QSTR \
- qst = ip[0] | ip[1] << 8; \
- ip += 2;
-#define DECODE_PTR \
- DECODE_UINT; \
- unum = mp_showbc_const_table[unum]
-#define DECODE_OBJ \
DECODE_UINT; \
- unum = mp_showbc_const_table[unum]
+ qst = mp_showbc_constants->qstr_table[unum]
#else
-#define DECODE_QSTR { \
- qst = 0; \
- do { \
- qst = (qst << 7) + (*ip & 0x7f); \
- } while ((*ip++ & 0x80) != 0); \
-}
-#define DECODE_PTR do { \
- ip = (byte *)MP_ALIGN(ip, sizeof(void *)); \
- unum = (uintptr_t)*(void **)ip; \
- ip += sizeof(void *); \
-} while (0)
-#define DECODE_OBJ do { \
- ip = (byte *)MP_ALIGN(ip, sizeof(mp_obj_t)); \
- unum = (mp_uint_t)*(mp_obj_t *)ip; \
- ip += sizeof(mp_obj_t); \
-} while (0)
+#define DECODE_QSTR \
+ DECODE_UINT; \
+ qst = unum;
#endif
-const byte *mp_showbc_code_start;
-const mp_uint_t *mp_showbc_const_table;
+#define DECODE_PTR \
+ DECODE_UINT;
+
+#define DECODE_OBJ \
+ DECODE_UINT; \
+ unum = (mp_uint_t)mp_showbc_constants->obj_table[unum]
-void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *ip, mp_uint_t len, const mp_uint_t *const_table) {
+const byte * mp_showbc_code_start;
+const mp_module_constants_t *mp_showbc_constants;
+
+#include "py/emitglue.h"
+void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *ip, mp_uint_t len, const mp_module_constants_t *cm) {
mp_showbc_code_start = ip;
// Decode prelude
@@ -85,16 +74,15 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
MP_BC_PRELUDE_SIZE_DECODE(ip);
const byte *code_info = ip;
- #if MICROPY_PERSISTENT_CODE
- qstr block_name = code_info[0] | (code_info[1] << 8);
- qstr source_file = code_info[2] | (code_info[3] << 8);
- code_info += 4;
- #else
qstr block_name = mp_decode_uint(&code_info);
- qstr source_file = mp_decode_uint(&code_info);
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ block_name = cm->qstr_table[block_name];
+ qstr source_file = cm->qstr_table[0];
+ #else
+ qstr source_file = cm->source_file;
#endif
- mp_printf(print, "File %s, code block '%s' (descriptor: %p, bytecode @%p " UINT_FMT " bytes)\n",
- qstr_str(source_file), qstr_str(block_name), descr, mp_showbc_code_start, len);
+ mp_printf(print, "File %s, code block '%s' (descriptor: %p, bytecode @%p " UINT_FMT " bytes)\n",// rct=%p\n",
+ qstr_str(source_file), qstr_str(block_name), descr, mp_showbc_code_start, len);// , ((mp_raw_code_t*)descr)->children);
// raw bytecode dump
size_t prelude_size = ip - mp_showbc_code_start + n_info + n_cell;
@@ -111,7 +99,11 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
// bytecode prelude: arg names (as qstr objects)
mp_printf(print, "arg names:");
for (mp_uint_t i = 0; i < n_pos_args + n_kwonly_args; i++) {
- mp_printf(print, " %s", qstr_str(MP_OBJ_QSTR_VALUE(const_table[i])));
+ qstr qst = mp_decode_uint(&code_info);
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ qst = cm->qstr_table[qst];
+ #endif
+ mp_printf(print, " %s", qstr_str(qst));
}
mp_printf(print, "\n");
@@ -120,6 +112,7 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
// skip over code_info
ip += n_info;
+ const byte *line_info_top = ip;
// bytecode prelude: initialise closed over variables
for (size_t i = 0; i < n_cell; ++i) {
@@ -132,7 +125,7 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
mp_int_t bc = 0;
mp_uint_t source_line = 1;
mp_printf(print, " bc=" INT_FMT " line=" UINT_FMT "\n", bc, source_line);
- for (const byte *ci = code_info; *ci;) {
+ for (const byte *ci = code_info; ci < line_info_top;) {
if ((ci[0] & 0x80) == 0) {
// 0b0LLBBBBB encoding
bc += ci[0] & 0x1f;
@@ -147,7 +140,7 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
mp_printf(print, " bc=" INT_FMT " line=" UINT_FMT "\n", bc, source_line);
}
}
- mp_bytecode_print2(print, ip, len - prelude_size, const_table);
+ mp_bytecode_print2(print, ip, len - prelude_size, cm);
}
const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
@@ -535,9 +528,9 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
return ip;
}
-void mp_bytecode_print2(const mp_print_t *print, const byte *ip, size_t len, const mp_uint_t *const_table) {
+void mp_bytecode_print2(const mp_print_t *print, const byte *ip, size_t len, const mp_module_constants_t *cm) {
mp_showbc_code_start = ip;
- mp_showbc_const_table = const_table;
+ mp_showbc_constants = cm;
while (ip < len + mp_showbc_code_start) {
mp_printf(print, "%02u ", (uint)(ip - mp_showbc_code_start));
ip = mp_bytecode_print_str(print, ip);
diff --git a/py/vm.c b/py/vm.c
index 0289bcfae5..b762c92bf2 100644
--- a/py/vm.c
+++ b/py/vm.c
@@ -31,9 +31,9 @@
#include "py/emitglue.h"
#include "py/objtype.h"
+#include "py/objfun.h"
#include "py/runtime.h"
#include "py/bc0.h"
-#include "py/bc.h"
#include "py/profile.h"
// *FORMAT-OFF*
@@ -44,7 +44,7 @@
#else
#define TRACE_PREFIX mp_printf(&mp_plat_print, "sp=%d ", (int)(sp - &code_state->state[0] + 1))
#endif
-#define TRACE(ip) TRACE_PREFIX; mp_bytecode_print2(&mp_plat_print, ip, 1, code_state->fun_bc->const_table);
+#define TRACE(ip) TRACE_PREFIX; mp_bytecode_print2(&mp_plat_print, ip, 1, &code_state->fun_bc->context->constants);
#else
#define TRACE(ip)
#endif
@@ -64,35 +64,28 @@
#define DECODE_ULABEL size_t ulab = (ip[0] | (ip[1] << 8)); ip += 2
#define DECODE_SLABEL size_t slab = (ip[0] | (ip[1] << 8)) - 0x8000; ip += 2
-#if MICROPY_PERSISTENT_CODE
+#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
#define DECODE_QSTR \
- qstr qst = ip[0] | ip[1] << 8; \
- ip += 2;
-#define DECODE_PTR \
- DECODE_UINT; \
- void *ptr = (void*)(uintptr_t)code_state->fun_bc->const_table[unum]
-#define DECODE_OBJ \
DECODE_UINT; \
- mp_obj_t obj = (mp_obj_t)code_state->fun_bc->const_table[unum]
+ qstr qst = qstr_table[unum]
#else
-#define DECODE_QSTR qstr qst = 0; \
- do { \
- qst = (qst << 7) + (*ip & 0x7f); \
- } while ((*ip++ & 0x80) != 0)
-#define DECODE_PTR \
- ip = (byte*)MP_ALIGN(ip, sizeof(void*)); \
- void *ptr = *(void**)ip; \
- ip += sizeof(void*)
-#define DECODE_OBJ \
- ip = (byte*)MP_ALIGN(ip, sizeof(mp_obj_t)); \
- mp_obj_t obj = *(mp_obj_t*)ip; \
- ip += sizeof(mp_obj_t)
+#define DECODE_QSTR \
+ DECODE_UINT; \
+ qstr qst = unum;
#endif
+#define DECODE_PTR \
+ DECODE_UINT; \
+ void *ptr = (void *)(uintptr_t)code_state->fun_bc->child_table[unum]
+
+#define DECODE_OBJ \
+ DECODE_UINT; \
+ mp_obj_t obj = (mp_obj_t)code_state->fun_bc->context->constants.obj_table[unum]
+
#define PUSH(val) *++sp = (val)
#define POP() (*sp--)
#define TOP() (*sp)
@@ -255,6 +248,9 @@ outer_dispatch_loop:
// local variables that are not visible to the exception handler
const byte *ip = code_state->ip;
mp_obj_t *sp = code_state->sp;
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ const qstr_short_t *qstr_table = code_state->fun_bc->context->constants.qstr_table;
+ #endif
mp_obj_t obj_shared;
MICROPY_VM_HOOK_INIT
@@ -859,15 +855,15 @@ unwind_jump:;
ENTRY(MP_BC_MAKE_FUNCTION): {
DECODE_PTR;
- PUSH(mp_make_function_from_raw_code(ptr, MP_OBJ_NULL, MP_OBJ_NULL));
+ PUSH(mp_make_function_from_raw_code(ptr, code_state->fun_bc->context, NULL));
DISPATCH();
}
ENTRY(MP_BC_MAKE_FUNCTION_DEFARGS): {
DECODE_PTR;
// Stack layout: def_tuple def_dict <- TOS
- mp_obj_t def_dict = POP();
- SET_TOP(mp_make_function_from_raw_code(ptr, TOP(), def_dict));
+ sp -= 1;
+ SET_TOP(mp_make_function_from_raw_code(ptr, code_state->fun_bc->context, sp));
DISPATCH();
}
@@ -876,7 +872,7 @@ unwind_jump:;
size_t n_closed_over = *ip++;
// Stack layout: closed_overs <- TOS
sp -= n_closed_over - 1;
- SET_TOP(mp_make_closure_from_raw_code(ptr, n_closed_over, sp));
+ SET_TOP(mp_make_closure_from_raw_code(ptr, code_state->fun_bc->context, n_closed_over, sp));
DISPATCH();
}
@@ -885,7 +881,7 @@ unwind_jump:;
size_t n_closed_over = *ip++;
// Stack layout: def_tuple def_dict closed_overs <- TOS
sp -= 2 + n_closed_over - 1;
- SET_TOP(mp_make_closure_from_raw_code(ptr, 0x100 | n_closed_over, sp));
+ SET_TOP(mp_make_closure_from_raw_code(ptr, code_state->fun_bc->context, 0x100 | n_closed_over, sp));
DISPATCH();
}
@@ -1384,23 +1380,20 @@ unwind_loop:
const byte *ip = code_state->fun_bc->bytecode;
MP_BC_PRELUDE_SIG_DECODE(ip);
MP_BC_PRELUDE_SIZE_DECODE(ip);
+ const byte *line_info_top = ip + n_info;
const byte *bytecode_start = ip + n_info + n_cell;
- #if !MICROPY_PERSISTENT_CODE
- // so bytecode is aligned
- bytecode_start = MP_ALIGN(bytecode_start, sizeof(mp_uint_t));
- #endif
size_t bc = code_state->ip - bytecode_start;
- #if MICROPY_PERSISTENT_CODE
- qstr block_name = ip[0] | (ip[1] << 8);
- qstr source_file = ip[2] | (ip[3] << 8);
- ip += 4;
- #else
qstr block_name = mp_decode_uint_value(ip);
- ip = mp_decode_uint_skip(ip);
- qstr source_file = mp_decode_uint_value(ip);
- ip = mp_decode_uint_skip(ip);
+ for (size_t i = 0; i < 1 + n_pos_args + n_kwonly_args; ++i) {
+ ip = mp_decode_uint_skip(ip);
+ }
+ #if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
+ block_name = code_state->fun_bc->context->constants.qstr_table[block_name];
+ qstr source_file = code_state->fun_bc->context->constants.qstr_table[0];
+ #else
+ qstr source_file = code_state->fun_bc->context->constants.source_file;
#endif
- size_t source_line = mp_bytecode_get_source_line(ip, bc);
+ size_t source_line = mp_bytecode_get_source_line(ip, line_info_top, bc);
mp_obj_exception_add_traceback(MP_OBJ_FROM_PTR(nlr.ret_val), source_file, source_line, block_name);
}