Merge tag 'v1.8.5' into parse-bytecode

New port to Zephyr, upip runs on baremetal, and reduction in code size

This release adds a new port of MicroPython to run on top of the Zephyr
real-time operating system.  As part of this there is now basic support for
using mbedTLS as the ussl module.  The release also brings initial support
for running the package manager upip on baremetal systems with low heap
memory (such as esp8266), through a Python module interface.

Work has been done in this release to clean up the core, removing redundant
and/or unreachable code, and factoring existing duplicated code patterns.
This brings a reduction of 828 bytes in code size to the bare-arm port, and
1368 bytes to the minimal port.  There is also improved coverage through
the addition of new tests for corner cases.

The "micropython" module sees the addition of the "const" identity function
which should be used as "from micropython import const" in scripts that
want to use the MicroPython constant optimisations in the compile stage.
There is also the addition of the "opt_level" function to change the
parser/compiler optimisation level at runtime.

The behaviour of "sys.exit" (and "raise SystemExit") on baremetal is now
changed: this function no longer does a soft-reset of the board, rather it
just stops the running script and drops to the REPL.  In order to do an
actual soft reset the "machine.soft_reset" function has been added (to the
stmhal port only, for the time being).

Following CPython, KeyError exceptions for dictionary lookups now have the
failed key stored as the argument of the exception instance, accessible as
exc.args[0].  The "ujson.load" function has also been added, to load JSON
data from an arbitrary stream.

The I2C support in the machine module now has clock stretching, the
addition of the "addrsize" parameter in memory transfer methods, and I2C
scanning now uses dummy writes instead of dummy reads to make the scanning
more reliable.

The CMSIS library has been upgrade to v4.30, and the boards section of
the stmhal port has been refactored to use a common.ld script.  The stmhal
port now has a full implementation of the machine.SPI class, with support
for both hardware SPI peripherals and fast software SPI.  The USB HID
driver in stmhal has added support to receive USB HID messages from the
host.

py core:
- asmthumb: flush D-cache, and invalidate I-cache on STM32F7
- makeqstrdefs.py: use python 2.6 syntax for set creation
- objnone: remove unnecessary handling of MP_UNARY_OP_BOOL
- move frozen modules rules from esp8266 port for reuse across ports
- combine 3 comprehension emit functions (list/dict/set) into 1
- combine 3 comprehension opcodes (list/dict/set) into 1
- vstr: remove vstr.had_error flag and inline basic vstr functions
- objnone: use mp_generic_unary_op instead of custom one
- showbc: make printf's go to the platform print stream
- remove 'name' member from mp_obj_module_t struct
- builtinimport: fix nanbox build after change to better handle -m modules
- stream: remove unnecessary check for NULL return from vstr_extend
- py.mk: suppress some compiler warnings when building berkeley-db
- shrink mp_arg_t struct by using reduced-size integer members
- update opcode format table because 3 opcodes were removed, 1 added
- parse: only replace constants that are standalone identifiers
- py.mk: add support for building modussl_mbedtls
- only store the exception instance on Py stack in bytecode try block
- vm: use MP_OBJ_FROM_PTR to cast a type to an object
- modmicropython: add micropython.const, alias for identity function
- objstr: remove unreachable function used only for terse error msgs
- emitbc: remove/refactor unreachable code, to improve coverage
- objfun: remove unnecessary check for viper fun with 5 or more args
- objfun: use if instead of switch to check return value of VM execute
- objset: use mp_check_self() to check args of set/frozenset methods
- objset: ensure that use of frozenset.update raises an exception
- compile: fix async-for/async-with to work with simpler exc on stack
- scope: use lookup-table to determine a scope's simple name
- scope: shrink scope_t struct by 1 machine word
- scope: factor common code to find locals and close over them
- compile: fix typo when checking for parse-node kind
- argcheck: simplify if-chain so that the last one is the default
- objbool: defer bool's unary op implementation to small int
- objbool: make a slight simplification of bool constructor
- modstruct: remove unreachable code, and add comment about CPy diff
- add mp_raise_OSError(errno) helper function
- objstringio: add readinto() method
- modmicropython: add micropython.opt_level([value]) function
- compile: remove unreachable code
- mpz: in divmod, replace check for rhs!=0 with assert
- mpz: use assert to verify mpz does not have a fixed digit buffer
- factor duplicated function to calculate size of formatted int
- objint: use size_t for arguments that measure bytes/sizes
- compile: remove debugging code for compiler dispatch
- lexer: remove unnecessary code, and unreachable code
- vstr: combine vstr_new_size with vstr_new since they are rarely used
- objdict: fix optimisation for allocating result in fromkeys
- objdict: actually provide the key that failed in KeyError exception
- use mp_raise_msg helper function where appropriate
- add the module specified by -m to sys.modules as '__main__'

extmod:
- modussl_mbedtls: initial implementation of mbedTLS ussl module
- uctypes: allow full 32-bit address range
- modubinascii: fix crc32() function on 32-bit platforms
- modussl_mbedtls: implement key= and cert= args to wrap_socket()
- modussl_mbedtls: use 2-component include paths
- machine_i2c: add clock stretching support
- modussl_mbedtls: add server_hostname param for wrap_socket()
- uzlib: add tinfgzip.c (gzip header parsing) from upstream
- moduzlib: DecompIO: Add support for gzip-formatted streams
- uzlib/: update uzlib to v2.0.3
- vfs_fat: add fat_vfs_statvfs(), reused from stmhal
- machine_i2c: add support for the addrsize parameter in mem xfers
- machine_spi: simplify SPI xfer function to only take one buf len
- machine_spi: factor out software SPI code from esp8266 to extmod
- machine_spi: use delay_half, not baudrate, for internal timing
- machine_spi: add optional support for fast software SPI
- vfs_fat: use mp_raise_OSError helper function
- modlwip: use mp_raise_OSError helper function
- use mp_raise_OSError helper function
- vfs_fat_file: use MP_Exxx errno constants
- uzlib: update to upstream v2.1
- machine_i2c: use writes not reads in i2c.scan()
- vfs_fat: add file and directory checks for remove and rmdir
- modujson: implement ujson.load() to load JSON from a stream
- modujson: fix nanbox build
- utime_mphal: factor out implementations in terms of mp_hal_* for reuse
- utime_mphal: sleep_us/ms(): Don't wait on negative argument
- modussl_mbedtls: add dummy setblocking() method

lib:
- interrupt_char: factor out typical Ctrl+C handling from esp8266 port
- cmsis: move CMSIS headers to lib/
- cmsis: remove CMSIS-DSP headers, they are unused
- cmsis: upgrade CMSIS-CORE to V4.30
- utils/pyexec: don't treat SystemExit as "forced exit"
- utils/pyexec: allow behaviour of SystemExit to be configurable

drivers:
- dht: use mp_raise_OSError helper function

tools:
- update upip to 0.8, fixes IPv6 support
- upgrade upip to 1.0, fully self-hosted release (without fallbacks), and
  uses stream gzip decompression (step towards support for limited-heap
  baremetal systems)
- upgrade upip to 1.1.3, initial support for running on a baremetal,
  low-heap systems (like esp8266), using Python module interface

tests:
- pyb: add test for ExtInt when doing swint while disabled
- pyb: update exp file for previously updated extint test
- extmod/urandom: add urandom tests for error cases
- basics: add errno1 test, to check basics of uerrno module
- extmod: add test for machine.time_pulse_us()
- struct1: test "l" specifier to improve coverage
- array1: add tests for "l", "L" array types to improve coverage
- get cmdline verbose tests running again
- run-tests: add --via-mpy option to run test from precompiled code
- uzlib_decompio_gz: test for DecompIO with gzip bitstream
- basics: add test case for overflowing Py stack in try-finally
- micropython: add tests for const names being replaced in parser
- cmdline/cmd_showbc: fix test now that 1 value is stored on stack
- extmod/vfs_fat_ramdisk: add test for VFS.statvfs()
- float: add test for parsing a float from an empty string
- basics: add test for set.difference_update with arg being itself
- basics: add further tests for nonlocal scoping and closures
- import: add test for compiling "import a.b as c"
- basics: add test constructing a set from a non-trivial expression
- basics: add test for printing OSError when errno is unknown
- run-tests: disable cmdline/cmd_showbc test on Windows
- extmod/btree1: checks for put, seq, string print and unsupported binary op
- fix expected output of verbose cmdline test
- extmod/uzlib: test adaptive huffman tree for tinflate coverage
- improve coverage of struct with test for non-compliant behaviour
- io/write_ext: add description comment
- io/bytesio_ext: add test for readinto()
- micropython: add test for micropython.opt_level() function
- improve test coverage of py/compile.c
- extmod/vfs_fat: test coverage for remove() and rmdir()
- extmod: add test for ujson.load()
- extmod/vfs_fat: replace asserts with prints and expected outputs
- micropython: add tests for heap_lock, and emergency exceptions
- cmdline: improve coverage test for printing bytecode
- improve coverage of array, range, dict, slice, exc, unicode
- add test to print full KeyError exc from failed dict lookup
- run-tests: enable extmod/machine1.py on pyboard

unix port:
- fix build for when MICROPY_PY_SOCKET=0
- modjni: implement subscription for object arrays
- modjni: add array() top-level function to create Java array
- modjni: array(): Support creation of object arrays
- enable btree module for coverage build
- use mp_raise_OSError helper function
- use common RAISE_ERRNO macro from mphalport.h

windows port:
- enable MICROPY_PY_UERRNO

qemu-arm port:
- enable lots of extmods and enable tests for them

stmhal port:
- lcd: de-assert chip select after completing SPI transmission
- {accel,lcd}: use GPIO_{set,clear}_pin
- extint: force 0 to 1 transition on swint()
- boards: add pllvalues.py script to compute PLL values for sysclk
- boards: for OLIMEX_E407, enable UART1 and fix I2C1 mapping
- use attribute to avoid inlining
- put common definitions from linker files to common.ld
- remove STM32CubeF2 HAL files, they are unused/unsupported
- modmachine: fix clearing of reset-cause flags
- add virtual com port support for STM32L476DISC
- remove CMSIS STM32F2xx files, they are unused/unsupported
- spi: simplify spi_transfer function to take only one buf len arg
- mphalport: implement mp_hal_pin_{input,output,write}
- spi: make machine.SPI class conform to correct API
- mphalport: fix mp_hal_pin_write to use correct pin_mask
- spi: use software SPI if no periph id given, even if pins given
- spi: enable use of fast software SPI
- fix linker map for STM32L476 chips
- usbdev: add OUT endpoint to HID interface
- usb: add support to receive USB HID messages from host
- usb: use correct ClassData structure for HID receive
- usb: use real packet size (not maximum) in HID receive
- fix ESPRUINO_PICO by adding ld scripts with correct flash size
- mphalport: change pin obj type to const pointer, to avoid casts
- moduos: implement total-number-of-blocks field in statvfs
- disable network and usocket for ESPRUINO_PICO
- enable machine.time_pulse_us() function
- use mp_raise_OSError helper function
- pybstdio: use size_t instead of mp_uint_t
- modutime: refactor to use extmod/utime_mphal.c
- implement machine.soft_reset()
- enable str.center(), str.[r]partition() and builtin compile()

cc3200 port:
- add ssl_version argument to ssl.wrap_socket()

esp8266 port:
- Makefile: rename SCRIPTDIR to FROZEN_DIR for consistency with FROZEN_MPY_DIR
- ets_alt_task: ets_post: Should return 0 on success, !0 - failure
- esp_mphal: add tentative change to mp_hal_stdin_rx_chr() to wait IRQ
- extend system microsecond counter to 64-bits; use in ticks_ms
- add uos.statvfs() to get filesystem status
- moduos: move stat/statvfs funcs to sit within #if VFS guard
- modmachine: idle(): Return number of CPU cycles spent idling
- main: put /lib before / in sys.path
- modpybrtc.c: implement machine.RTC.alarm_left()
- make PY_UHASHLIB_SHA1 config depend on PY_USSL and SSL_AXTLS
- add FLASH_MODE,FLASH_SIZE options for make deploy target
- use mp_raise_OSError helper function
- make neopixel support configurable
- mpconfigport: enable MICROPY_PY_BUILTINS_SLICE_ATTRS
- enable sys.{stdin,stdout,stderr}.buffer for raw serial access
- enable importing of precompiled .mpy files
- enable micropython.alloc_emergency_exception_buf()

zephyr port:
- initial Zephyr RTOS port, MicroPython part
- initial Zephyr RTOS port, Zephyr part
- add zephyr_getchar module to handle console input
- switch to microkernel, required for network to work in background
- automatically derive ARCH
- support extra make targets
- Makefile: automatically derive target-specific CFLAGS
- use recently added "make outputexports" Zephyr target
- add README
- enable stack checking and micropython.mem_info()
- enable frozen modules support
- main: execute main.py frozen module on boot, if available
- zephyr_getchar: add support for Ctrl+C handling
- add Ctrl+C handling
- implement the help() function
- add copyright blurbs

README:
- remove issue-stats badges, the service is no longer available
- mention _thread module availability in select ports

docs:
- library/pyb.SPI: init(): describe "bits" argument
- library/machine: update description of disable/enable IRQ funcs
- uos: add uos.statvfs() documentation
- wipy: correct deep sleep current figure
- wipy: small doc fixes
- reference: add constrained.rst doc

travis:
- abandon mingw32 in favour of mingw-w64
- run feature and coverage test for precompiled mpy files

examples:
- network/http_client*: use \r\n line-endings in request

1 files changed, 456 insertions, 0 deletions

diff --git a/docs/reference/constrained.rst b/docs/reference/constrained.rst
new file mode 100644
index 0000000000..7c1b6a3eb2
--- /dev/null
+++ b/docs/reference/constrained.rst
@@ -0,0 +1,456 @@
+.. _constrained:
+
+MicroPython on Microcontrollers
+===============================
+
+MicroPython is designed to be capable of running on microcontrollers. These
+have hardware limitations which may be unfamiliar to programmers more familiar
+with conventional computers. In particular the amount of RAM and nonvolatile
+"disk" (flash memory) storage is limited. This tutorial offers ways to make
+the most of the limited resources. Because MicroPython runs on controllers
+based on a variety of architectures, the methods presented are generic: in some
+cases it will be necessary to obtain detailed information from platform specific
+documentation.
+
+Flash Memory
+------------
+
+On the Pyboard the simple way to address the limited capacity is to fit a micro
+SD card. In some cases this is impractical, either because the device does not
+have an SD card slot or for reasons of cost or power consumption; hence the
+on-chip flash must be used. The firmware including the MicroPython subsystem is
+stored in the onboard flash. The remaining capacity is available for use. For
+reasons connected with the physical architecture of the flash memory part of
+this capacity may be inaccessible as a filesystem. In such cases this space may
+be employed by incorporating user modules into a firmware build which is then
+flashed to the device.
+
+There are two ways to achieve this: frozen modules and frozen bytecode. Frozen
+modules store the Python source with the firmware. Frozen bytecode uses the
+cross compiler to convert the source to bytecode which is then stored with the
+firmware. In either case the module may be accessed with an import statement:
+
+.. code::
+
+    import mymodule
+
+The procedure for producing frozen modules and bytecode is platform dependent;
+instructions for building the firmware can be found in the README files in the
+relevant part of the source tree.
+
+In general terms the steps are as follows:
+
+* Clone the MicroPython `repository <https://github.com/micropython/micropython>`_.
+* Acquire the (platform specific) toolchain to build the firmware.
+* Build the cross compiler.
+* Place the modules to be frozen in a specified directory (dependent on whether
+  the module is to be frozen as source or as bytecode).
+* Build the firmware. A specific command may be required to build frozen
+  code of either type - see the platform documentation.
+* Flash the firmware to the device.
+
+RAM
+---
+
+When reducing RAM usage there are two phases to consider: compilation and
+execution. In addition to memory consumption, there is also an issue known as
+heap fragmentation. In general terms it is best to minimise the repeated
+creation and destruction of objects. The reason for this is covered in the
+section covering the `heap`_.
+
+Compilation Phase
+~~~~~~~~~~~~~~~~~
+
+When a module is imported, MicroPython compiles the code to bytecode which is
+then executed by the MicroPython virtual machine (VM). The bytecode is stored
+in RAM. The compiler itself requires RAM, but this becomes available for use
+when the compilation has completed.
+
+If a number of modules have already been imported the situation can arise where
+there is insufficient RAM to run the compiler. In this case the import
+statement will produce a memory exception.
+
+If a module instantiates global objects on import it will consume RAM at the
+time of import, which is then unavailable for the compiler to use on subsequent
+imports. In general it is best to avoid code which runs on import; a better
+approach is to have initialisation code which is run by the application after
+all modules have been imported. This maximises the RAM available to the
+compiler.
+
+If RAM is still insufficient to compile all modules one solution is to
+precompile modules. MicroPython has a cross compiler capable of compiling Python
+modules to bytecode (see the README in the mpy-cross directory). The resulting
+bytecode file has a .mpy extension; it may be copied to the filesystem and
+imported in the usual way. Alternatively some or all modules may be implemented
+as frozen bytecode: on most platforms this saves even more RAM as the bytecode
+is run directly from flash rather than being stored in RAM.
+
+Execution Phase
+~~~~~~~~~~~~~~~
+
+There are a number of coding techniques for reducing RAM usage.
+
+**Constants**
+
+MicroPython provides a ``const`` keyword which may be used as follows:
+
+.. code::
+
+    from micropython import const
+    ROWS = const(33)
+    _COLS = const(0x10)
+    a = ROWS
+    b = _COLS
+
+In both instances where the constant is assigned to a variable the compiler
+will avoid coding a lookup to the name of the constant by substituting its
+literal value. This saves bytecode and hence RAM. However the ``ROWS`` value
+will occupy at least two machine words, one each for the key and value in the
+globals dictionary. The presence in the dictionary is necessary because another
+module might import or use it. This RAM can be saved by prepending the name
+with an underscore as in ``_COLS``: this symbol is not visible outside the
+module so will not occupy RAM.
+
+The argument to ``const()`` may be anything which, at compile time, evaluates
+to an integer e.g. ``0x100`` or ``1 << 8``. It can even include other const
+symbols that have already been defined, e.g. ``1 << BIT``.
+
+**Constant data structures**
+
+Where there is a substantial volume of constant data and the platform supports
+execution from Flash, RAM may be saved as follows. The data should be located in
+Python modules and frozen as bytecode. The data must be defined as ``bytes``
+objects. The compiler 'knows' that ``bytes`` objects are immutable and ensures
+that the objects remain in flash memory rather than being copied to RAM. The
+``ustruct`` module can assist in converting between ``bytes`` types and other
+Python built-in types.
+
+When considering the implications of frozen bytecode, note that in Python
+strings, floats, bytes, integers and complex numbers are immutable. Accordingly
+these will be frozen into flash. Thus, in the line
+
+.. code::
+
+    mystring = "The quick brown fox"
+
+the actual string "The quick brown fox" will reside in flash. At runtime a
+reference to the string is assigned to the *variable* ``mystring``. The reference
+occupies a single machine word. In principle a long integer could be used to
+store constant data:
+
+.. code::
+
+    bar = 0xDEADBEEF0000DEADBEEF
+
+As in the string example, at runtime a reference to the arbitrarily large
+integer is assigned to the variable ``bar``. That reference occupies a
+single machine word. 
+
+It might be expected that tuples of integers could be employed for the purpose
+of storing constant data with minimal RAM use. With the current compiler this
+is ineffective (the code works, but RAM is not saved).
+
+.. code::
+
+    foo = (1, 2, 3, 4, 5, 6, 100000)
+
+At runtime the tuple will be located in RAM. This may be subject to future
+improvement.
+
+**Needless object creation**
+
+There are a number of situations where objects may unwittingly be created and
+destroyed. This can reduce the usability of RAM through fragmentation. The
+following sections discuss instances of this.
+
+**String concatenation**
+
+Consider the following code fragments which aim to produce constant strings:
+
+.. code::
+
+    var = "foo" + "bar"
+    var1 = "foo" "bar"
+    var2 = """\
+    foo\
+    bar"""
+
+Each produces the same outcome, however the first needlessly creates two string
+objects at runtime, allocates more RAM for concatenation before producing the
+third. The others perform the concatenation at compile time which is more
+efficient, reducing fragmentation.
+
+Where strings must be dynamically created before being fed to a stream such as
+a file it will save RAM if this is done in a piecemeal fashion. Rather than
+creating a large string object, create a substring and feed it to the stream
+before dealing with the next.
+
+The best way to create dynamic strings is by means of the string ``format``
+method:
+
+.. code::
+
+    var = "Temperature {:5.2f} Pressure {:06d}\n".format(temp, press)
+
+**Buffers**
+
+When accessing devices such as instances of UART, I2C and SPI interfaces, using
+pre-allocated buffers avoids the creation of needless objects. Consider these
+two loops:
+
+.. code::
+
+    while True:
+        var = spi.read(100)
+        # process data
+
+    buf = bytearray(100)
+    while True:
+        spi.readinto(buf)
+        # process data in buf
+
+The first creates a buffer on each pass whereas the second re-uses a pre-allocated
+buffer; this is both faster and more efficient in terms of memory fragmentation.
+
+**Bytes are smaller than ints**
+
+On most platforms an integer consumes four bytes. Consider the two calls to the
+function ``foo()``:
+
+.. code::
+
+    def foo(bar):
+        for x in bar:
+            print(x)
+    foo((1, 2, 0xff))
+    foo(b'\1\2\xff')
+
+In the first call a tuple of integers is created in RAM. The second efficiently
+creates a ``bytes`` object consuming the minimum amount of RAM. If the module
+were frozen as bytecode, the ``bytes`` object would reside in flash.
+
+**Strings Versus Bytes**
+
+Python3 introduced Unicode support. This introduced a distinction between a
+string and an array of bytes. MicroPython ensures that Unicode strings take no
+additional space so long as all characters in the string are ASCII (i.e. have
+a value < 126). If values in the full 8-bit range are required ``bytes`` and
+``bytearray`` objects can be used to ensure that no additional space will be
+required. Note that most string methods (e.g. ``strip()``) apply also to ``bytes``
+instances so the process of eliminating Unicode can be painless.
+
+.. code::
+
+    s = 'the quick brown fox'  # A string instance
+    b = b'the quick brown fox'  # a bytes instance
+
+Where it is necessary to convert between strings and bytes the string ``encode``
+and the bytes ``decode`` methods can be used. Note that both strings and bytes
+are immutable. Any operation which takes as input such an object and produces
+another implies at least one RAM allocation to produce the result. In the
+second line below a new bytes object is allocated. This would also occur if ``foo``
+were a string.
+
+.. code::
+
+    foo = b'   empty whitespace'
+    foo = foo.lstrip()
+
+**Runtime compiler execution**
+
+The Python keywords ``eval`` and ``exec`` invoke the compiler at runtime, which
+requires significant amounts of RAM. Note that the ``pickle`` library employs
+``exec``. It may be more RAM efficient to use the ``json`` library for object
+serialisation.
+
+**Storing strings in flash**
+
+Python strings are immutable hence have the potential to be stored in read only
+memory. The compiler can place in flash strings defined in Python code. As with
+frozen modules it is necessary to have a copy of the source tree on the PC and
+the toolchain to build the firmware. The procedure will work even if the
+modules have not been fully debugged, so long as they can be imported and run.
+
+After importing the modules, execute:
+
+.. code::
+
+    micropython.qstr_info(1)
+
+Then copy and paste all the Q(xxx) lines into a text editor. Check for and
+remove lines which are obviously invalid. Open the file qstrdefsport.h which
+will be found in stmhal (or the equivalent directory for the architecture in
+use). Copy and paste the corrected lines at the end of the file. Save the file,
+rebuild and flash the firmware. The outcome can be checked by importing the
+modules and again issuing:
+
+.. code::
+
+    micropython.qstr_info(1)
+
+The Q(xxx) lines should be gone.
+
+.. _heap:
+
+The Heap
+--------
+
+When a running program instantiates an object the necessary RAM is allocated
+from a fixed size pool known as the heap. When the object goes out of scope (in
+other words becomes inaccessible to code) the redundant object is known as
+"garbage". A process known as "garbage collection" (GC) reclaims that memory,
+returning it to the free heap. This process runs automatically, however it can
+be invoked directly by issuing ``gc.collect()``.
+
+The discourse on this is somewhat involved. For a 'quick fix' issue the
+following periodically:
+
+.. code::
+
+    gc.collect()
+    gc.threshold(gc.mem_free() // 4 + gc.mem_alloc())
+
+Fragmentation
+~~~~~~~~~~~~~
+
+Say a program creates an object ``foo``, then an object ``bar``. Subsequently
+``foo`` goes out of scope but ``bar`` remains. The RAM used by ``foo`` will be
+reclaimed by GC. However if ``bar`` was allocated to a higher address, the
+RAM reclaimed from ``foo`` will only be of use for objects no bigger than
+``foo``. In a complex or long running program the heap can become fragmented:
+despite there being a substantial amount of RAM available, there is insufficient
+contiguous space to allocate a particular object, and the program fails with a
+memory error.
+
+The techniques outlined above aim to minimise this. Where large permanent buffers
+or other objects are required it is best to instantiate these early in the
+process of program execution before fragmentation can occur. Further improvements
+may be made by monitoring the state of the heap and by controlling GC; these are
+outlined below.
+
+Reporting
+~~~~~~~~~
+
+A number of library functions are available to report on memory allocation and
+to control GC. These are to be found in the ``gc`` and ``micropython`` modules.
+The following example may be pasted at the REPL (``ctrl e`` to enter paste mode,
+``ctrl d`` to run it).
+
+.. code::
+
+    import gc
+    import micropython
+    gc.collect()
+    micropython.mem_info()
+    print('-----------------------------')
+    print('Initial free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
+    def func():
+        a = bytearray(10000)
+    gc.collect()
+    print('Func definition: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
+    func()
+    print('Func run free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
+    gc.collect()
+    print('Garbage collect free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
+    print('-----------------------------')
+    micropython.mem_info(1)
+
+Methods employed above:
+
+* ``gc.collect()`` Force a garbage collection. See footnote.
+* ``micropython.mem_info()`` Print a summary of RAM utilisation.
+* ``gc.mem_free()`` Return the free heap size in bytes.
+* ``gc.mem_alloc()`` Return the number of bytes currently allocated.
+* ``micropython.mem_info(1)`` Print a table of heap utilisation (detailed below).
+
+The numbers produced are dependent on the platform, but it can be seen that
+declaring the function uses a small amount of RAM in the form of bytecode
+emitted by the compiler (the RAM used by the compiler has been reclaimed).
+Running the function uses over 10KiB, but on return ``a`` is garbage because it
+is out of scope and cannot be referenced. The final ``gc.collect()`` recovers
+that memory.
+
+The final output produced by ``micropython.mem_info(1)`` will vary in detail but
+may be interpreted as follows:
+
+====== =================
+Symbol Meaning
+====== =================
+   .   free block
+   h   head block
+   =   tail block
+   m   marked head block
+   T   tuple
+   L   list
+   D   dict
+   F   float
+   B   byte code
+   M   module
+====== =================
+
+Each letter represents a single block of memory, a block being 16 bytes. So each
+line of the heap dump represents 0x400 bytes or 1KiB of RAM.
+
+Control of Garbage Collection
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A GC can be demanded at any time by issuing ``gc.collect()``. It is advantageous
+to do this at intervals, firstly to pre-empt fragmentation and secondly for
+performance. A GC can take several milliseconds but is quicker when there is
+little work to do (about 1ms on the Pyboard). An explicit call can minimise that
+delay while ensuring it occurs at points in the program when it is acceptable.
+
+Automatic GC is provoked under the following circumstances. When an attempt at
+allocation fails, a GC is performed and the allocation re-tried. Only if this
+fails is an exception raised. Secondly an automatic GC will be triggered if the
+amount of free RAM falls below a threshold. This threshold can be adapted as
+execution progresses:
+
+.. code::
+
+    gc.collect()
+    gc.threshold(gc.mem_free() // 4 + gc.mem_alloc())
+
+This will provoke a GC when more than 25% of the currently free heap becomes
+occupied.
+
+In general modules should instantiate data objects at runtime using constructors
+or other initialisation functions. The reason is that if this occurs on
+initialisation the compiler may be starved of RAM when subsequent modules are
+imported. If modules do instantiate data on import then ``gc.collect()`` issued
+after the import will ameliorate the problem.
+
+String Operations
+-----------------
+
+MicroPython handles strings in an efficient manner and understanding this can
+help in designing applications to run on microcontrollers. When a module
+is compiled, strings which occur multiple times are stored once only, a process
+known as string interning. In MicroPython an interned string is known as a ``qstr``.
+In a module imported normally that single instance will be located in RAM, but
+as described above, in modules frozen as bytecode it will be located in flash.
+
+String comparisons are also performed efficiently using hashing rather than
+character by character. The penalty for using strings rather than integers may
+hence be small both in terms of performance and RAM usage - a fact which may
+come as a surprise to C programmers.
+
+Postscript
+----------
+
+MicroPython passes, returns and (by default) copies objects by reference. A
+reference occupies a single machine word so these processes are efficient in
+RAM usage and speed.
+
+Where variables are required whose size is neither a byte nor a machine word
+there are standard libraries which can assist in storing these efficiently and
+in performing conversions. See the ``array``, ``ustruct`` and ``uctypes``
+modules.
+
+Footnote: gc.collect() return value
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+On Unix and Windows platforms the ``gc.collect()`` method returns an integer
+which signifies the number of distinct memory regions that were reclaimed in the
+collection (more precisely, the number of heads that were turned into frees). For
+efficiency reasons bare metal ports do not return this value.