Merge tag 'v1.8.4' into parse-bytecode

Support for stream decompression in uzlib, and more ESP8266 features

This release includes some bug fixes, code clean-up, updates to the docs,
more tests, and various feature additions.  The uzlib module now supports
efficient stream decompression in the form of the uzlib.DecompIO class.
Freezing of bytecode now supports floats for the ESP8266 port, as well as
complex numbers for all ports.  The stmhal port has ADC working on L4
microcontrollers, fixed initialisation for DAC, and addition of the
machine.WDT class and machine.reset_cause function.

For the ESP8266 port Pin(16) now works as an input pin and the hardware
SPI peripheral is exposed as machine.SPI(1).  The os.umount function is
implemented and the port supports mounting of externally connected SD
cards.  The machine.WDT class is added, wlan.scan() is fixed to return all
access points, and there is support for DS18S20 devices.

py core:
- runtime: factor out exception raising helpers
- runtime: define mp_check_self(pred) helper macro
- objdict: get rid of asserts (remove/replace with mp_check_self())
- get rid of assert() in method argument checking functions
- objtuple: in tuple_cmp_helper, use mp_check_self instead of raising
- objstr: use mp_raise_{Type,Value}Error instead of mp_raise_msg
- obj.h: for obj reprs A,B,C use void* explicitly for mp_obj_t typedef
- mpconfigport.h: remove typedef of machine_ptr_t, it's no longer needed
- sequence: allow to use bignums as indices in slice objects
- stream.c: use mp_obj_get_type in mp_get_stream_raise
- gc: add MICROPY_GC_CONSERVATIVE_CLEAR option to always zero memory
- compile: don't compile assert statements when optimisations enabled
- modstruct: use more compact mp_raise_ValueError function
- emitglue: use more compact mp_raise_ValueError function
- rename struct mp_code_state to mp_code_state_t
- mkrules.mk: allow to override name of libmicropython.a
- mpprint: fail an assertion with unsupported format specifiers
- makeqstrdata.py: compute the qstr hash from bytes, not characters
- if str/bytes hash is 0 then explicitly compute it
- emitglue.c: provide mp_raw_code_load_file for any unix architecture
- add MICROPY_USE_INTERNAL_PRINTF option, defaults to enabled

extmod:
- modwebrepl: set_password(): raise exception for too long password
- uzlib/: update uzlib to v2.0: new API supporting stream decompression
- moduzlib: refactor to new stream-compatible uzlib 2.0 API
- uzlib/: update uzlib to v2.0.1: fixes for pedantic compiler warnings
- uzlib/: update uzlib to v2.0.2: consistently use stdint types
- modbtree: do CHECK_ERROR after __bt_seq()
- modubinascii: implement binascii.crc32
- modubinascii: make crc32() support configurable
- modframebuf: fix pixel accessor to return a 1-bit result
- add machine_spi with generic SPI C-protocol and helper methods
- modframebuf: fix fill and scroll when height not divisible by 8
- moduzlib: implement zlib stream decompressor class, DecompIO
- moduzlib: use mperrno.h for error constants
- modframebuf: include font from stmhal directory explicitly
- moduzlib: support wbits arg to DecompIO
- framebuf: add the xstep!=0 case to scroll() method

lib:
- utils/stdout_helpers: fix function signature to match py/mphal.h
- berkeley-db-1.xx: update to upstream, fixes MacOSX build
- utils/pyexec: qstr_pool_info() requires size_t* parameters

drivers:
- sdcard: port the SDCard driver to new machine API, with backwards
  compatibility for pyboard

tools:
- mpy-tool.py: support freezing float literals with obj-repr C
- mpy-tool.py: store qstr config values in global config object
- mpy-tool.py: compute the hash value for str/bytes objects
- mpy-tool.py: support freezing of complex numbers

tests:
- rename zlibd_decompress.py -> uzlib_decompress.py
- basics: add more tuple tests to improve coverage testing
- basics: add more list tests to improve coverage testing
- misc/non_compliant: add tests to improve coverage testing
- basics: add test for break from within try within a for-loop
- basics: add a test file for overriding special methods
- basics/special_methods: enable tests for extra special methods
- uzlib_decompress: actually test raw DEFLATE stream
- run-tests: disable thread/thread_lock4.py on Travis
- run-tests: disable thread/stress_heap.py when running on Travis
- cmdline: add test for -O option to check optimisation value
- extmod/vfs_fat_ramdisk: add tests for VFS.umount()
- run-tests: disable thread_gc1.py test on Travis
- unix/extra_coverage: add test for str/bytes with invalid hash
- extmod: add test for uzlib.DecompIO
- extmod: add a test for framebuf module, tested by coverage build
- extmod/uzlib_decompio: add zlib bitstream testcases
- extmod/framebuf1: add tests for scrolling in the x-direction
- run-tests: disable thread/stress_recurse.py test on Travis

unix port:
- mpconfigport.h: don't include stdio.h on MacOS
- when find'ing frozen files don't use extra slash, do follow symlinks

qemu-arm port:
- enable MICROPY_PY_ALL_SPECIAL_METHODS

stmhal port:
- boards: update STM32L476 pin defs to include ADC channels
- adc.c: get ADC working on STM32L4 MCUs
- fix timer capture/compare interrupt handling for TIM1 and TIM8
- remove obsolete code for special handling of TIM3 irq settings
- make ADC channel 16 available on L4 MCUs
- update pin print to print new constants
- modusocket: set self->nic to MP_OBJ_NULL after socket close
- update boot.py files to use VCP instead of CDC
- spi: factor out SPI transfer code to a single function
- spi: support new machine SPI methods in legacy SPI object
- add machine.WDT class
- set STM32F7DISC CPU Frequency to 216 MHz
- dac: fix DAC (re-)initialisation by resetting DMA
- wdt: implement keyword args to WDT constructor
- modmachine: implement machine.reset_cause() function, and consts
- machine.POWER_ON is renamed to machine.PWRON_RESET
- when find'ing frozen files don't use extra slash, do follow symlinks

cc3200 port:
- add machine.PWRON_RESET constant (machine.POWER_ON is now deprecated)

teensy port:
- fix execution of frozen boot.py and main.py

esp8266 port:
- fix reading of pin object for GPIO16; Pin(16) now works as an input
- PULL_UP is not supported on Pin(16), so raise an exception in this case
- enable support for all special methods
- modpybhspi: add a HSPI module for hardware SPI support
- modmachinespi: add a factory method for SoftSPI/HSPI
- esp_mphal: no longer disable watchdog on startup
- modpybrtc: use 64-bit arithmetic when computing alarm expiry
- hspi: enable duplex operation of hardware SPI
- modous: add os.umount method to unmount a filesystem
- modmachinewdt: implement machine.WDT class
- modules: split onewire.py into OneWire and DS18X20 driver
- modules/onewire: change onewire.read() to onewire.readinto()
- modules/ds18x20.py: add support for DS18S20 devices
- modpybspi: use generic SPI helper methods to implement SPI
- modpybhspi: simplify HSPI driver by using 1 function for xfers
- modmachinewdt: add .deinit() method
- modmachine: add WDT_RESET and SOFT_RESET constants
- modmachine: don't expose internal SoftSPI and HSPI classes
- modmachine: simplify SPI class implementation multiplexing
- espneopixel: disable IRQs during eps.neopixel_write
- modnetwork: fix wlan.scan() method so it returns all networks
- modmachine: map PWR_ON_RESET to vendor's REASON_DEFAULT_RST
- machine.PWR_ON_RESET is renamed to machine.PWRON_RESET
- when find'ing frozen files don't use extra slash, do follow symlinks

docs:
- esp8266/tutorial/pins: fix typo in commands for pin input mode
- esp8266/intro: add command to install esptool.py 1.0.1 via pip
- library/machine.WDT: add note that WDT is only available on WiPy
- esp8266/quickref: fix and update the SPI docs
- esp8266: update quickref and tutorial for OneWire/DS18X20 driver
- pyboard: update USB mouse tutorial to use VCP instead of CDC
- pyboard: update USB mouse tutorial to use pyb.USB_HID()
- library: add reference for pyb.usb_mode and pyb.USB_HID
- pyboard/quickref: add links to pinouts for other pyboard variants
- pyboard/quickref: add section on "delay and timing" for utime mod
- esp8266/quickref: add internal links to docs for some modules
- esp8266/quickref: update information on SPI classes
- esp8266/quickref: further improvements for SPI subsections
- library/machine.WDT: add that WDT is available on pyboard
- reference/isr_rules.rst: two minor additions to docs for using ISR

misc:
- add *.pyc to .gitignore, because Python 2 doesn't use __pycache__
- build mpy-cross as part of the Travis process

9 files changed, 199 insertions, 0 deletions

diff --git a/tests/basics/builtin_hash.py b/tests/basics/builtin_hash.py
index c9731a3b59..ffea08e575 100644
--- a/tests/basics/builtin_hash.py
+++ b/tests/basics/builtin_hash.py
@@ -3,6 +3,7 @@
 print(hash(False))
 print(hash(True))
 print({():1}) # hash tuple
+print({(1,):1}) # hash non-empty tuple
 print({1 << 66:1}) # hash big int
 print({-(1 << 66):2}) # hash negative big int
 print(hash in {hash:1}) # hash function
diff --git a/tests/basics/list1.py b/tests/basics/list1.py
index c8317baa3d..fa426c0e58 100644
--- a/tests/basics/list1.py
+++ b/tests/basics/list1.py
@@ -22,3 +22,9 @@ print(x)
 print(x[1:])
 print(x[:-1])
 print(x[2:3])
+
+# unsupported type on RHS of add
+try:
+    [] + None
+except TypeError:
+    print('TypeError')
diff --git a/tests/basics/list_mult.py b/tests/basics/list_mult.py
index 16948f74c2..548f88534e 100644
--- a/tests/basics/list_mult.py
+++ b/tests/basics/list_mult.py
@@ -10,3 +10,9 @@ for i in (-4, -2, 0, 2, 4):
 a = [1, 2, 3]
 c = a * 3
 print(a, c)
+
+# unsupported type on RHS
+try:
+    [] * None
+except TypeError:
+    print('TypeError')
diff --git a/tests/basics/list_pop.py b/tests/basics/list_pop.py
index bb2ccc6d67..87ed456f85 100644
--- a/tests/basics/list_pop.py
+++ b/tests/basics/list_pop.py
@@ -9,3 +9,9 @@ except IndexError:
     print("IndexError raised")
 else:
     raise AssertionError("No IndexError raised")
+
+# popping such that list storage shrinks (tests implementation detail of uPy)
+l = list(range(20))
+for i in range(len(l)):
+    l.pop()
+print(l)
diff --git a/tests/basics/slice_bignum.py b/tests/basics/slice_bignum.py
new file mode 100644
index 0000000000..cc820522b0
--- /dev/null
+++ b/tests/basics/slice_bignum.py
@@ -0,0 +1,5 @@
+# test slicing when arguments are bignums
+
+print(list(range(10))[(1<<66)>>65:])
+print(list(range(10))[:(1<<66)>>65])
+print(list(range(10))[::(1<<66)>>65])
diff --git a/tests/basics/special_methods.py b/tests/basics/special_methods.py
new file mode 100644
index 0000000000..1df7a7c4c7
--- /dev/null
+++ b/tests/basics/special_methods.py
@@ -0,0 +1,146 @@
+class Cud():
+
+    def __init__(self):
+        print("__init__ called")
+
+    def __repr__(self):
+        print("__repr__ called")
+        return ""
+
+    def __lt__(self, other):
+        print("__lt__ called")
+
+    def __le__(self, other):
+        print("__le__ called")
+
+    def __eq__(self, other):
+        print("__eq__ called")
+
+    def __ne__(self, other):
+        print("__ne__ called")
+
+    def __ge__(self, other):
+        print("__ge__ called")
+
+    def __gt__(self, other):
+        print("__gt__ called")
+
+    def __abs__(self):
+        print("__abs__ called")
+
+    def __add__(self, other):
+        print("__add__ called")
+
+    def __and__(self, other):
+        print("__and__ called")
+
+    def __floordiv__(self, other):
+        print("__floordiv__ called")
+
+    def __index__(self, other):
+        print("__index__ called")
+
+    def __inv__(self):
+        print("__inv__ called")
+
+    def __invert__(self):
+        print("__invert__ called")
+
+    def __lshift__(self, val):
+        print("__lshift__ called")
+
+    def __mod__(self, val):
+        print("__mod__ called")
+
+    def __mul__(self, other):
+        print("__mul__ called")
+
+    def __matmul__(self, other):
+        print("__matmul__ called")
+
+    def __neg__(self):
+        print("__neg__ called")
+
+    def __or__(self, other):
+        print("__or__ called")
+
+    def __pos__(self):
+        print("__pos__ called")
+
+    def __pow__(self, val):
+        print("__pow__ called")
+
+    def __rshift__(self, val):
+        print("__rshift__ called")
+
+    def __sub__(self, other):
+        print("__sub__ called")
+
+    def __truediv__(self, other):
+        print("__truediv__ called")
+
+    def __div__(self, other):
+        print("__div__ called")
+
+    def __xor__(self, other):
+        print("__xor__ called")
+
+    def __iadd__(self, other):
+        print("__iadd__ called")
+        return self
+
+    def __isub__(self, other):
+        print("__isub__ called")
+        return self
+
+cud1 = Cud()
+cud2 = Cud()
+
+str(cud1)
+cud1 < cud2
+cud1 <= cud2
+cud1 == cud2
+cud1 >= cud2
+cud1 > cud2
+cud1 + cud2
+cud1 - cud2
+
+# the following require MICROPY_PY_ALL_SPECIAL_METHODS
++cud1
+-cud1
+~cud1
+cud1 * cud2
+cud1 / cud2
+cud2 // cud1
+cud1 += cud2
+cud1 -= cud2
+
+# TODO: the following operations are not supported on every ports
+#
+# ne is not supported, !(eq) is called instead
+#cud1 != cud2
+#
+# binary and is not supported
+# cud1 & cud2
+#
+# binary lshift is not supported
+# cud1<<1
+#
+# modulus is not supported
+# cud1 % 2
+#
+# binary or is not supported
+# cud1 | cud2
+#
+# pow is not supported
+# cud1**2
+#
+# rshift is not suported
+# cud1>>1
+#
+# xor is not supported
+# cud1^cud2
+#
+# in the followin test, cpython still calls __eq__
+# cud3=cud1
+# cud3==cud1
diff --git a/tests/basics/try_finally_loops.py b/tests/basics/try_finally_loops.py
index 28a8373dc0..06a6b4a0ce 100644
--- a/tests/basics/try_finally_loops.py
+++ b/tests/basics/try_finally_loops.py
@@ -33,3 +33,11 @@ for i in range(4):
         print('here')
     finally:
         print('finnaly 3')
+
+# break from within try-finally, within for-loop
+for i in [1]:
+    try:
+        print(i)
+        break
+    finally:
+        print('finally 4')
diff --git a/tests/basics/tuple1.py b/tests/basics/tuple1.py
index 53eac2a306..2993391d53 100644
--- a/tests/basics/tuple1.py
+++ b/tests/basics/tuple1.py
@@ -16,3 +16,18 @@ print(x[:-1])
 print(x[2:3])
 
 print(x + (10, 100, 10000))
+
+# construction of tuple from large iterator (tests implementation detail of uPy)
+print(tuple(range(20)))
+
+# unsupported unary operation
+try:
+    +()
+except TypeError:
+    print('TypeError')
+
+# unsupported type on RHS of add
+try:
+    () + None
+except TypeError:
+    print('TypeError')
diff --git a/tests/basics/tuple_mult.py b/tests/basics/tuple_mult.py
index 0f52bce44e..b128b29689 100644
--- a/tests/basics/tuple_mult.py
+++ b/tests/basics/tuple_mult.py
@@ -10,3 +10,9 @@ for i in (-4, -2, 0, 2, 4):
 a = (1, 2, 3)
 c = a * 3
 print(a, c)
+
+# unsupported type on RHS
+try:
+    () * None
+except TypeError:
+    print('TypeError')