Merge tag 'v1.8.3' into parse-bytecode

Many small improvements and additions, with btree support in ESP8266 port

This release brings various improvements and additions to the core,
extended modules and the ESP8266 port, as well as enhancements to the docs.
There is now a "threshold" function in the gc module for the user to
configure the garbage collector to run earlier than usual, in order to help
reduce fragmentation of the heap.  The btree module is now available in the
ESP8266 port, and there is improved WebREPL file transfer: get file is now
non-blocking (this change requires an update of the client software).

py core:
- nlrx64.S: prefix mp_thread_get_state with an underscore on Mac
- stream: implement 2- and 3-arg write() method as an extension to CPython
- gc: implement GC running by allocation threshold, user configurable
- fix nlrthumb.c when DEBUG=1 is defined
- obj: issue a warning when str and bytes objects are compared
- mpconfig.h: fix description for MICROPY_PY_STR_BYTES_CMP_WARN
- objstrunicode: str_index_to_ptr: should handle bytes too
- objstrunicode: str_index_to_ptr: implement positive indexing properly
- stream: implement generic flush() method, in terms of C-level ioctl
- objstringio: add MP_STREAM_FLUSH ioctl and flush() method
- objstringio: implement MP_STREAM_SEEK ioctl and add seek() method
- stream: add adapter methods with POSIX-compatible signatures
- mpconfig.h: add MICROPY_STREAMS_POSIX_API setting
- py.mk: extra switches to build "embedded" BerkeleyDB BTree lib
- mkrules.mk: allow to add more items for "clean" target using CLEAN_EXTRA
- objstr: make .partition()/.rpartition() methods configurable
- mpconfig.h: define MP_ALWAYSINLINE for reuse

extmod:
- modussl: rename to modussl_axtls.c, to allow impl using other SSL libs
- modussl_axtls: further changes to allow alternative SSL modules
- modussl_axtls: add dummy setblocking() method
- vfs_fat: implement rmdir() method (aliases to unlink)
- modussl_axtls: use mp_stream_close() method
- modwebrepl: use mp_stream_close() method
- modbtree: check __bt_open() return value for error
- modbtree: switch to accepting stream object instead of filename
- modbtree: implement __contains__ operation
- modwebrepl: factor out "GET" iteration to write_file_chunk()
- modwebrepl: make GET_FILE operation non-blocking
- modbtree: open(): add option kwargs
- modwebsocket: add readline method
- modwebsocket: make compatible with non-default object models
- modwebsocket: use mp_rom_map_elem_t and friends
- modwebrepl: add GET_VER operation to query MicroPython version

lib:
- axtls: upgrade to axTLS 1.5.4 + MicroPython patchset
- axtls: update, fixes esp8266 build
- utils/stdout_helpers: move from minimal/uart_extra.c for reuse
- embed/abort_: implementation of abort_() function raising uPy exception
- berkeley-db-1.xx: switch to "embedded" branch
- berkeley-db-1.xx: update, sets default page size to 4096

tests:
- add test for extended arguments to stream .write() method
- extmod/machine_pinbase: skip if machine.PinBase is not available
- bytes_compare: rework test for bytes <-> str comparison
- extmod/btree1: close database at the end of test
- unicode_subscr.py: detailed test for subscripting unicode strings
- run-tests: make "regex'ed .exp" facility available to device tests
- io/bytesio_ext: test for .seek()/.flush() on BytesIO
- extmod/btree1: tests against in-memory DB (using io.BytesIO)
- machine_mem.py: too non-portable, rework as an example for unix port
- machine1: revamp to work with unix port (which has "umachine")
- basics: bytes/str.partition/rpartition are now optional

mpy-cross:
- fix Makefile to handle gc-sections linker flags on OS X
- use binary file translation mode for creating mpy files on windows
- fix mingw and msys2 compilation

minimal port:
- disable MICROPY_GC_ALLOC_THRESHOLD

unix port:
- mpthreadport: adjust minimum thread stack, and stack limit check
- cache libaxtls.a in local build dir
- disable MICROPY_GC_ALLOC_THRESHOLD for minimal build
- enable MICROPY_PY_STR_BYTES_CMP_WARN
- mpconfigport.h: include stdio.h by default
- file: implement MP_STREAM_FLUSH ioctl
- file: ioctl(): check that file is open before operations
- file: fdfile_ioctl(): fix argument to check_fd_is_open()
- file: use generic stream flush() method
- enable websocket module
- moduselect: allow poll.register(), etc. accept fd-like objects

stmhal port:
- fix malloc when used with external libraries
- make SPI NSS pin definition optional
- fix I2C mappings for STM32F429DISC board

esp8266 port:
- select axTLS for SSL implementation, following recent refactor
- moduos: add rmdir() function
- cache Xtensa-built libaxtls.a in local build dir
- enable MICROPY_PY_STR_BYTES_CMP_WARN
- dupterm_task_init() should be called before running _boot.py, etc
- _boot.py: set GC alloc threshold to half of heap size
- _boot.py: decrease GC alloc threshold to quarter of heap size
- modpybuart: fix UART parity setting
- axtls_helpers: remove abort_(), now in lib/embed/
- mpconfigport.h: include sys/types.h for POSIX types definitions
- esp_mphal: implement libc's errno
- enable btree module
- eagle.rom.addr.v6.ld: add Enable_QMode symbol from SDK 2.0.0
- make APA102 driver inclusion configurable
- makeimg.py: store firmware size as last 4 bytes of padding area
- makeimg.py: append md5 hash to the generated binary
- modesp: add check_fw() function to check integrity of the firmware
- scripts/port_diag.py: include esp.check_fw() call
- flashbdev: reserve extra sysparam sector for SDK 2.0.0 compatibility
- scripts/inisetup: add commented-out call to esp.osdebug(None)
- modmachine: implement idle() function
- esp_mphal.h: add mp_hal_ticks_cpu() for reuse
- modutime: actually implement ticks_cpu()
- modmachine: implement dummy sleep() function
- tutorial/intro: reword para abou -fm dio switch
- modules/flashbdev: start filesystem at 0x90000
- esp8266.ld: increase firmware image size to 0x90000 (576K)

docs:
- add DHT to ESP8266 Quick Ref and Tutorial
- fix some spelling mistakes
- array: document array module
- library/index: include array module in ToC
- esp8266/intro: rename to "Getting started" from "Introduction"
- esp8266/intro: add troubleshooting section
- esp8266/quickref: link to installation instructions
- esp8266/tutorial/intro: add anchor for link from quickeref
- esp8266/intro: focus on hazards of unearthed power wrt electronics
- uio: mention seek()/flush() support for io.BytesIO

misc:
- logo/1bit-logo A black & white version of the logo

examples:
- http_server*: update for buffered-like streams (read line by line)
- embedding: example for embedding MicroPython in an app
- embedding: add README
- http_client*: be sure to close socket
- network: split recv- and read-based HTTP servers

qemu-arm:
- enable gcc LTO option for nlrthumb.c

4 files changed, 131 insertions, 5 deletions

diff --git a/docs/esp8266/tutorial/dht.rst b/docs/esp8266/tutorial/dht.rst
new file mode 100644
index 0000000000..1602e8a337
--- /dev/null
+++ b/docs/esp8266/tutorial/dht.rst
@@ -0,0 +1,65 @@
+Temperature and Humidity
+========================
+
+DHT (Digital Humidity & Temperature) sensors are low cost digital sensors with
+capacitive humidity sensors and thermistors to measure the surrounding air.
+They feature a chip that handles analog to digital conversion and provide a
+1-wire interface. Newer sensors additionally provide an I2C interface.
+
+The DHT11 (blue) and DHT22 (white) sensors provide the same 1-wire interface,
+however, the DHT22 requires a separate object as it has more complex
+calculation. DHT22 have 1 decimal place resolution for both humidity and
+temperature readings. DHT11 have whole number for both.
+
+A custom 1-wire protocol, which is different to Dallas 1-wire, is used to get
+the measurements from the sensor. The payload consists of a humidity value,
+a temperature value and a checksum.
+
+To use the 1-wire interface, construct the objects referring to their data pin::
+
+    >>> import dht
+    >>> import machine
+    >>> d = dht.DHT11(machine.Pin(4))
+
+    >>> import dht
+    >>> import machine
+    >>> d = dht.DHT22(machine.Pin(4))
+
+Then measure and read their values with::
+
+    >>> d.measure()
+    >>> d.temperature()
+    >>> d.humidity()
+
+Values returned from ``temperature()`` are in degrees Celsius and values
+returned from ``humidity()`` are a percentage of relative humidity.
+
+The DHT11 can be called no more than once per second and the DHT22 once every
+two seconds for most accurate results. Sensor accuracy will degrade over time.
+Each sensor supports a different operating range. Refer to the product
+datasheets for specifics.
+
+In 1-wire mode, only three of the four pins are used and in I2C mode, all four
+pins are used. Older sensors may still have 4 pins even though they do not
+support I2C. The 3rd pin is simply not connected.
+
+Pin configurations:
+
+Sensor without I2C in 1-wire mode (eg. DHT11, DHT22, AM2301, AM2302):
+
+    1=VDD, 2=Data, 3=NC, 4=GND
+
+Sensor with I2C in 1-wire mode (eg. DHT12, AM2320, AM2321, AM2322):
+
+    1=VDD, 2=Data, 3=GND, 4=GND
+
+Sensor with I2C in I2C mode (eg. DHT12, AM2320, AM2321, AM2322):
+
+    1=VDD, 2=SDA, 3=GND, 4=SCL
+
+You should use pull-up resistors for the Data, SDA and SCL pins.
+
+To make newer I2C sensors work in backwards compatible 1-wire mode, you must
+connect both pins 3 and 4 to GND. This disables the I2C interface.
+
+DHT22 sensors are now sold under the name AM2302 and are otherwise identical.
diff --git a/docs/esp8266/tutorial/index.rst b/docs/esp8266/tutorial/index.rst
index 1a00afd853..39b4592600 100644
--- a/docs/esp8266/tutorial/index.rst
+++ b/docs/esp8266/tutorial/index.rst
@@ -29,4 +29,5 @@ to `<https://www.python.org>`__.
    powerctrl.rst
    onewire.rst
    neopixel.rst
+   dht.rst
    nextsteps.rst
diff --git a/docs/esp8266/tutorial/intro.rst b/docs/esp8266/tutorial/intro.rst
index a2c5d1838e..8c356b913f 100644
--- a/docs/esp8266/tutorial/intro.rst
+++ b/docs/esp8266/tutorial/intro.rst
@@ -1,5 +1,7 @@
-Introduction to MicroPython on the ESP8266
-==========================================
+.. _intro:
+
+Getting started with MicroPython on the ESP8266
+===============================================
 
 Using MicroPython is a great way to get the most of your ESP8266 board.  And
 vice versa, the ESP8266 chip is a great platform for using MicroPython.  This
@@ -74,8 +76,9 @@ PC.  You may also need to reduce the baudrate if you get errors when flashing
 (eg down to 115200).  The filename of the firmware should also match the file
 that you have.
 
-If you have a NodeMCU board, you may need to use the following command to deploy
-the firmware (note the "-fm dio" option)::
+For some boards with a particular FlashROM configuration (e.g. some variants of
+a NodeMCU board) you may need to use the following command to deploy
+the firmware (note the ``-fm dio`` option)::
 
     esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=8m -fm dio 0 esp8266-2016-05-03-v1.8.bin
 
@@ -100,3 +103,60 @@ be the same everytime, and most likely different for all ESP8266 chips).  The
 password for the WiFi is micropythoN (note the upper-case N).  Its IP address
 will be 192.168.4.1 once you connect to its network.  WiFi configuration will
 be discussed in more detail later in the tutorial.
+
+Troubleshooting installation problems
+-------------------------------------
+
+If you experience problems during flashing or with running firmware immediately
+after it, here are troubleshooting recommendations:
+
+* Be aware of and try to exclude hardware problems. There are 2 common problems:
+  bad power source quality and worn-out/defective FlashROM. Speaking of power
+  source, not just raw amperage is important, but also low ripple and noise/EMI
+  in general. If you experience issues with self-made or wall-wart style power
+  supply, try USB power from a computer. Unearthed power supplies are also known
+  to cause problems as they source of increased EMI (electromagnetic interference)
+  - at the very least, and may lead to electrical devices breakdown. So, you are
+  advised to avoid using unearthed power connections when working with ESP8266
+  and other boards. In regard to FlashROM hardware problems, there are independent
+  (not related to MicroPython in any way) reports
+  `(e.g.) <http://internetofhomethings.com/homethings/?p=538>`_
+  that on some ESP8266 modules, FlashROM can be programmed as little as 20 times
+  before programming errors occur. This is *much* less than 100,000 programming
+  cycles cited for FlashROM chips of a type used with ESP8266 by reputable
+  vendors, which points to either production rejects, or second-hand worn-out
+  flash chips to be used on some (apparently cheap) modules/boards. You may want
+  to use your best judgement about source, price, documentation, warranty,
+  post-sales support for the modules/boards you purchase.
+
+* The flashing instructions above use flashing speed of 460800 baud, which is
+  good compromise between speed and stability. However, depending on your
+  module/board, USB-UART convertor, cables, host OS, etc., the above baud
+  rate may be too high and lead to errors. Try a more common 115200 baud
+  rate instead in such cases.
+
+* The ``--flash_size`` option in the commands above is mandatory. Omitting
+  it will lead to a corrupted firmware.
+
+* To catch incorrect flash content (e.g. from a defective sector on a chip),
+  add ``--verify`` switch to the commands above.
+
+* Additionally, you can check the firmware integrity from a MicroPython REPL
+  prompt (assuming you were able to flash it and ``--verify`` option doesn't
+  report errors)::
+    import esp
+    esp.check_fw()
+  If the last output value is True, the firmware is OK. Otherwise, it's
+  corrupted and need to be reflashed correctly.
+
+* If you experience any issues with another flashing application (not
+  esptool.py), try esptool.py, it is a generally accepted flashing
+  application in the ESP8266 community.
+
+* If you still experience problems with even flashing the firmware, please
+  refer to esptool.py project page, https://github.com/themadinventor/esptool
+  for additional documentation and bug tracker where you can report problems.
+
+* If you are able to flash firmware, but ``--verify`` option or
+  ``esp.check_fw()`` return errors even after multiple retries, you
+  may have a defective FlashROM chip, as explained above.
diff --git a/docs/esp8266/tutorial/repl.rst b/docs/esp8266/tutorial/repl.rst
index 078f31357c..338e9fdd8f 100644
--- a/docs/esp8266/tutorial/repl.rst
+++ b/docs/esp8266/tutorial/repl.rst
@@ -36,7 +36,7 @@ WebREPL - a prompt over WiFi
 WebREPL allows you to use the Python prompt over WiFi, connecting through a
 browser. The latest versions of Firefox and Chrome are supported.
 
-For your convinience, WebREPL client is hosted at
+For your convenience, WebREPL client is hosted at
 `<http://micropython.org/webrepl>`__ . Alternatively, you can install it
 locally from the the GitHub repository
 `<https://github.com/micropython/webrepl>`__ .