diff options
145 files changed, 1880 insertions, 384 deletions
diff --git a/cc3200/mpconfigport.h b/cc3200/mpconfigport.h index fbf227b86e..331db8100e 100644 --- a/cc3200/mpconfigport.h +++ b/cc3200/mpconfigport.h @@ -198,14 +198,6 @@ typedef long mp_off_t; // disabling/enabling and sleep mode enter/exit #include "cc3200_asm.h" -// There is no classical C heap in bare-metal ports, only Python -// garbage-collected heap. For completeness, emulate C heap via -// GC heap. Note that MicroPython core never uses malloc() and friends, -// so these defines are mostly to help extension module writers. -#define malloc gc_alloc -#define free gc_free -#define realloc gc_realloc - // We need to provide a declaration/definition of alloca() #include <alloca.h> diff --git a/docs/README.md b/docs/README.md index 2602a15225..faf386710a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -35,6 +35,6 @@ This can be achieved with: make MICROPY_PORT=<port_name> latexpdf but require rather complete install of LaTeX with various extensions. On -Debiab/Ubuntu, try (500MB+ download): +Debian/Ubuntu, try (500MB+ download): apt-get install texlive-latex-recommended texlive-latex-extra diff --git a/docs/conf.py b/docs/conf.py index ba69cf7f7a..fbc89afc6a 100755 --- a/docs/conf.py +++ b/docs/conf.py @@ -99,7 +99,7 @@ copyright = '2014-2016, Damien P. George and contributors' # The short X.Y version. version = '1.8' # The full version, including alpha/beta/rc tags. -release = '1.8.2' +release = '1.8.3' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/esp8266/general.rst b/docs/esp8266/general.rst index 313e6074c1..3ffb2ff339 100644 --- a/docs/esp8266/general.rst +++ b/docs/esp8266/general.rst @@ -52,7 +52,7 @@ For your convenience, some of technical specifications are provided below: external FlashROM, UART, deep sleep wake-up, etc.) * UART: One RX/TX UART (no hardware handshaking), one TX-only UART. * SPI: 2 SPI interfaces (one used for FlashROM). -* I2C: No native extenal I2C (bitbang implementation available on any pins). +* I2C: No native external I2C (bitbang implementation available on any pins). * I2S: 1. * Programming: using BootROM bootloader from UART. Due to external FlashROM and always-available BootROM bootloader, ESP8266 is not brickable. diff --git a/docs/esp8266/quickref.rst b/docs/esp8266/quickref.rst index 779248369f..48543dfab6 100644 --- a/docs/esp8266/quickref.rst +++ b/docs/esp8266/quickref.rst @@ -9,6 +9,12 @@ Quick reference for the ESP8266 The Adafruit Feather HUZZAH board (image attribution: Adafruit). +Installing MicroPython +---------------------- + +See the corresponding section of tutorial: :ref:`intro`. It also includes +a troubleshooting subsection. + General board control --------------------- @@ -291,6 +297,24 @@ For low-level driving of an APA102:: import esp esp.apa102_write(clock_pin, data_pin, rgbi_buf) +DHT driver +---------- + +The DHT driver is implemented in software and works on all pins:: + + import dht + import machine + + d = dht.DHT11(machine.Pin(4)) + d.measure() + d.temperature() # eg. 23 (°C) + d.humidity() # eg. 41 (% RH) + + d = dht.DHT22(machine.Pin(4)) + d.measure() + d.temperature() # eg. 23.6 (°C) + d.humidity() # eg. 41.3 (% RH) + WebREPL (web browser interactive prompt) ---------------------------------------- diff --git a/docs/esp8266/tutorial/dht.rst b/docs/esp8266/tutorial/dht.rst new file mode 100644 index 0000000000..1602e8a337 --- /dev/null +++ b/docs/esp8266/tutorial/dht.rst @@ -0,0 +1,65 @@ +Temperature and Humidity +======================== + +DHT (Digital Humidity & Temperature) sensors are low cost digital sensors with +capacitive humidity sensors and thermistors to measure the surrounding air. +They feature a chip that handles analog to digital conversion and provide a +1-wire interface. Newer sensors additionally provide an I2C interface. + +The DHT11 (blue) and DHT22 (white) sensors provide the same 1-wire interface, +however, the DHT22 requires a separate object as it has more complex +calculation. DHT22 have 1 decimal place resolution for both humidity and +temperature readings. DHT11 have whole number for both. + +A custom 1-wire protocol, which is different to Dallas 1-wire, is used to get +the measurements from the sensor. The payload consists of a humidity value, +a temperature value and a checksum. + +To use the 1-wire interface, construct the objects referring to their data pin:: + + >>> import dht + >>> import machine + >>> d = dht.DHT11(machine.Pin(4)) + + >>> import dht + >>> import machine + >>> d = dht.DHT22(machine.Pin(4)) + +Then measure and read their values with:: + + >>> d.measure() + >>> d.temperature() + >>> d.humidity() + +Values returned from ``temperature()`` are in degrees Celsius and values +returned from ``humidity()`` are a percentage of relative humidity. + +The DHT11 can be called no more than once per second and the DHT22 once every +two seconds for most accurate results. Sensor accuracy will degrade over time. +Each sensor supports a different operating range. Refer to the product +datasheets for specifics. + +In 1-wire mode, only three of the four pins are used and in I2C mode, all four +pins are used. Older sensors may still have 4 pins even though they do not +support I2C. The 3rd pin is simply not connected. + +Pin configurations: + +Sensor without I2C in 1-wire mode (eg. DHT11, DHT22, AM2301, AM2302): + + 1=VDD, 2=Data, 3=NC, 4=GND + +Sensor with I2C in 1-wire mode (eg. DHT12, AM2320, AM2321, AM2322): + + 1=VDD, 2=Data, 3=GND, 4=GND + +Sensor with I2C in I2C mode (eg. DHT12, AM2320, AM2321, AM2322): + + 1=VDD, 2=SDA, 3=GND, 4=SCL + +You should use pull-up resistors for the Data, SDA and SCL pins. + +To make newer I2C sensors work in backwards compatible 1-wire mode, you must +connect both pins 3 and 4 to GND. This disables the I2C interface. + +DHT22 sensors are now sold under the name AM2302 and are otherwise identical. diff --git a/docs/esp8266/tutorial/index.rst b/docs/esp8266/tutorial/index.rst index 1a00afd853..39b4592600 100644 --- a/docs/esp8266/tutorial/index.rst +++ b/docs/esp8266/tutorial/index.rst @@ -29,4 +29,5 @@ to `<https://www.python.org>`__. powerctrl.rst onewire.rst neopixel.rst + dht.rst nextsteps.rst diff --git a/docs/esp8266/tutorial/intro.rst b/docs/esp8266/tutorial/intro.rst index a2c5d1838e..8c356b913f 100644 --- a/docs/esp8266/tutorial/intro.rst +++ b/docs/esp8266/tutorial/intro.rst @@ -1,5 +1,7 @@ -Introduction to MicroPython on the ESP8266 -========================================== +.. _intro: + +Getting started with MicroPython on the ESP8266 +=============================================== Using MicroPython is a great way to get the most of your ESP8266 board. And vice versa, the ESP8266 chip is a great platform for using MicroPython. This @@ -74,8 +76,9 @@ PC. You may also need to reduce the baudrate if you get errors when flashing (eg down to 115200). The filename of the firmware should also match the file that you have. -If you have a NodeMCU board, you may need to use the following command to deploy -the firmware (note the "-fm dio" option):: +For some boards with a particular FlashROM configuration (e.g. some variants of +a NodeMCU board) you may need to use the following command to deploy +the firmware (note the ``-fm dio`` option):: esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=8m -fm dio 0 esp8266-2016-05-03-v1.8.bin @@ -100,3 +103,60 @@ be the same everytime, and most likely different for all ESP8266 chips). The password for the WiFi is micropythoN (note the upper-case N). Its IP address will be 192.168.4.1 once you connect to its network. WiFi configuration will be discussed in more detail later in the tutorial. + +Troubleshooting installation problems +------------------------------------- + +If you experience problems during flashing or with running firmware immediately +after it, here are troubleshooting recommendations: + +* Be aware of and try to exclude hardware problems. There are 2 common problems: + bad power source quality and worn-out/defective FlashROM. Speaking of power + source, not just raw amperage is important, but also low ripple and noise/EMI + in general. If you experience issues with self-made or wall-wart style power + supply, try USB power from a computer. Unearthed power supplies are also known + to cause problems as they source of increased EMI (electromagnetic interference) + - at the very least, and may lead to electrical devices breakdown. So, you are + advised to avoid using unearthed power connections when working with ESP8266 + and other boards. In regard to FlashROM hardware problems, there are independent + (not related to MicroPython in any way) reports + `(e.g.) <http://internetofhomethings.com/homethings/?p=538>`_ + that on some ESP8266 modules, FlashROM can be programmed as little as 20 times + before programming errors occur. This is *much* less than 100,000 programming + cycles cited for FlashROM chips of a type used with ESP8266 by reputable + vendors, which points to either production rejects, or second-hand worn-out + flash chips to be used on some (apparently cheap) modules/boards. You may want + to use your best judgement about source, price, documentation, warranty, + post-sales support for the modules/boards you purchase. + +* The flashing instructions above use flashing speed of 460800 baud, which is + good compromise between speed and stability. However, depending on your + module/board, USB-UART convertor, cables, host OS, etc., the above baud + rate may be too high and lead to errors. Try a more common 115200 baud + rate instead in such cases. + +* The ``--flash_size`` option in the commands above is mandatory. Omitting + it will lead to a corrupted firmware. + +* To catch incorrect flash content (e.g. from a defective sector on a chip), + add ``--verify`` switch to the commands above. + +* Additionally, you can check the firmware integrity from a MicroPython REPL + prompt (assuming you were able to flash it and ``--verify`` option doesn't + report errors):: + import esp + esp.check_fw() + If the last output value is True, the firmware is OK. Otherwise, it's + corrupted and need to be reflashed correctly. + +* If you experience any issues with another flashing application (not + esptool.py), try esptool.py, it is a generally accepted flashing + application in the ESP8266 community. + +* If you still experience problems with even flashing the firmware, please + refer to esptool.py project page, https://github.com/themadinventor/esptool + for additional documentation and bug tracker where you can report problems. + +* If you are able to flash firmware, but ``--verify`` option or + ``esp.check_fw()`` return errors even after multiple retries, you + may have a defective FlashROM chip, as explained above. diff --git a/docs/esp8266/tutorial/repl.rst b/docs/esp8266/tutorial/repl.rst index 078f31357c..338e9fdd8f 100644 --- a/docs/esp8266/tutorial/repl.rst +++ b/docs/esp8266/tutorial/repl.rst @@ -36,7 +36,7 @@ WebREPL - a prompt over WiFi WebREPL allows you to use the Python prompt over WiFi, connecting through a browser. The latest versions of Firefox and Chrome are supported. -For your convinience, WebREPL client is hosted at +For your convenience, WebREPL client is hosted at `<http://micropython.org/webrepl>`__ . Alternatively, you can install it locally from the the GitHub repository `<https://github.com/micropython/webrepl>`__ . diff --git a/docs/library/array.rst b/docs/library/array.rst new file mode 100644 index 0000000000..f52b4b385e --- /dev/null +++ b/docs/library/array.rst @@ -0,0 +1,30 @@ +:mod:`array` -- arrays of numeric data +====================================== + +.. module:: array + :synopsis: efficient arrays of numeric data + +See `Python array <https://docs.python.org/3/library/array.html>`_ for more +information. + +Supported format codes: ``b``, ``B``, ``h``, ``H``, ``i``, ``I``, ``l``, +``L``, ``q``, ``Q``, ``f``, ``d`` (the latter 2 depending on the +floating-point support). + +Classes +------- + +.. class:: array.array(typecode, [iterable]) + + Create array with elements of given type. Initial contents of the + array are given by an `iterable`. If it is not provided, an empty + array is created. + + .. method:: append(val) + + Append new element to the end of array, growing it. + + .. method:: extend(iterable) + + Append new elements as contained in an iterable to the end of + array, growing it. diff --git a/docs/library/builtins.rst b/docs/library/builtins.rst index d99eadce3e..42768ab999 100644 --- a/docs/library/builtins.rst +++ b/docs/library/builtins.rst @@ -1,7 +1,7 @@ Builtin Functions ================= -All builtin functions are described here. They are alse available via +All builtin functions are described here. They are also available via ``builtins`` module. .. function:: abs() diff --git a/docs/library/esp.rst b/docs/library/esp.rst index 34d3c278d9..0836bba72e 100644 --- a/docs/library/esp.rst +++ b/docs/library/esp.rst @@ -15,7 +15,7 @@ Functions Get or set the sleep type. If the ``sleep_type`` parameter is provided, sets the sleep type to its - value. If the function is called wihout parameters, returns the current + value. If the function is called without parameters, returns the current sleep type. The possible sleep types are defined as constants: diff --git a/docs/library/index.rst b/docs/library/index.rst index 9a105697b7..d502ffdfc6 100644 --- a/docs/library/index.rst +++ b/docs/library/index.rst @@ -51,6 +51,7 @@ library. .. toctree:: :maxdepth: 1 + array.rst builtins.rst cmath.rst gc.rst @@ -75,6 +76,7 @@ library. .. toctree:: :maxdepth: 1 + array.rst builtins.rst cmath.rst gc.rst @@ -99,6 +101,7 @@ library. .. toctree:: :maxdepth: 1 + array.rst builtins.rst gc.rst select.rst @@ -116,6 +119,7 @@ library. .. toctree:: :maxdepth: 1 + array.rst builtins.rst gc.rst math.rst diff --git a/docs/library/machine.Pin.rst b/docs/library/machine.Pin.rst index fdada2f383..db25787f94 100644 --- a/docs/library/machine.Pin.rst +++ b/docs/library/machine.Pin.rst @@ -84,7 +84,7 @@ Methods - ``None`` - no pull up or down resistor. - ``Pin.PULL_UP`` - pull up resistor enabled. - - ``Pin.PULL_DOWN`` - pull down resitor enabled. + - ``Pin.PULL_DOWN`` - pull down resistor enabled. - ``drive`` can be one of: diff --git a/docs/library/machine.RTC.rst b/docs/library/machine.RTC.rst index b6e98e9d1f..6dc8b3e9ab 100644 --- a/docs/library/machine.RTC.rst +++ b/docs/library/machine.RTC.rst @@ -39,7 +39,7 @@ Methods .. method:: RTC.alarm(id, time, /*, repeat=False) - Set the RTC alarm. Time might be either a milllisecond value to program the alarm to + Set the RTC alarm. Time might be either a millisecond value to program the alarm to current time + time_in_ms in the future, or a datetimetuple. If the time passed is in milliseconds, repeat can be set to ``True`` to make the alarm periodic. diff --git a/docs/library/machine.SPI.rst b/docs/library/machine.SPI.rst index f3c95181dc..73b3a3996d 100644 --- a/docs/library/machine.SPI.rst +++ b/docs/library/machine.SPI.rst @@ -45,7 +45,7 @@ Methods respectively. - ``bits`` is the width of each transfer, accepted values are 8, 16 and 32. - ``firstbit`` can be ``SPI.MSB`` only. - - ``pins`` is an optional tupple with the pins to assign to the SPI bus. + - ``pins`` is an optional tuple with the pins to assign to the SPI bus. .. method:: SPI.deinit() diff --git a/docs/library/machine.Timer.rst b/docs/library/machine.Timer.rst index 344d1ea86e..12db58d5c4 100644 --- a/docs/library/machine.Timer.rst +++ b/docs/library/machine.Timer.rst @@ -104,7 +104,7 @@ Methods If only a channel identifier passed, then a previously initialized channel object is returned (or ``None`` if there is no previous channel). - Othwerwise, a TimerChannel object is initialized and returned. + Otherwise, a TimerChannel object is initialized and returned. The operating mode is is the one configured to the Timer object that was used to create the channel. @@ -152,7 +152,7 @@ Methods .. method:: timerchannel.irq(\*, trigger, priority=1, handler=None) - The behavior of this callback is heaviliy dependent on the operating + The behavior of this callback is heavily dependent on the operating mode of the timer channel: - If mode is ``Timer.PERIODIC`` the callback is executed periodically diff --git a/docs/library/machine.UART.rst b/docs/library/machine.UART.rst index 27a8801900..f832cf4664 100644 --- a/docs/library/machine.UART.rst +++ b/docs/library/machine.UART.rst @@ -167,7 +167,7 @@ Methods .. data:: UART.EVEN .. data:: UART.ODD - parity types (anlong with ``None``) + parity types (along with ``None``) .. data:: UART.RX_ANY diff --git a/docs/library/machine.rst b/docs/library/machine.rst index b103ec393d..0f361a7cb7 100644 --- a/docs/library/machine.rst +++ b/docs/library/machine.rst @@ -57,7 +57,7 @@ Power related functions Gates the clock to the CPU, useful to reduce power consumption at any time during short or long periods. Peripherals continue working and execution resumes as soon as any interrupt is triggered (on many ports this includes system timer - interrupt occuring at regular intervals on the order of millisecond). + interrupt occurring at regular intervals on the order of millisecond). .. function:: sleep() @@ -96,7 +96,7 @@ Miscellaneous functions .. function:: unique_id() - Returns a byte string with a unique idenifier of a board/SoC. It will vary + Returns a byte string with a unique identifier of a board/SoC. It will vary from a board/SoC instance to another, if underlying hardware allows. Length varies by hardware (so use substring of a full value if you expect a short ID). In some MicroPython ports, ID corresponds to the network MAC address. diff --git a/docs/library/network.rst b/docs/library/network.rst index b183ef9a4b..0e18106aa8 100644 --- a/docs/library/network.rst +++ b/docs/library/network.rst @@ -237,7 +237,7 @@ For example:: Get or set the PHY mode. If the ``mode`` parameter is provided, sets the mode to its value. If - the function is called wihout parameters, returns the current mode. + the function is called without parameters, returns the current mode. The possible modes are defined as constants: * ``MODE_11B`` -- IEEE 802.11b, @@ -319,7 +319,7 @@ For example:: * ``STAT_WRONG_PASSWORD`` -- failed due to incorrect password, * ``STAT_NO_AP_FOUND`` -- failed because no access point replied, * ``STAT_CONNECT_FAIL`` -- failed due to other problems, - * ``STAT_GOT_IP`` -- connection susccessful. + * ``STAT_GOT_IP`` -- connection successful. .. method:: wlan.isconnected() @@ -329,7 +329,7 @@ For example:: .. method:: wlan.ifconfig([(ip, subnet, gateway, dns)]) - Get/set IP-level network interface paremeters: IP address, subnet mask, + Get/set IP-level network interface parameters: IP address, subnet mask, gateway and DNS server. When called with no arguments, this method returns a 4-tuple with the above information. To set the above values, pass a 4-tuple with the required information. For example:: @@ -343,8 +343,8 @@ For example:: with additional parameters beyond standard IP configuration (as dealt with by ``wlan.ifconfig()``). These include network-specific and hardware-specific parameters. For setting parameters, keyword argument syntax should be used, - multiple parameters can be set at once. For querying, paremeters name should - be quoted as a string, and only one paramter can be queries at time:: + multiple parameters can be set at once. For querying, parameters name should + be quoted as a string, and only one parameter can be queries at time:: # Set WiFi access point name (formally known as ESSID) and WiFi channel ap.config(essid='My AP', channel=11) @@ -397,7 +397,7 @@ For example:: .. note:: The ``WLAN`` constructor is special in the sense that if no arguments besides the id are given, - it will return the already exisiting ``WLAN`` instance without re-configuring it. This is + it will return the already existing ``WLAN`` instance without re-configuring it. This is because ``WLAN`` is a system feature of the WiPy. If the already existing instance is not initialized it will do the same as the other constructors an will initialize it with default values. diff --git a/docs/library/pyb.ADC.rst b/docs/library/pyb.ADC.rst index 2ffd4e16f5..51021fdc1a 100644 --- a/docs/library/pyb.ADC.rst +++ b/docs/library/pyb.ADC.rst @@ -13,7 +13,7 @@ class ADC -- analog to digital conversion adc = pyb.ADC(pin) # create an analog object from a pin val = adc.read() # read an analog value - adc = pyb.ADCAll(resolution) # creale an ADCAll object + adc = pyb.ADCAll(resolution) # create an ADCAll object val = adc.read_channel(channel) # read the given channel val = adc.read_core_temp() # read MCU temperature val = adc.read_core_vbat() # read MCU VBAT diff --git a/docs/library/pyb.DAC.rst b/docs/library/pyb.DAC.rst index 4b9651e27e..fd786b63b6 100644 --- a/docs/library/pyb.DAC.rst +++ b/docs/library/pyb.DAC.rst @@ -79,7 +79,7 @@ Methods .. method:: DAC.triangle(freq) Generate a triangle wave. The value on the DAC output changes at - the given frequency, and the frequence of the repeating triangle wave + the given frequency, and the frequency of the repeating triangle wave itself is 2048 times smaller. .. method:: DAC.write(value) diff --git a/docs/library/pyb.ExtInt.rst b/docs/library/pyb.ExtInt.rst index d8f4d92bb8..814217cef0 100644 --- a/docs/library/pyb.ExtInt.rst +++ b/docs/library/pyb.ExtInt.rst @@ -7,7 +7,7 @@ class ExtInt -- configure I/O pins to interrupt on external events There are a total of 22 interrupt lines. 16 of these can come from GPIO pins and the remaining 6 are from internal sources. -For lines 0 thru 15, a given line can map to the corresponding line from an +For lines 0 through 15, a given line can map to the corresponding line from an arbitrary port. So line 0 can map to Px0 where x is A, B, C, ... and line 1 can map to Px1 where x is A, B, C, ... :: @@ -27,7 +27,7 @@ explanation, along with various techniques for debouncing. Trying to register 2 callbacks onto the same pin will throw an exception. If pin is passed as an integer, then it is assumed to map to one of the -internal interrupt sources, and must be in the range 16 thru 22. +internal interrupt sources, and must be in the range 16 through 22. All other pin objects go through the pin mapper to come up with one of the gpio pins. :: diff --git a/docs/library/pyb.I2C.rst b/docs/library/pyb.I2C.rst index 210b4ccf21..67131feec0 100644 --- a/docs/library/pyb.I2C.rst +++ b/docs/library/pyb.I2C.rst @@ -39,7 +39,7 @@ Printing the i2c object gives you information about its configuration. You can specify a timeout (in ms):: - i2c.send(b'123', timeout=2000) # timout after 2 seconds + i2c.send(b'123', timeout=2000) # timeout after 2 seconds A master must specify the recipient's address:: diff --git a/docs/library/pyb.Pin.rst b/docs/library/pyb.Pin.rst index 154217ebb7..4f589fff89 100644 --- a/docs/library/pyb.Pin.rst +++ b/docs/library/pyb.Pin.rst @@ -231,7 +231,7 @@ Constants class PinAF -- Pin Alternate Functions ====================================== - A Pin represents a physical pin on the microcprocessor. Each pin + A Pin represents a physical pin on the microprocessor. Each pin can have a variety of functions (GPIO, I2C SDA, etc). Each PinAF object represents a particular function for a pin. @@ -240,7 +240,7 @@ Constants x3 = pyb.Pin.board.X3 x3_af = x3.af_list() - x3_af will now contain an array of PinAF objects which are availble on + x3_af will now contain an array of PinAF objects which are available on pin X3. For the pyboard, x3_af would contain: diff --git a/docs/library/pyb.RTC.rst b/docs/library/pyb.RTC.rst index 383ddf20c2..2628554526 100644 --- a/docs/library/pyb.RTC.rst +++ b/docs/library/pyb.RTC.rst @@ -73,7 +73,7 @@ Methods value, which is an integer in the range [-511 : 512]. With one argument it sets the RTC calibration. - The RTC Smooth Calibration mechanism addjusts the RTC clock rate by + The RTC Smooth Calibration mechanism adjusts the RTC clock rate by adding or subtracting the given number of ticks from the 32768 Hz clock over a 32 second period (corresponding to 2^20 clock ticks.) Each tick added will speed up the clock by 1 part in 2^20, or 0.954 diff --git a/docs/library/pyb.Timer.rst b/docs/library/pyb.Timer.rst index ffb092caa3..052bce2efd 100644 --- a/docs/library/pyb.Timer.rst +++ b/docs/library/pyb.Timer.rst @@ -78,7 +78,7 @@ Methods Keyword arguments: - - ``freq`` --- specifies the periodic frequency of the timer. You migh also + - ``freq`` --- specifies the periodic frequency of the timer. You might also view this as the frequency with which the timer goes through one complete cycle. - ``prescaler`` [0-0xffff] - specifies the value to be loaded into the @@ -97,7 +97,7 @@ Methods - ``Timer.UP`` - configures the timer to count from 0 to ARR (default) - ``Timer.DOWN`` - configures the timer to count from ARR down to 0. - - ``Timer.CENTER`` - confgures the timer to count from 0 to ARR and + - ``Timer.CENTER`` - configures the timer to count from 0 to ARR and then back down to 0. - ``div`` can be one of 1, 2, or 4. Divides the timer clock to determine @@ -109,7 +109,7 @@ Methods transitions on complimentary channels (both channels will be inactive) for this time). ``deadtime`` may be an integer between 0 and 1008, with the following restrictions: 0-128 in steps of 1. 128-256 in steps of - 2, 256-512 in steps of 8, and 512-1008 in steps of 16. ``deadime`` + 2, 256-512 in steps of 8, and 512-1008 in steps of 16. ``deadtime`` measures ticks of ``source_freq`` divided by ``div`` clock ticks. ``deadtime`` is only available on timers 1 and 8. @@ -141,7 +141,7 @@ Methods If only a channel number is passed, then a previously initialized channel object is returned (or ``None`` if there is no previous channel). - Othwerwise, a TimerChannel object is initialized and returned. + Otherwise, a TimerChannel object is initialized and returned. Each channel can be configured to perform pwm, output compare, or input capture. All channels share the same underlying timer, which means @@ -183,7 +183,7 @@ Methods - ``polarity`` can be one of: - ``Timer.HIGH`` - output is active high - - ``Timer.LOW`` - output is acive low + - ``Timer.LOW`` - output is active low Optional keyword arguments for Timer.IC modes: diff --git a/docs/library/pyb.rst b/docs/library/pyb.rst index 7be39a5d8e..2f3e7d36bd 100644 --- a/docs/library/pyb.rst +++ b/docs/library/pyb.rst @@ -46,7 +46,7 @@ Time related functions Returns the number of milliseconds which have elapsed since ``start``. This function takes care of counter wrap, and always returns a positive - number. This means it can be used to measure periods upto about 12.4 days. + number. This means it can be used to measure periods up to about 12.4 days. Example:: @@ -59,7 +59,7 @@ Time related functions Returns the number of microseconds which have elapsed since ``start``. This function takes care of counter wrap, and always returns a positive - number. This means it can be used to measure periods upto about 17.8 minutes. + number. This means it can be used to measure periods up to about 17.8 minutes. Example:: diff --git a/docs/library/ubinascii.rst b/docs/library/ubinascii.rst index 0a9adb50d4..4931f90482 100644 --- a/docs/library/ubinascii.rst +++ b/docs/library/ubinascii.rst @@ -17,7 +17,7 @@ Functions .. admonition:: Difference to CPython :class: attention - If additional argument, `sep` is supplied, it is used as a seperator + If additional argument, `sep` is supplied, it is used as a separator between hexadecimal values. .. function:: unhexlify(data) diff --git a/docs/library/ucollections.rst b/docs/library/ucollections.rst index c7ed068c7e..4e9de9ac67 100644 --- a/docs/library/ucollections.rst +++ b/docs/library/ucollections.rst @@ -13,10 +13,10 @@ Classes .. function:: namedtuple(name, fields) This is factory function to create a new namedtuple type with a specific - name and set of fields. A namedtyple is a subclass of tuple which allows + name and set of fields. A namedtuple is a subclass of tuple which allows to access its fields not just by numeric index, but also with an attribute access syntax using symbolic field names. Fields is a sequence of strings - specifying field names. For compatibily with CPython it can also be a + specifying field names. For compatibility with CPython it can also be a a string with space-separated field named (but this is less efficient). Example of use:: diff --git a/docs/library/uctypes.rst b/docs/library/uctypes.rst index 630a3a36fb..2a9c4dd5c0 100644 --- a/docs/library/uctypes.rst +++ b/docs/library/uctypes.rst @@ -118,7 +118,7 @@ Module contents .. data:: BIG_ENDIAN - Layour type for a big-endian packed structure. + Layout type for a big-endian packed structure. .. data:: NATIVE @@ -184,7 +184,7 @@ Subscripting a pointer with other integer values but 0 are supported too, with the same semantics as in C. Summing up, accessing structure fields generally follows C syntax, -except for pointer derefence, when you need to use ``[0]`` operator +except for pointer dereference, when you need to use ``[0]`` operator instead of ``*``. Limitations diff --git a/docs/library/uhashlib.rst b/docs/library/uhashlib.rst index 5de4b6f7ba..223c72dfe7 100644 --- a/docs/library/uhashlib.rst +++ b/docs/library/uhashlib.rst @@ -9,7 +9,7 @@ This module implements binary data hashing algorithms. Currently, it implements SHA256 algorithm. Choosing SHA256 was a deliberate choice, as a modern, cryptographically secure algorithm. This means that a - single algorithm can cover both usecases of "any hash algorithm" and + single algorithm can cover both use cases of "any hash algorithm" and security-related usage, and thus save space omitting legacy algorithms like MD5 or SHA1. @@ -53,7 +53,7 @@ Constructors Example:: - hash = uhashlib.sha1('abcd1234', 1001) # lenght of the initial piece is multiple of 4 bytes + hash = uhashlib.sha1('abcd1234', 1001) # length of the initial piece is multiple of 4 bytes hash.update('1234') # also multiple of 4 bytes ... hash.update('12345') # last chunk may be of any length @@ -68,7 +68,7 @@ Methods .. method:: hash.digest() - Return hash for all data passed thru hash, as a bytes object. After this + Return hash for all data passed through hash, as a bytes object. After this method is called, more data cannot be fed into hash any longer. .. only:: port_wipy diff --git a/docs/library/uio.rst b/docs/library/uio.rst index 1b3e2a0822..9b4c87df88 100644 --- a/docs/library/uio.rst +++ b/docs/library/uio.rst @@ -12,7 +12,7 @@ Functions .. function:: open(name, mode='r', **kwargs) - Open a file. Builtin ``open()`` function is alised to this function. + Open a file. Builtin ``open()`` function is aliased to this function. All ports (which provide access to file system) are required to support `mode` parameter, but support for other arguments vary by port. @@ -38,8 +38,9 @@ Classes opened with "b" modifier). Initial contents of file-like objects can be specified with `string` parameter (should be normal string for `StringIO` or bytes object for `BytesIO`). All the usual file - methods like ``read()``, ``write()``, ``close()`` are available on - these objects, and additionally, following method: + methods like ``read()``, ``write()``, ``seek()``, ``flush()``, + ``close()`` are available on these objects, and additionally, a + following method: .. method:: getvalue() diff --git a/docs/library/uos.rst b/docs/library/uos.rst index bb95107c04..77a61ca449 100644 --- a/docs/library/uos.rst +++ b/docs/library/uos.rst @@ -81,7 +81,7 @@ Functions .. function:: unmount(path) - Unmounts a prevoulsy mounted block device from the given path. + Unmounts a previously mounted block device from the given path. .. function:: mkfs(block_device or path) diff --git a/docs/library/utime.rst b/docs/library/utime.rst index 0bca4692ac..8310673830 100644 --- a/docs/library/utime.rst +++ b/docs/library/utime.rst @@ -117,7 +117,7 @@ Functions .. function:: time() Returns the number of seconds, as an integer, since the Epoch, assuming that underlying - RTC is set and maintained as decsribed above. If an RTC is not set, this function returns + RTC is set and maintained as described above. If an RTC is not set, this function returns number of seconds since a port-specific reference point in time (for embedded boards without a battery-backed RTC, usually since power up or reset). If you want to develop portable MicroPython application, you should not rely on this function to provide higher than second diff --git a/docs/pyboard/tutorial/timer.rst b/docs/pyboard/tutorial/timer.rst index fee01e17b7..aedaaa13c5 100644 --- a/docs/pyboard/tutorial/timer.rst +++ b/docs/pyboard/tutorial/timer.rst @@ -95,7 +95,7 @@ We set up timer 2 as follows:: The prescaler is set at 83, which makes this timer count at 1 MHz. This is because the CPU clock, running at 168 MHz, is divided by -2 and then by prescaler+1, giving a freqency of 168 MHz/2/(83+1)=1 MHz +2 and then by prescaler+1, giving a frequency of 168 MHz/2/(83+1)=1 MHz for timer 2. The period is set to a large number so that the timer can count up to a large number before wrapping back around to zero. In this case it will take about 17 minutes before it cycles back to diff --git a/docs/pyboard/tutorial/usb_mouse.rst b/docs/pyboard/tutorial/usb_mouse.rst index 4c3a85e548..ac1de6e275 100644 --- a/docs/pyboard/tutorial/usb_mouse.rst +++ b/docs/pyboard/tutorial/usb_mouse.rst @@ -61,7 +61,7 @@ The first argument to the function ``osc`` is the number of mouse events to send and the second argument is the delay (in milliseconds) between events. Try playing around with different numbers. -**Excercise: make the mouse go around in a circle.** +**Exercise: make the mouse go around in a circle.** Making a mouse with the accelerometer ------------------------------------- @@ -85,14 +85,14 @@ the [safe mode tutorial](tut-reset), but we repeat the instructions here: 3. The LEDs will then cycle green to orange to green+orange and back again. 4. Keep holding down USR until *only the orange LED is lit*, and then let go of the USR switch. -5. The orange LED should flash quickly 4 times, and then turn off. +5. The orange LED should flash quickly 4 times, and then turn off. 6. You are now in safe mode. In safe mode, the ``boot.py`` and ``main.py`` files are not executed, and so the pyboard boots up with default settings. This means you now have access to the filesystem (the USB drive should appear), and you can edit ``main.py``. (Leave ``boot.py`` as-is, because we still want to go back to HID-mode after -we finish editting ``main.py``.) +we finish editing ``main.py``.) In ``main.py`` put the following code:: diff --git a/docs/reference/asm_thumb2_hints_tips.rst b/docs/reference/asm_thumb2_hints_tips.rst index f0826e47ef..062a3c8448 100644 --- a/docs/reference/asm_thumb2_hints_tips.rst +++ b/docs/reference/asm_thumb2_hints_tips.rst @@ -129,7 +129,7 @@ Non-integer data types ~~~~~~~~~~~~~~~~~~~~~~ These may be handled by means of arrays of the appropriate data type. For -example, single precison floating point data may be processed as follows. +example, single precision floating point data may be processed as follows. This code example takes an array of floats and replaces its contents with their squares. @@ -172,7 +172,7 @@ thus: The const() construct causes MicroPython to replace the variable name with its value at compile time. If constants are declared in an outer -Python scope they can be shared between mutiple assembler functions and +Python scope they can be shared between multiple assembler functions and with Python code. Assembler code as class methods diff --git a/docs/reference/asm_thumb2_label_branch.rst b/docs/reference/asm_thumb2_label_branch.rst index 2c5e527185..8c85e55f0e 100644 --- a/docs/reference/asm_thumb2_label_branch.rst +++ b/docs/reference/asm_thumb2_label_branch.rst @@ -23,7 +23,7 @@ specifiers: * ne Not equal * cs Carry set * cc Carry clear -* mi Minus (negaive) +* mi Minus (negative) * pl Plus (positive) * vs Overflow set * vc Overflow clear diff --git a/docs/reference/isr_rules.rst b/docs/reference/isr_rules.rst index 2be4243f9e..b33e4dd6f2 100644 --- a/docs/reference/isr_rules.rst +++ b/docs/reference/isr_rules.rst @@ -24,7 +24,7 @@ This summarises the points detailed below and lists the principal recommendation * Where an ISR returns multiple bytes use a pre-allocated ``bytearray``. If multiple integers are to be shared between an ISR and the main program consider an array (``array.array``). * Where data is shared between the main program and an ISR, consider disabling interrupts prior to accessing - the data in the main program and re-enabling them immediately afterwards (see Critcal Sections). + the data in the main program and re-enabling them immediately afterwards (see Critical Sections). * Allocate an emergency exception buffer (see below). diff --git a/docs/reference/repl.rst b/docs/reference/repl.rst index 8ca89a0d1a..7a683ca220 100644 --- a/docs/reference/repl.rst +++ b/docs/reference/repl.rst @@ -50,7 +50,7 @@ Finally type ``print(i)``, press RETURN, press BACKSPACE and press RETURN again: >>> Auto-indent won't be applied if the previous two lines were all spaces. This -means that you can finish entering a compound statment by pressing RETURN +means that you can finish entering a compound statement by pressing RETURN twice, and then a third press will finish and execute. Auto-completion @@ -80,7 +80,7 @@ expansions: Interrupting a running program ------------------------------ -You can interupt a running program by pressing Ctrl-C. This will raise a KeyboardInterrupt +You can interrupt a running program by pressing Ctrl-C. This will raise a KeyboardInterrupt which will bring you back to the REPL, providing your program doesn't intercept the KeyboardInterrupt exception. @@ -184,8 +184,8 @@ variables no longer exist: The special variable _ (underscore) ----------------------------------- -When you use the REPL, you may perfom computations and see the results. -MicroPython stores the results of the previous statment in the variable _ (underscore). +When you use the REPL, you may perform computations and see the results. +MicroPython stores the results of the previous statement in the variable _ (underscore). So you can use the underscore to save the result in a variable. For example: >>> 1 + 2 + 3 + 4 + 5 diff --git a/docs/wipy/general.rst b/docs/wipy/general.rst index c1e9696601..2f18e7a62f 100644 --- a/docs/wipy/general.rst +++ b/docs/wipy/general.rst @@ -16,7 +16,7 @@ Before applying power .. warning:: - The GPIO pins of the WiPy are NOT 5V tolerant, connecting them to voltages higer + The GPIO pins of the WiPy are NOT 5V tolerant, connecting them to voltages higher than 3.6V will cause irreparable damage to the board. ADC pins, when configured in analog mode cannot withstand voltages above 1.8V. Keep these considerations in mind when wiring your electronics. diff --git a/docs/wipy/quickref.rst b/docs/wipy/quickref.rst index 6fd77a81bd..3ce7e01325 100644 --- a/docs/wipy/quickref.rst +++ b/docs/wipy/quickref.rst @@ -205,7 +205,7 @@ See :ref:`network.Server <network.Server>` :: server = Server(login=('user', 'password'), timeout=60) server.timeout(300) # change the timeout server.timeout() # get the timeout - server.isrunning() # check wether the server is running or not + server.isrunning() # check whether the server is running or not Heart beat LED -------------- diff --git a/docs/wipy/tutorial/intro.rst b/docs/wipy/tutorial/intro.rst index 7dc5f1c13d..3acc0510f1 100644 --- a/docs/wipy/tutorial/intro.rst +++ b/docs/wipy/tutorial/intro.rst @@ -17,7 +17,7 @@ Because the WiPy/expansion board does not have a housing it needs a bit of care: If you experience a lot of static electricity in your area (eg dry and cold climates), take extra care not to shock the WiPy. If your WiPy came in a ESD bag, then this bag is the best way to store and carry the - WiPy as it will protect it agains static discharges. + WiPy as it will protect it against static discharges. As long as you take care of the hardware, you should be okay. It's almost impossible to break the software on the WiPy, so feel free to play around diff --git a/drivers/wiznet5k/ethernet/socket.c b/drivers/wiznet5k/ethernet/socket.c index 7a96d9cb66..129473ad4d 100644 --- a/drivers/wiznet5k/ethernet/socket.c +++ b/drivers/wiznet5k/ethernet/socket.c @@ -157,7 +157,7 @@ int8_t WIZCHIP_EXPORT(socket)(uint8_t sn, uint8_t protocol, uint16_t port, uint8 setSn_PORT(sn,port); setSn_CR(sn,Sn_CR_OPEN); while(getSn_CR(sn)); - sock_io_mode |= ((flag & SF_IO_NONBLOCK) << sn); + sock_io_mode |= ((flag & SF_IO_NONBLOCK) << sn); sock_is_sending &= ~(1<<sn); sock_remained_size[sn] = 0; sock_pack_info[sn] = 0; diff --git a/esp8266/Makefile b/esp8266/Makefile index 00826cfe46..ea9c7eb871 100644 --- a/esp8266/Makefile +++ b/esp8266/Makefile @@ -4,6 +4,8 @@ include ../py/mkenv.mk QSTR_DEFS = qstrdefsport.h #$(BUILD)/pins_qstr.h MICROPY_PY_USSL = 1 +MICROPY_SSL_AXTLS = 1 +MICROPY_PY_BTREE = 1 # include py core make definitions include ../py/py.mk @@ -87,6 +89,7 @@ SRC_C = \ $(BUILD)/frozen.c \ fatfs_port.c \ axtls_helpers.c \ + $(SRC_MOD) STM_SRC_C = $(addprefix stmhal/,\ pybstdio.c \ @@ -153,7 +156,7 @@ SRC_QSTR += $(SRC_C) $(STM_SRC_C) $(EXTMOD_SRC_C) $(DRIVERS_SRC_C) # Append any auto-generated sources that are needed by sources listed in SRC_QSTR SRC_QSTR_AUTO_DEPS += -all: $(BUILD)/firmware-combined.bin +all: $(BUILD)/libaxtls.a $(BUILD)/firmware-combined.bin CONFVARS_FILE = $(BUILD)/confvars @@ -229,8 +232,11 @@ $(BUILD)/firmware.elf: $(OBJ) include ../py/mkrules.mk -axtls: +axtls: $(BUILD)/libaxtls.a + +$(BUILD)/libaxtls.a: cd ../lib/axtls; cp config/upyconfig config/.config cd ../lib/axtls; make oldconfig -B cd ../lib/axtls; make clean cd ../lib/axtls; make all CC="$(CC)" LD="$(LD)" AR="$(AR)" CFLAGS_EXTRA="$(CFLAGS_XTENSA) -Dabort=abort_ -DRT_MAX_PLAIN_LENGTH=1024 -DRT_EXTRA=3072" + cp ../lib/axtls/_stage/libaxtls.a $@ diff --git a/esp8266/axtls_helpers.c b/esp8266/axtls_helpers.c index 097d9ed4c3..6d508fdeb1 100644 --- a/esp8266/axtls_helpers.c +++ b/esp8266/axtls_helpers.c @@ -43,9 +43,6 @@ void *calloc(size_t nmemb, size_t size) { void *realloc(void *ptr, size_t size) { return gc_realloc(ptr, size, true); } -void abort_(void) { - printf("Aborted\n"); -} #define PLATFORM_HTONL(_n) ((uint32_t)( (((_n) & 0xff) << 24) | (((_n) & 0xff00) << 8) | (((_n) >> 8) & 0xff00) | (((_n) >> 24) & 0xff) )) #undef htonl diff --git a/esp8266/eagle.rom.addr.v6.ld b/esp8266/eagle.rom.addr.v6.ld index ce071a3bb9..1b3ce55d01 100644 --- a/esp8266/eagle.rom.addr.v6.ld +++ b/esp8266/eagle.rom.addr.v6.ld @@ -20,6 +20,7 @@ PROVIDE ( SPI_read_status = 0x400043c8 ); PROVIDE ( SPI_write_status = 0x40004400 ); PROVIDE ( SPI_write_enable = 0x4000443c ); PROVIDE ( Wait_SPI_Idle = 0x4000448c ); +PROVIDE ( Enable_QMode = 0x400044c0 ); PROVIDE ( SPIEraseArea = 0x40004b44 ); PROVIDE ( SPIEraseBlock = 0x400049b4 ); PROVIDE ( SPIEraseChip = 0x40004984 ); diff --git a/esp8266/esp8266.ld b/esp8266/esp8266.ld index 79bdf87f71..a55aff52e5 100644 --- a/esp8266/esp8266.ld +++ b/esp8266/esp8266.ld @@ -5,7 +5,7 @@ MEMORY dport0_0_seg : org = 0x3ff00000, len = 0x10 dram0_0_seg : org = 0x3ffe8000, len = 0x14000 iram1_0_seg : org = 0x40100000, len = 0x8000 - irom0_0_seg : org = 0x40209000, len = 0x80000 + irom0_0_seg : org = 0x40209000, len = 0x87000 } /* define the top of RAM */ @@ -121,6 +121,7 @@ SECTIONS *lib/fatfs/*.o*(.literal*, .text*) */libaxtls.a:(.literal*, .text*) + *lib/berkeley-db-1.xx/*.o(.literal*, .text*) *lib/libm/*.o*(.literal*, .text*) *lib/mp-readline/*.o(.literal*, .text*) *lib/netutils/*.o*(.literal*, .text*) diff --git a/esp8266/esp_mphal.c b/esp8266/esp_mphal.c index 3cc4610825..a2710af2e6 100644 --- a/esp8266/esp_mphal.c +++ b/esp8266/esp_mphal.c @@ -259,3 +259,8 @@ int ets_esf_free_bufs(int idx) { } return cnt; } + +extern int mp_stream_errno; +int *__errno() { + return &mp_stream_errno; +} diff --git a/esp8266/esp_mphal.h b/esp8266/esp_mphal.h index 91fee21198..fa52ae53a2 100644 --- a/esp8266/esp_mphal.h +++ b/esp8266/esp_mphal.h @@ -47,6 +47,12 @@ void mp_hal_init(void); void mp_hal_rtc_init(void); uint32_t mp_hal_ticks_us(void); +__attribute__((always_inline)) static inline uint32_t mp_hal_ticks_cpu(void) { + uint32_t ccount; + __asm__ __volatile__("rsr %0,ccount":"=a" (ccount)); + return ccount; +} + void mp_hal_delay_us(uint32_t); void mp_hal_set_interrupt_char(int c); uint32_t mp_hal_get_cpu_freq(void); diff --git a/esp8266/espapa102.c b/esp8266/espapa102.c index e5f3024b10..4295fe42d8 100644 --- a/esp8266/espapa102.c +++ b/esp8266/espapa102.c @@ -24,6 +24,9 @@ * THE SOFTWARE. */ +#include "py/mpconfig.h" +#if MICROPY_ESP8266_APA102 + #include <stdio.h> #include "c_types.h" #include "eagle_soc.h" @@ -108,3 +111,5 @@ void esp_apa102_write(uint8_t clockPin, uint8_t dataPin, uint8_t *pixels, uint32 _esp_apa102_append_additionial_cycles(clockPinMask, dataPinMask, numBytes); _esp_apa102_end_frame(clockPinMask, dataPinMask); } + +#endif diff --git a/esp8266/espneopixel.c b/esp8266/espneopixel.c index 26776f025e..0f12f4c820 100644 --- a/esp8266/espneopixel.c +++ b/esp8266/espneopixel.c @@ -9,16 +9,10 @@ #include "eagle_soc.h" #include "user_interface.h" #include "espneopixel.h" +#include "esp_mphal.h" #define NEO_KHZ400 (1) -static uint32_t _getCycleCount(void) __attribute__((always_inline)); -static inline uint32_t _getCycleCount(void) { - uint32_t ccount; - __asm__ __volatile__("rsr %0,ccount":"=a" (ccount)); - return ccount; -} - void /*ICACHE_RAM_ATTR*/ esp_neopixel_write(uint8_t pin, uint8_t *pixels, uint32_t numBytes, bool is800KHz) { uint8_t *p, *end, pix, mask; @@ -49,10 +43,10 @@ void /*ICACHE_RAM_ATTR*/ esp_neopixel_write(uint8_t pin, uint8_t *pixels, uint32 for(t = time0;; t = time0) { if(pix & mask) t = time1; // Bit high duration - while(((c = _getCycleCount()) - startTime) < period); // Wait for bit start + while(((c = mp_hal_ticks_cpu()) - startTime) < period); // Wait for bit start GPIO_REG_WRITE(GPIO_OUT_W1TS_ADDRESS, pinMask); // Set high startTime = c; // Save start time - while(((c = _getCycleCount()) - startTime) < t); // Wait high duration + while(((c = mp_hal_ticks_cpu()) - startTime) < t); // Wait high duration GPIO_REG_WRITE(GPIO_OUT_W1TC_ADDRESS, pinMask); // Set low if(!(mask >>= 1)) { // Next bit/byte if(p >= end) break; @@ -60,5 +54,5 @@ void /*ICACHE_RAM_ATTR*/ esp_neopixel_write(uint8_t pin, uint8_t *pixels, uint32 mask = 0x80; } } - while((_getCycleCount() - startTime) < period); // Wait for last bit + while((mp_hal_ticks_cpu() - startTime) < period); // Wait for last bit } diff --git a/esp8266/etshal.h b/esp8266/etshal.h index 0185a9e22c..dd61ddec93 100644 --- a/esp8266/etshal.h +++ b/esp8266/etshal.h @@ -20,6 +20,13 @@ void ets_timer_arm_new(os_timer_t *tim, uint32_t millis, bool repeat, bool is_mi void ets_timer_setfn(os_timer_t *tim, ETSTimerFunc callback, void *cb_data); void ets_timer_disarm(os_timer_t *tim); +// Opaque structure +typedef char MD5_CTX[64]; + +void MD5Init(MD5_CTX *context); +void MD5Update(MD5_CTX *context, const void *data, unsigned int len); +void MD5Final(unsigned char digest[16], MD5_CTX *context); + // These prototypes are for recent SDKs with "malloc tracking" void *pvPortMalloc(unsigned sz, const char *fname, int line); void vPortFree(void *p, const char *fname, int line); diff --git a/esp8266/main.c b/esp8266/main.c index 21bb6e0e0f..71dfeb2ac6 100644 --- a/esp8266/main.c +++ b/esp8266/main.c @@ -60,6 +60,7 @@ STATIC void mp_reset(void) { MP_STATE_PORT(dupterm_arr_obj) = MP_OBJ_NULL; pin_init0(); readline_init0(); + dupterm_task_init(); #if MICROPY_MODULE_FROZEN pyexec_frozen_module("_boot.py"); pyexec_file("boot.py"); @@ -85,7 +86,6 @@ void init_done(void) { #if MICROPY_REPL_EVENT_DRIVEN pyexec_event_repl_init(); #endif - dupterm_task_init(); #if !MICROPY_REPL_EVENT_DRIVEN soft_reset: diff --git a/esp8266/makeimg.py b/esp8266/makeimg.py index e63f956bdd..091854fa4d 100644 --- a/esp8266/makeimg.py +++ b/esp8266/makeimg.py @@ -1,23 +1,40 @@ import sys +import struct +import hashlib SEGS_MAX_SIZE = 0x9000 assert len(sys.argv) == 4 +md5 = hashlib.md5() + with open(sys.argv[3], 'wb') as fout: with open(sys.argv[1], 'rb') as f: data_flash = f.read() fout.write(data_flash) + # First 4 bytes include flash size, etc. which may be changed + # by esptool.py, etc. + md5.update(data_flash[4:]) print('flash ', len(data_flash)) + with open(sys.argv[2], 'rb') as f: + data_rom = f.read() + pad = b'\xff' * (SEGS_MAX_SIZE - len(data_flash)) - fout.write(pad) + assert len(pad) >= 4 + fout.write(pad[:-4]) + md5.update(pad[:-4]) + len_data = struct.pack("I", SEGS_MAX_SIZE + len(data_rom)) + fout.write(len_data) + md5.update(len_data) print('padding ', len(pad)) - with open(sys.argv[2], 'rb') as f: - data_rom = f.read() - fout.write(data_rom) - print('irom0text', len(data_rom)) + fout.write(data_rom) + md5.update(data_rom) + print('irom0text', len(data_rom)) + + fout.write(md5.digest()) print('total ', SEGS_MAX_SIZE + len(data_rom)) + print('md5 ', md5.hexdigest()) diff --git a/esp8266/modesp.c b/esp8266/modesp.c index 99f063a756..39cc514afc 100644 --- a/esp8266/modesp.c +++ b/esp8266/modesp.c @@ -633,6 +633,24 @@ STATIC mp_obj_t esp_flash_size(void) { } STATIC MP_DEFINE_CONST_FUN_OBJ_0(esp_flash_size_obj, esp_flash_size); +STATIC mp_obj_t esp_check_fw(void) { + MD5_CTX ctx; + uint32_t *sz_p = (uint32_t*)0x40208ffc; + printf("size: %d\n", *sz_p); + MD5Init(&ctx); + MD5Update(&ctx, (char*)0x40200004, *sz_p - 4); + unsigned char digest[16]; + MD5Final(digest, &ctx); + printf("md5: "); + for (int i = 0; i < 16; i++) { + printf("%02x", digest[i]); + } + printf("\n"); + return mp_obj_new_bool(memcmp(digest, (void*)(0x40200000 + *sz_p), sizeof(digest)) == 0); +} +STATIC MP_DEFINE_CONST_FUN_OBJ_0(esp_check_fw_obj, esp_check_fw); + + STATIC mp_obj_t esp_neopixel_write_(mp_obj_t pin, mp_obj_t buf, mp_obj_t is800k) { mp_buffer_info_t bufinfo; mp_get_buffer_raise(buf, &bufinfo, MP_BUFFER_READ); @@ -642,6 +660,7 @@ STATIC mp_obj_t esp_neopixel_write_(mp_obj_t pin, mp_obj_t buf, mp_obj_t is800k) } STATIC MP_DEFINE_CONST_FUN_OBJ_3(esp_neopixel_write_obj, esp_neopixel_write_); +#if MICROPY_ESP8266_APA102 STATIC mp_obj_t esp_apa102_write_(mp_obj_t clockPin, mp_obj_t dataPin, mp_obj_t buf) { mp_buffer_info_t bufinfo; mp_get_buffer_raise(buf, &bufinfo, MP_BUFFER_READ); @@ -651,6 +670,7 @@ STATIC mp_obj_t esp_apa102_write_(mp_obj_t clockPin, mp_obj_t dataPin, mp_obj_t return mp_const_none; } STATIC MP_DEFINE_CONST_FUN_OBJ_3(esp_apa102_write_obj, esp_apa102_write_); +#endif STATIC mp_obj_t esp_freemem() { return MP_OBJ_NEW_SMALL_INT(system_get_free_heap_size()); @@ -695,10 +715,13 @@ STATIC const mp_map_elem_t esp_module_globals_table[] = { { MP_OBJ_NEW_QSTR(MP_QSTR_getaddrinfo), (mp_obj_t)&esp_getaddrinfo_obj }, #endif { MP_OBJ_NEW_QSTR(MP_QSTR_neopixel_write), (mp_obj_t)&esp_neopixel_write_obj }, + #if MICROPY_ESP8266_APA102 { MP_OBJ_NEW_QSTR(MP_QSTR_apa102_write), (mp_obj_t)&esp_apa102_write_obj }, + #endif { MP_OBJ_NEW_QSTR(MP_QSTR_dht_readinto), (mp_obj_t)&dht_readinto_obj }, { MP_OBJ_NEW_QSTR(MP_QSTR_freemem), (mp_obj_t)&esp_freemem_obj }, { MP_OBJ_NEW_QSTR(MP_QSTR_meminfo), (mp_obj_t)&esp_meminfo_obj }, + { MP_OBJ_NEW_QSTR(MP_QSTR_check_fw), (mp_obj_t)&esp_check_fw_obj }, { MP_OBJ_NEW_QSTR(MP_QSTR_info), (mp_obj_t)&pyb_info_obj }, // TODO delete/rename/move elsewhere { MP_OBJ_NEW_QSTR(MP_QSTR_malloc), (mp_obj_t)&esp_malloc_obj }, { MP_OBJ_NEW_QSTR(MP_QSTR_free), (mp_obj_t)&esp_free_obj }, diff --git a/esp8266/modmachine.c b/esp8266/modmachine.c index 80ae7ac2a0..b3a42618e5 100644 --- a/esp8266/modmachine.c +++ b/esp8266/modmachine.c @@ -26,6 +26,7 @@ */ #include <stdint.h> +#include <stdio.h> #include "py/obj.h" #include "py/runtime.h" @@ -82,6 +83,18 @@ STATIC mp_obj_t machine_unique_id(void) { } STATIC MP_DEFINE_CONST_FUN_OBJ_0(machine_unique_id_obj, machine_unique_id); +STATIC mp_obj_t machine_idle(void) { + asm("waiti 0"); + return mp_const_none; +} +STATIC MP_DEFINE_CONST_FUN_OBJ_0(machine_idle_obj, machine_idle); + +STATIC mp_obj_t machine_sleep(void) { + printf("Warning: not yet implemented\n"); + return mp_const_none; +} +STATIC MP_DEFINE_CONST_FUN_OBJ_0(machine_sleep_obj, machine_sleep); + STATIC mp_obj_t machine_deepsleep(void) { // default to sleep forever uint32_t sleep_us = 0; @@ -222,6 +235,8 @@ STATIC const mp_rom_map_elem_t machine_module_globals_table[] = { { MP_ROM_QSTR(MP_QSTR_reset), MP_ROM_PTR(&machine_reset_obj) }, { MP_ROM_QSTR(MP_QSTR_reset_cause), MP_ROM_PTR(&machine_reset_cause_obj) }, { MP_ROM_QSTR(MP_QSTR_unique_id), MP_ROM_PTR(&machine_unique_id_obj) }, + { MP_ROM_QSTR(MP_QSTR_idle), MP_ROM_PTR(&machine_idle_obj) }, + { MP_ROM_QSTR(MP_QSTR_sleep), MP_ROM_PTR(&machine_sleep_obj) }, { MP_ROM_QSTR(MP_QSTR_deepsleep), MP_ROM_PTR(&machine_deepsleep_obj) }, { MP_ROM_QSTR(MP_QSTR_disable_irq), MP_ROM_PTR(&machine_disable_irq_obj) }, diff --git a/esp8266/modpybuart.c b/esp8266/modpybuart.c index f27ee4e2d2..25320fa1e7 100644 --- a/esp8266/modpybuart.c +++ b/esp8266/modpybuart.c @@ -112,9 +112,11 @@ STATIC void pyb_uart_init_helper(pyb_uart_obj_t *self, size_t n_args, const mp_o if (args[ARG_parity].u_obj != MP_OBJ_NULL) { if (args[ARG_parity].u_obj == mp_const_none) { UartDev.parity = UART_NONE_BITS; + UartDev.exist_parity = UART_STICK_PARITY_DIS; self->parity = 0; } else { mp_int_t parity = mp_obj_get_int(args[ARG_parity].u_obj); + UartDev.exist_parity = UART_STICK_PARITY_EN; if (parity & 1) { UartDev.parity = UART_ODD_BITS; self->parity = 1; diff --git a/esp8266/modules/_boot.py b/esp8266/modules/_boot.py index 2cacb56760..c200b3d3f9 100644 --- a/esp8266/modules/_boot.py +++ b/esp8266/modules/_boot.py @@ -1,4 +1,5 @@ import gc +gc.threshold((gc.mem_free() + gc.mem_alloc()) // 4) import uos from flashbdev import bdev diff --git a/esp8266/modules/flashbdev.py b/esp8266/modules/flashbdev.py index 07ed966020..98b0208fb6 100644 --- a/esp8266/modules/flashbdev.py +++ b/esp8266/modules/flashbdev.py @@ -3,8 +3,8 @@ import esp class FlashBdev: SEC_SIZE = 4096 - START_SEC = 0x89000 // SEC_SIZE - NUM_BLK = 0x73 + START_SEC = 0x90000 // SEC_SIZE + NUM_BLK = 0x6b def __init__(self, blocks=NUM_BLK): self.blocks = blocks @@ -64,5 +64,5 @@ size = esp.flash_size() if size < 1024*1024: bdev = None else: - # 16K at the flash end is reserved for SDK params storage - bdev = FlashBdev((size - 16384) // FlashBdev.SEC_SIZE - FlashBdev.START_SEC) + # 20K at the flash end is reserved for SDK params storage + bdev = FlashBdev((size - 20480) // FlashBdev.SEC_SIZE - FlashBdev.START_SEC) diff --git a/esp8266/moduos.c b/esp8266/moduos.c index 74262d86b1..af149625d6 100644 --- a/esp8266/moduos.c +++ b/esp8266/moduos.c @@ -95,6 +95,11 @@ STATIC mp_obj_t os_mkdir(mp_obj_t path_in) { } STATIC MP_DEFINE_CONST_FUN_OBJ_1(os_mkdir_obj, os_mkdir); +STATIC mp_obj_t os_rmdir(mp_obj_t path_in) { + return vfs_proxy_call(MP_QSTR_rmdir, 1, &path_in); +} +STATIC MP_DEFINE_CONST_FUN_OBJ_1(os_rmdir_obj, os_rmdir); + STATIC mp_obj_t os_chdir(mp_obj_t path_in) { return vfs_proxy_call(MP_QSTR_chdir, 1, &path_in); } @@ -155,6 +160,7 @@ STATIC const mp_rom_map_elem_t os_module_globals_table[] = { { MP_ROM_QSTR(MP_QSTR_VfsFat), MP_ROM_PTR(&mp_fat_vfs_type) }, { MP_ROM_QSTR(MP_QSTR_listdir), MP_ROM_PTR(&os_listdir_obj) }, { MP_ROM_QSTR(MP_QSTR_mkdir), MP_ROM_PTR(&os_mkdir_obj) }, + { MP_ROM_QSTR(MP_QSTR_rmdir), MP_ROM_PTR(&os_rmdir_obj) }, { MP_ROM_QSTR(MP_QSTR_chdir), MP_ROM_PTR(&os_chdir_obj) }, { MP_ROM_QSTR(MP_QSTR_getcwd), MP_ROM_PTR(&os_getcwd_obj) }, { MP_ROM_QSTR(MP_QSTR_remove), MP_ROM_PTR(&os_remove_obj) }, diff --git a/esp8266/modutime.c b/esp8266/modutime.c index 356785d3a9..bf56feb5ce 100644 --- a/esp8266/modutime.c +++ b/esp8266/modutime.c @@ -134,8 +134,7 @@ STATIC mp_obj_t time_ticks_us(void) { STATIC MP_DEFINE_CONST_FUN_OBJ_0(time_ticks_us_obj, time_ticks_us); STATIC mp_obj_t time_ticks_cpu(void) { - // TODO - return MP_OBJ_NEW_SMALL_INT(0 & MP_SMALL_INT_POSITIVE_MASK); + return MP_OBJ_NEW_SMALL_INT(mp_hal_ticks_cpu() & MP_SMALL_INT_POSITIVE_MASK); } STATIC MP_DEFINE_CONST_FUN_OBJ_0(time_ticks_cpu_obj, time_ticks_cpu); diff --git a/esp8266/mpconfigport.h b/esp8266/mpconfigport.h index 90dbc00098..7b3558ffc2 100644 --- a/esp8266/mpconfigport.h +++ b/esp8266/mpconfigport.h @@ -72,7 +72,10 @@ #define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_MPZ) #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_FLOAT) #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_NORMAL) +#define MICROPY_WARNINGS (1) +#define MICROPY_PY_STR_BYTES_CMP_WARN (1) #define MICROPY_STREAMS_NON_BLOCK (1) +#define MICROPY_STREAMS_POSIX_API (1) #define MICROPY_MODULE_FROZEN_STR (1) #define MICROPY_MODULE_FROZEN_MPY (1) #define MICROPY_MODULE_FROZEN_LEXER mp_lexer_new_from_str32 @@ -85,6 +88,7 @@ #define MICROPY_FATFS_LFN_CODE_PAGE (437) /* 1=SFN/ANSI 437=LFN/U.S.(OEM) */ #define MICROPY_FSUSERMOUNT (1) #define MICROPY_VFS_FAT (1) +#define MICROPY_ESP8266_APA102 (1) #define MICROPY_EVENT_POLL_HOOK {ets_event_poll();} #define MICROPY_VM_HOOK_COUNT (10) @@ -114,6 +118,8 @@ typedef void *machine_ptr_t; // must be of pointer size typedef const void *machine_const_ptr_t; // must be of pointer size typedef long mp_off_t; typedef uint32_t sys_prot_t; // for modlwip +// ssize_t, off_t as required by POSIX-signatured functions in stream.h +#include <sys/types.h> #define MP_PLAT_PRINT_STRN(str, len) mp_hal_stdout_tx_strn_cooked(str, len) diff --git a/esp8266/scripts/inisetup.py b/esp8266/scripts/inisetup.py index 7d8f6fa0a8..1cb9f0dd95 100644 --- a/esp8266/scripts/inisetup.py +++ b/esp8266/scripts/inisetup.py @@ -40,6 +40,8 @@ def setup(): with open("/boot.py", "w") as f: f.write("""\ # This file is executed on every boot (including wake-boot from deepsleep) +#import esp +#esp.osdebug(None) import gc #import webrepl #webrepl.start() diff --git a/esp8266/scripts/port_diag.py b/esp8266/scripts/port_diag.py index aa696b1abb..f2abd8066d 100644 --- a/esp8266/scripts/port_diag.py +++ b/esp8266/scripts/port_diag.py @@ -17,6 +17,8 @@ def main(): FREQ_MAP = {0: "40MHZ", 1: "26MHZ", 2: "20MHz", 0xf: "80MHz"} print("Byte @2: %02x" % ROM[2]) print("Byte @3: %02x (Flash size: %s Flash freq: %s)" % (ROM[3], SZ_MAP.get(ROM[3] >> 4, "?"), FREQ_MAP.get(ROM[3] & 0xf))) + print("Firmware checksum:") + print(esp.check_fw()) print("\nNetworking:") print("STA ifconfig:", network.WLAN(network.STA_IF).ifconfig()) diff --git a/esp8266/uart.h b/esp8266/uart.h index 21894d3fa6..2b97683ff8 100644 --- a/esp8266/uart.h +++ b/esp8266/uart.h @@ -1,6 +1,8 @@ #ifndef _INCLUDED_UART_H_ #define _INCLUDED_UART_H_ +#include <eagle_soc.h> + #define UART0 (0) #define UART1 (1) @@ -18,14 +20,14 @@ typedef enum { } UartStopBitsNum; typedef enum { - UART_NONE_BITS = 0, - UART_ODD_BITS = 0, - UART_EVEN_BITS = BIT4 + UART_NONE_BITS = 0, + UART_ODD_BITS = BIT0, + UART_EVEN_BITS = 0 } UartParityMode; typedef enum { UART_STICK_PARITY_DIS = 0, - UART_STICK_PARITY_EN = BIT3 | BIT5 + UART_STICK_PARITY_EN = BIT1 } UartExistParity; typedef enum { diff --git a/examples/embedding/Makefile b/examples/embedding/Makefile new file mode 100644 index 0000000000..99f239a7c5 --- /dev/null +++ b/examples/embedding/Makefile @@ -0,0 +1,8 @@ +MPTOP = ../.. +CFLAGS = -std=c99 -I. -I$(MPTOP) -DNO_QSTR +LDFLAGS = -L. + +hello-embed: hello-embed.o -lmicropython + +-lmicropython: + $(MAKE) -f $(MPTOP)/examples/embedding/Makefile.upylib MPTOP=$(MPTOP) diff --git a/examples/embedding/Makefile.upylib b/examples/embedding/Makefile.upylib new file mode 100644 index 0000000000..d8dbade3fe --- /dev/null +++ b/examples/embedding/Makefile.upylib @@ -0,0 +1,200 @@ +MPTOP = ../.. +-include mpconfigport.mk +include $(MPTOP)/py/mkenv.mk + +all: lib + +# OS name, for simple autoconfig +UNAME_S := $(shell uname -s) + +# include py core make definitions +include $(MPTOP)/py/py.mk + +INC += -I. +INC += -I.. +INC += -I$(MPTOP) +INC += -I$(MPTOP)/unix +#INC += -I../lib/timeutils +INC += -I$(BUILD) + +# compiler settings +CWARN = -Wall -Werror +CWARN += -Wpointer-arith -Wuninitialized +CFLAGS = $(INC) $(CWARN) -ansi -std=gnu99 -DUNIX $(CFLAGS_MOD) $(COPT) $(CFLAGS_EXTRA) + +# Debugging/Optimization +ifdef DEBUG +CFLAGS += -g +COPT = -O0 +else +COPT = -Os #-DNDEBUG +# _FORTIFY_SOURCE is a feature in gcc/glibc which is intended to provide extra +# security for detecting buffer overflows. Some distros (Ubuntu at the very least) +# have it enabled by default. +# +# gcc already optimizes some printf calls to call puts and/or putchar. When +# _FORTIFY_SOURCE is enabled and compiling with -O1 or greater, then some +# printf calls will also be optimized to call __printf_chk (in glibc). Any +# printfs which get redirected to __printf_chk are then no longer synchronized +# with printfs that go through mp_printf. +# +# In MicroPython, we don't want to use the runtime library's printf but rather +# go through mp_printf, so that stdout is properly tied into streams, etc. +# This means that we either need to turn off _FORTIFY_SOURCE or provide our +# own implementation of __printf_chk. We've chosen to turn off _FORTIFY_SOURCE. +# It should also be noted that the use of printf in MicroPython is typically +# quite limited anyways (primarily for debug and some error reporting, etc +# in the unix version). +# +# Information about _FORTIFY_SOURCE seems to be rather scarce. The best I could +# find was this: https://securityblog.redhat.com/2014/03/26/fortify-and-you/ +# Original patchset was introduced by +# https://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html . +# +# Turning off _FORTIFY_SOURCE is only required when compiling with -O1 or greater +CFLAGS += -U _FORTIFY_SOURCE +endif + +# On OSX, 'gcc' is a symlink to clang unless a real gcc is installed. +# The unix port of micropython on OSX must be compiled with clang, +# while cross-compile ports require gcc, so we test here for OSX and +# if necessary override the value of 'CC' set in py/mkenv.mk +ifeq ($(UNAME_S),Darwin) +CC = clang +# Use clang syntax for map file +LDFLAGS_ARCH = -Wl,-map,$@.map +else +# Use gcc syntax for map file +LDFLAGS_ARCH = -Wl,-Map=$@.map,--cref +endif +LDFLAGS = $(LDFLAGS_MOD) $(LDFLAGS_ARCH) -lm $(LDFLAGS_EXTRA) + +ifeq ($(MICROPY_FORCE_32BIT),1) +# Note: you may need to install i386 versions of dependency packages, +# starting with linux-libc-dev:i386 +ifeq ($(MICROPY_PY_FFI),1) +ifeq ($(UNAME_S),Linux) +CFLAGS_MOD += -I/usr/include/i686-linux-gnu +endif +endif +endif + +ifeq ($(MICROPY_USE_READLINE),1) +INC += -I../lib/mp-readline +CFLAGS_MOD += -DMICROPY_USE_READLINE=1 +LIB_SRC_C_EXTRA += mp-readline/readline.c +endif +ifeq ($(MICROPY_USE_READLINE),2) +CFLAGS_MOD += -DMICROPY_USE_READLINE=2 +LDFLAGS_MOD += -lreadline +# the following is needed for BSD +#LDFLAGS_MOD += -ltermcap +endif +ifeq ($(MICROPY_PY_TIME),1) +CFLAGS_MOD += -DMICROPY_PY_TIME=1 +SRC_MOD += modtime.c +endif +ifeq ($(MICROPY_PY_TERMIOS),1) +CFLAGS_MOD += -DMICROPY_PY_TERMIOS=1 +SRC_MOD += modtermios.c +endif +ifeq ($(MICROPY_PY_SOCKET),1) +CFLAGS_MOD += -DMICROPY_PY_SOCKET=1 +SRC_MOD += modsocket.c +endif + +ifeq ($(MICROPY_PY_FFI),1) + +ifeq ($(MICROPY_STANDALONE),1) +LIBFFI_CFLAGS_MOD := -I$(shell ls -1d ../lib/libffi/build_dir/out/lib/libffi-*/include) + ifeq ($(MICROPY_FORCE_32BIT),1) + LIBFFI_LDFLAGS_MOD = ../lib/libffi/build_dir/out/lib32/libffi.a + else + LIBFFI_LDFLAGS_MOD = ../lib/libffi/build_dir/out/lib/libffi.a + endif +else +LIBFFI_CFLAGS_MOD := $(shell pkg-config --cflags libffi) +LIBFFI_LDFLAGS_MOD := $(shell pkg-config --libs libffi) +endif + +ifeq ($(UNAME_S),Linux) +LIBFFI_LDFLAGS_MOD += -ldl +endif + +CFLAGS_MOD += $(LIBFFI_CFLAGS_MOD) -DMICROPY_PY_FFI=1 +LDFLAGS_MOD += $(LIBFFI_LDFLAGS_MOD) +SRC_MOD += modffi.c +endif + +MAIN_C = main.c + +# source files +SRC_C = $(addprefix $(MPTOP)/unix/,\ + $(MAIN_C) \ + gccollect.c \ + unix_mphal.c \ + input.c \ + file.c \ + modmachine.c \ + modos.c \ + moduselect.c \ + alloc.c \ + coverage.c \ + fatfs_port.c \ + $(SRC_MOD) \ + ) + +LIB_SRC_C = $(addprefix lib/,\ + $(LIB_SRC_C_EXTRA) \ + utils/printf.c \ + timeutils/timeutils.c \ + ) + +ifeq ($(MICROPY_FATFS),1) +LIB_SRC_C += $(addprefix lib/,\ + fatfs/ff.c \ + fatfs/option/ccsbcs.c \ + ) +endif + +OBJ = $(PY_O) +OBJ += $(addprefix $(BUILD)/, $(SRC_C:.c=.o)) +OBJ += $(addprefix $(BUILD)/, $(LIB_SRC_C:.c=.o)) +OBJ += $(addprefix $(BUILD)/, $(STMHAL_SRC_C:.c=.o)) + +# List of sources for qstr extraction +SRC_QSTR += $(SRC_C) $(LIB_SRC_C) +# Append any auto-generated sources that are needed by sources listed in +# SRC_QSTR +SRC_QSTR_AUTO_DEPS += + +include $(MPTOP)/py/mkrules.mk + +# Value of configure's --host= option (required for cross-compilation). +# Deduce it from CROSS_COMPILE by default, but can be overriden. +ifneq ($(CROSS_COMPILE),) +CROSS_COMPILE_HOST = --host=$(patsubst %-,%,$(CROSS_COMPILE)) +else +CROSS_COMPILE_HOST = +endif + +deplibs: libffi axtls + +# install-exec-recursive & install-data-am targets are used to avoid building +# docs and depending on makeinfo +libffi: + cd ../lib/libffi; git clean -d -x -f + cd ../lib/libffi; ./autogen.sh + mkdir -p ../lib/libffi/build_dir; cd ../lib/libffi/build_dir; \ + ../configure $(CROSS_COMPILE_HOST) --prefix=$$PWD/out CC="$(CC)" CXX="$(CXX)" LD="$(LD)"; \ + make install-exec-recursive; make -C include install-data-am + +axtls: ../lib/axtls/README + cd ../lib/axtls; cp config/upyconfig config/.config + cd ../lib/axtls; make oldconfig -B + cd ../lib/axtls; make clean + cd ../lib/axtls; make all CC="$(CC)" LD="$(LD)" + +../lib/axtls/README: + @echo "You cloned without --recursive, fetching submodules for you." + (cd ..; git submodule update --init --recursive) diff --git a/examples/embedding/README b/examples/embedding/README new file mode 100644 index 0000000000..0475e8739a --- /dev/null +++ b/examples/embedding/README @@ -0,0 +1,66 @@ +Example of embedding MicroPython in a standlone C application +============================================================= + +This directory contains a (very simple!) example of how to embed a MicroPython +in an existing C application. + +A C application is represented by the file hello-embed.c. It executes a simple +Python statement which prints to the standard output. + + +Building the example +-------------------- + +Build the example is as simple as running: + + make + +It's worth to trace what's happening behind the scenes though: + +1. As a first step, a MicroPython library is built. This is handled by a +seperate makefile, Makefile.upylib. It is more or less complex, but the +good news is that you won't need to change anything in it, just use it +as is, the main Makefile shows how. What may need editing though is +MicroPython configuration file. MicroPython is highly configurable, so +you would need to build a library suiting your application well, while +not bloating its size. Check the options in the file "mpconfigport.h". +Included is a copy of "minimal" Unix port, which should be good start +for minimal embedding. For list of all available options, see py/mpconfig.h. + +2. Once the library is built, your application is compiled and linked with +the MicroPython library produced in the previous step. The main Makefile +is very simple and shows that changes you would need to do to your +application's Makefile (or other build configuration) are also simple: + +a) You would need to use C99 standard (you're using 15+ years old standard +already, not a 25+ years old one, right?). + +b) You need to provide path to MicroPython's top-level dir, for includes. + +c) You need to include -DNO_QSTR compile-time flag. + +d) Otherwise, just link with micropython library produced in step 1. + + +Out of tree build +----------------- + +This example set up to work out of the box, being part of the MicroPython +tree. Your application of course will be outside of its tree, but the +only thing you need to do is to pass MPTOP variable pointing to +MicroPython directory to both Makefiles (in this example, the main Makefile +automatically pass it to Makefile.upylib; in your own Makefile, don't forget +to use suitable value). + +A practical way to embed MicroPython in your application is to include it +as a git submodule. Suppose you included it as libs/micropython. Then in +your main Makefile you would have something like: + +~~~ +MPTOP = libs/micropython + +my_app: $(MY_OBJS) -lmicropython + +-lmicropython: + $(MAKE) -f $(MPTOP)/examples/embedding/Makefile.upylib MPTOP=$(MPTOP) +~~~ diff --git a/examples/embedding/hello-embed.c b/examples/embedding/hello-embed.c new file mode 100644 index 0000000000..1949305184 --- /dev/null +++ b/examples/embedding/hello-embed.c @@ -0,0 +1,74 @@ +/* + * This file is part of the Micro Python project, http://micropython.org/ + * + * The MIT License (MIT) + * + * Copyright (c) 2016 Paul Sokolovsky + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include <string.h> +#include <stdio.h> +#include <stdlib.h> + +#include "py/compile.h" +#include "py/runtime.h" +#include "py/gc.h" +#include "py/stackctrl.h" + +static char heap[16384]; + +mp_obj_t execute_from_lexer(mp_lexer_t *lex) { + nlr_buf_t nlr; + if (nlr_push(&nlr) == 0) { + mp_parse_tree_t pt = mp_parse(lex, MP_PARSE_FILE_INPUT); + mp_obj_t module_fun = mp_compile(&pt, lex->source_name, MP_EMIT_OPT_NONE, false); + mp_call_function_0(module_fun); + nlr_pop(); + return 0; + } else { + // uncaught exception + return (mp_obj_t)nlr.ret_val; + } +} + +int main() { + // Initialized stack limit + mp_stack_set_limit(40000 * (BYTES_PER_WORD / 4)); + // Initialize heap + gc_init(heap, heap + sizeof(heap)); + // Initialize interpreter + mp_init(); + + const char str[] = "print('Hello world of easy embedding!')"; + mp_lexer_t *lex = mp_lexer_new_from_str_len(0/*MP_QSTR_*/, str, strlen(str), false); + if (execute_from_lexer(lex)) { + printf("Error\n"); + } +} + +uint mp_import_stat(const char *path) { + return MP_IMPORT_STAT_NO_EXIST; +} + +void nlr_jump_fail(void *val) { + printf("FATAL: uncaught NLR %p\n", val); + exit(1); +} diff --git a/examples/embedding/mpconfigport.h b/examples/embedding/mpconfigport.h new file mode 120000 index 0000000000..142e5d6f43 --- /dev/null +++ b/examples/embedding/mpconfigport.h @@ -0,0 +1 @@ +mpconfigport_minimal.h
\ No newline at end of file diff --git a/examples/embedding/mpconfigport_minimal.h b/examples/embedding/mpconfigport_minimal.h new file mode 100644 index 0000000000..9f080ab0e1 --- /dev/null +++ b/examples/embedding/mpconfigport_minimal.h @@ -0,0 +1,138 @@ +/* + * This file is part of the Micro Python project, http://micropython.org/ + * + * The MIT License (MIT) + * + * Copyright (c) 2015 Damien P. George + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +// options to control how Micro Python is built + +#define MICROPY_ALLOC_PATH_MAX (PATH_MAX) +#define MICROPY_ENABLE_GC (1) +#define MICROPY_ENABLE_FINALISER (0) +#define MICROPY_STACK_CHECK (0) +#define MICROPY_COMP_CONST (0) +#define MICROPY_MEM_STATS (0) +#define MICROPY_DEBUG_PRINTERS (0) +#define MICROPY_HELPER_REPL (1) +#define MICROPY_HELPER_LEXER_UNIX (1) +#define MICROPY_ENABLE_SOURCE_LINE (0) +#define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_TERSE) +#define MICROPY_WARNINGS (0) +#define MICROPY_ENABLE_EMERGENCY_EXCEPTION_BUF (0) +#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_NONE) +#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE) +#define MICROPY_STREAMS_NON_BLOCK (0) +#define MICROPY_OPT_COMPUTED_GOTO (0) +#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0) +#define MICROPY_CAN_OVERRIDE_BUILTINS (0) +#define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (0) +#define MICROPY_CPYTHON_COMPAT (0) +#define MICROPY_PY_BUILTINS_BYTEARRAY (0) +#define MICROPY_PY_BUILTINS_MEMORYVIEW (0) +#define MICROPY_PY_BUILTINS_COMPILE (0) +#define MICROPY_PY_BUILTINS_ENUMERATE (0) +#define MICROPY_PY_BUILTINS_FILTER (0) +#define MICROPY_PY_BUILTINS_FROZENSET (0) +#define MICROPY_PY_BUILTINS_REVERSED (0) +#define MICROPY_PY_BUILTINS_SET (0) +#define MICROPY_PY_BUILTINS_SLICE (0) +#define MICROPY_PY_BUILTINS_STR_UNICODE (0) +#define MICROPY_PY_BUILTINS_PROPERTY (0) +#define MICROPY_PY_BUILTINS_MIN_MAX (0) +#define MICROPY_PY___FILE__ (0) +#define MICROPY_PY_MICROPYTHON_MEM_INFO (0) +#define MICROPY_PY_GC (0) +#define MICROPY_PY_GC_COLLECT_RETVAL (0) +#define MICROPY_PY_ARRAY (0) +#define MICROPY_PY_COLLECTIONS (0) +#define MICROPY_PY_MATH (0) +#define MICROPY_PY_CMATH (0) +#define MICROPY_PY_IO (0) +#define MICROPY_PY_IO_FILEIO (0) +#define MICROPY_PY_STRUCT (0) +#define MICROPY_PY_SYS (1) +#define MICROPY_PY_SYS_EXIT (0) +#define MICROPY_PY_SYS_PLATFORM "linux" +#define MICROPY_PY_SYS_MAXSIZE (0) +#define MICROPY_PY_SYS_STDFILES (0) +#define MICROPY_PY_CMATH (0) +#define MICROPY_PY_UCTYPES (0) +#define MICROPY_PY_UZLIB (0) +#define MICROPY_PY_UJSON (0) +#define MICROPY_PY_URE (0) +#define MICROPY_PY_UHEAPQ (0) +#define MICROPY_PY_UHASHLIB (0) +#define MICROPY_PY_UBINASCII (0) + +extern const struct _mp_obj_module_t mp_module_os; + +#define MICROPY_PORT_BUILTIN_MODULES \ + { MP_OBJ_NEW_QSTR(MP_QSTR_uos), (mp_obj_t)&mp_module_os }, \ + +#define MICROPY_PORT_ROOT_POINTERS \ + mp_obj_t keyboard_interrupt_obj; + +////////////////////////////////////////// +// Do not change anything beyond this line +////////////////////////////////////////// + +// Define to 1 to use undertested inefficient GC helper implementation +// (if more efficient arch-specific one is not available). +#ifndef MICROPY_GCREGS_SETJMP + #ifdef __mips__ + #define MICROPY_GCREGS_SETJMP (1) + #else + #define MICROPY_GCREGS_SETJMP (0) + #endif +#endif + +// type definitions for the specific machine + +#ifdef __LP64__ +typedef long mp_int_t; // must be pointer size +typedef unsigned long mp_uint_t; // must be pointer size +#else +// These are definitions for machines where sizeof(int) == sizeof(void*), +// regardless for actual size. +typedef int mp_int_t; // must be pointer size +typedef unsigned int mp_uint_t; // must be pointer size +#endif + +#define BYTES_PER_WORD sizeof(mp_int_t) + +// Cannot include <sys/types.h>, as it may lead to symbol name clashes +#if _FILE_OFFSET_BITS == 64 && !defined(__LP64__) +typedef long long mp_off_t; +#else +typedef long mp_off_t; +#endif + +typedef void *machine_ptr_t; // must be of pointer size +typedef const void *machine_const_ptr_t; // must be of pointer size + +// We need to provide a declaration/definition of alloca() +#ifdef __FreeBSD__ +#include <stdlib.h> +#else +#include <alloca.h> +#endif diff --git a/examples/network/http_client.py b/examples/network/http_client.py index 3701e75e16..df66ace2a5 100644 --- a/examples/network/http_client.py +++ b/examples/network/http_client.py @@ -24,5 +24,7 @@ def main(use_stream=False): s.send(b"GET / HTTP/1.0\n\n") print(s.recv(4096)) + s.close() + main() diff --git a/examples/network/http_client_ssl.py b/examples/network/http_client_ssl.py index 53b1c732bc..46e039830f 100644 --- a/examples/network/http_client_ssl.py +++ b/examples/network/http_client_ssl.py @@ -32,5 +32,7 @@ def main(use_stream=True): s.send(b"GET / HTTP/1.0\n\n") print(s.recv(4096)) + s.close() + main() diff --git a/examples/network/http_server.py b/examples/network/http_server.py index 80dfc5db02..e3a66e8283 100644 --- a/examples/network/http_server.py +++ b/examples/network/http_server.py @@ -10,7 +10,7 @@ HTTP/1.0 200 OK Hello #%d from MicroPython! """ -def main(use_stream=False): +def main(micropython_optimize=False): s = socket.socket() # Binding to all interfaces - server will be accessible to other hosts! @@ -26,20 +26,37 @@ def main(use_stream=False): counter = 0 while True: res = s.accept() - client_s = res[0] + client_sock = res[0] client_addr = res[1] print("Client address:", client_addr) - print("Client socket:", client_s) - print("Request:") - if use_stream: - # MicroPython socket objects support stream (aka file) interface - # directly. - print(client_s.read(4096)) - client_s.write(CONTENT % counter) + print("Client socket:", client_sock) + + if not micropython_optimize: + # To read line-oriented protocol (like HTTP) from a socket (and + # avoid short read problem), it must be wrapped in a stream (aka + # file-like) object. That's how you do it in CPython: + client_stream = client_sock.makefile("rwb") else: - print(client_s.recv(4096)) - client_s.send(CONTENT % counter) - client_s.close() + # .. but MicroPython socket objects support stream interface + # directly, so calling .makefile() method is not required. If + # you develop application which will run only on MicroPython, + # especially on a resource-constrained embedded device, you + # may take this shortcut to save resources. + client_stream = client_sock + + print("Request:") + req = client_stream.readline() + print(req) + while True: + h = client_stream.readline() + if h == b"" or h == b"\r\n": + break + print(h) + client_stream.write(CONTENT % counter) + + client_stream.close() + if not micropython_optimize: + client_sock.close() counter += 1 print() diff --git a/examples/network/http_server_simplistic.py b/examples/network/http_server_simplistic.py new file mode 100644 index 0000000000..f932e48f56 --- /dev/null +++ b/examples/network/http_server_simplistic.py @@ -0,0 +1,38 @@ +try: + import usocket as socket +except: + import socket + + +CONTENT = b"""\ +HTTP/1.0 200 OK + +Hello #%d from MicroPython! +""" + +def main(): + s = socket.socket() + ai = socket.getaddrinfo("0.0.0.0", 8080) + addr = ai[0][-1] + + s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + + s.bind(addr) + s.listen(5) + print("Listening, connect your browser to http://<this_host>:8080/") + + counter = 0 + while True: + res = s.accept() + client_s = res[0] + client_addr = res[1] + req = client_s.recv(4096) + print("Request:") + print(req) + client_s.send(CONTENT % counter) + client_s.close() + counter += 1 + print() + + +main() diff --git a/examples/network/http_server_simplistic_commented.py b/examples/network/http_server_simplistic_commented.py new file mode 100644 index 0000000000..b58e9eeb60 --- /dev/null +++ b/examples/network/http_server_simplistic_commented.py @@ -0,0 +1,76 @@ +# +# MicroPython http_server_simplistic.py example +# +# This example shows how to write the smallest possible HTTP +# server in MicroPython. With comments and convenience code +# removed, this example can be compressed literally to ten +# lines. There's a catch though - read comments below for +# details, and use this code only for quick hacks, preferring +# http_server.py for "real thing". +# +try: + import usocket as socket +except: + import socket + + +CONTENT = b"""\ +HTTP/1.0 200 OK + +Hello #%d from MicroPython! +""" + +def main(): + s = socket.socket() + + # Bind to (allow to be connected on ) all interfaces. This means + # this server will be accessible to other hosts on your local + # network, and if your server has direct (non-firewalled) connection + # to the Internet, then to anyone on the Internet. We bind to all + # interfaces to let this example work easily on embedded MicroPython + # targets, which you will likely access from another machine on your + # local network. Take care when running this on an Internet-connected + # machine though! Replace "0.0.0.0" with "127.0.0.1" if in doubt, to + # make the server accessible only on the machine it runs on. + ai = socket.getaddrinfo("0.0.0.0", 8080) + print("Bind address info:", ai) + addr = ai[0][-1] + + # A port on which a socket listened remains inactive during some time. + # This means that if you run this sample, terminate it, and run again + # you will likely get an error. To avoid this timeout, set SO_REUSEADDR + # socket option. + s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + + s.bind(addr) + s.listen(5) + print("Listening, connect your browser to http://<this_host>:8080/") + + counter = 0 + while True: + res = s.accept() + client_s = res[0] + client_addr = res[1] + print("Client address:", client_addr) + print("Client socket:", client_s) + # We assume here that .recv() call will read entire HTTP request + # from client. This is usually true, at least on "big OS" systems + # like Linux/MacOS/Windows. But that doesn't have to be true in + # all cases, in particular on embedded systems, when there can + # easily be "short recv", where it returns much less than requested + # data size. That's why this example is called "simplistic" - it + # shows that writing a web server in Python that *usually works* is + # ten lines of code, and you can use this technique for quick hacks + # and experimentation. But don't do it like that in production + # applications - instead, parse HTTP request properly, as shown + # by http_server.py example. + req = client_s.recv(4096) + print("Request:") + print(req) + client_s.send(CONTENT % counter) + client_s.close() + counter += 1 + print() + + +main() diff --git a/examples/network/http_server_ssl.py b/examples/network/http_server_ssl.py index 04e0913448..9a69ca9d41 100644 --- a/examples/network/http_server_ssl.py +++ b/examples/network/http_server_ssl.py @@ -42,8 +42,13 @@ def main(use_stream=True): # next request they issue will likely be more well-behaving and # will succeed. try: - req = client_s.read(4096) + req = client_s.readline() print(req) + while True: + h = client_s.readline() + if h == b"" or h == b"\r\n": + break + print(h) if req: client_s.write(CONTENT % counter) except Exception as e: diff --git a/examples/unix/machine_bios.py b/examples/unix/machine_bios.py new file mode 100644 index 0000000000..f62e4dbdb4 --- /dev/null +++ b/examples/unix/machine_bios.py @@ -0,0 +1,9 @@ +# This example shows how to access Video BIOS memory area via machine.mem +# It requires root privilege and x86 legacy harfware (which has mentioned +# Video BIOS at all). +# It is expected to print 0xaa55, which is a signature at the start of +# Video BIOS. + +import umachine as machine + +print(hex(machine.mem16[0xc0000])) diff --git a/extmod/modbtree.c b/extmod/modbtree.c index 0a0c5271c0..f21e7e4421 100644 --- a/extmod/modbtree.c +++ b/extmod/modbtree.c @@ -31,6 +31,8 @@ #include "py/nlr.h" #include "py/runtime.h" +#include "py/runtime0.h" +#include "py/stream.h" #if MICROPY_PY_BTREE @@ -291,6 +293,24 @@ STATIC mp_obj_t btree_subscr(mp_obj_t self_in, mp_obj_t index, mp_obj_t value) { } } +STATIC mp_obj_t btree_binary_op(mp_uint_t op, mp_obj_t lhs_in, mp_obj_t rhs_in) { + mp_obj_btree_t *self = MP_OBJ_TO_PTR(lhs_in); + switch (op) { + case MP_BINARY_OP_IN: { + mp_uint_t v; + DBT key, val; + key.data = (void*)mp_obj_str_get_data(rhs_in, &v); + key.size = v; + int res = __bt_get(self->db, &key, &val, 0); + CHECK_ERROR(res); + return mp_obj_new_bool(res != RET_SPECIAL); + } + default: + // op not supported + return MP_OBJ_NULL; + } +} + STATIC const mp_rom_map_elem_t btree_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_close), MP_ROM_PTR(&btree_close_obj) }, { MP_ROM_QSTR(MP_QSTR_get), MP_ROM_PTR(&btree_get_obj) }, @@ -310,27 +330,47 @@ STATIC const mp_obj_type_t btree_type = { .print = btree_print, .getiter = btree_getiter, .iternext = btree_iternext, + .binary_op = btree_binary_op, .subscr = btree_subscr, .locals_dict = (void*)&btree_locals_dict, }; +STATIC FILEVTABLE btree_stream_fvtable = { + mp_stream_posix_read, + mp_stream_posix_write, + mp_stream_posix_lseek, + mp_stream_posix_fsync +}; + STATIC mp_obj_t mod_btree_open(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) { static const mp_arg_t allowed_args[] = { - { MP_QSTR_server_side, MP_ARG_KW_ONLY | MP_ARG_BOOL, {.u_bool = false} }, + { MP_QSTR_flags, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, + { MP_QSTR_cachesize, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, + { MP_QSTR_pagesize, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, + { MP_QSTR_minkeypage, MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, }; - const char *fname = NULL; - if (pos_args[0] != mp_const_none) { - fname = mp_obj_str_get_str(pos_args[0]); - } + // Make sure we got a stream object + mp_get_stream_raise(pos_args[0], MP_STREAM_OP_READ | MP_STREAM_OP_WRITE | MP_STREAM_OP_IOCTL); struct { - mp_arg_val_t server_side; + mp_arg_val_t flags; + mp_arg_val_t cachesize; + mp_arg_val_t pagesize; + mp_arg_val_t minkeypage; } args; mp_arg_parse_all(n_args - 1, pos_args + 1, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, (mp_arg_val_t*)&args); - - DB *db = __bt_open(fname, /*flags*/O_CREAT | O_RDWR, /*mode*/0770, /*openinfo*/NULL, /*dflags*/0); + BTREEINFO openinfo = {0}; + openinfo.flags = args.flags.u_int; + openinfo.cachesize = args.cachesize.u_int; + openinfo.psize = args.pagesize.u_int; + openinfo.minkeypage = args.minkeypage.u_int; + + DB *db = __bt_open(pos_args[0], &btree_stream_fvtable, &openinfo, /*dflags*/0); + if (db == NULL) { + nlr_raise(mp_obj_new_exception_arg1(&mp_type_OSError, MP_OBJ_NEW_SMALL_INT(errno))); + } return MP_OBJ_FROM_PTR(btree_new(db)); } STATIC MP_DEFINE_CONST_FUN_OBJ_KW(mod_btree_open_obj, 1, mod_btree_open); diff --git a/extmod/modussl.c b/extmod/modussl_axtls.c index 4a3e297e1e..ce86263c2c 100644 --- a/extmod/modussl.c +++ b/extmod/modussl_axtls.c @@ -32,7 +32,7 @@ #include "py/runtime.h" #include "py/stream.h" -#if MICROPY_PY_USSL +#if MICROPY_PY_USSL && MICROPY_SSL_AXTLS #include "ssl.h" @@ -122,14 +122,21 @@ STATIC mp_uint_t socket_write(mp_obj_t o_in, const void *buf, mp_uint_t size, in return r; } +STATIC mp_obj_t socket_setblocking(mp_obj_t self_in, mp_obj_t flag_in) { + // Currently supports only blocking mode + (void)self_in; + if (!mp_obj_is_true(flag_in)) { + mp_not_implemented(""); + } + return mp_const_none; +} +STATIC MP_DEFINE_CONST_FUN_OBJ_2(socket_setblocking_obj, socket_setblocking); + STATIC mp_obj_t socket_close(mp_obj_t self_in) { mp_obj_ssl_socket_t *self = MP_OBJ_TO_PTR(self_in); ssl_free(self->ssl_sock); ssl_ctx_free(self->ssl_ctx); - - mp_obj_t dest[2]; - mp_load_method(self->sock, MP_QSTR_close, dest); - return mp_call_method_n_kw(0, 0, dest); + return mp_stream_close(self->sock); } STATIC MP_DEFINE_CONST_FUN_OBJ_1(socket_close_obj, socket_close); @@ -139,6 +146,7 @@ STATIC const mp_rom_map_elem_t ussl_socket_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_readinto), MP_ROM_PTR(&mp_stream_readinto_obj) }, { MP_ROM_QSTR(MP_QSTR_readline), MP_ROM_PTR(&mp_stream_unbuffered_readline_obj) }, { MP_ROM_QSTR(MP_QSTR_write), MP_ROM_PTR(&mp_stream_write_obj) }, + { MP_ROM_QSTR(MP_QSTR_setblocking), MP_ROM_PTR(&socket_setblocking_obj) }, { MP_ROM_QSTR(MP_QSTR_close), MP_ROM_PTR(&socket_close_obj) }, }; @@ -192,34 +200,4 @@ const mp_obj_module_t mp_module_ussl = { .globals = (mp_obj_dict_t*)&mp_module_ssl_globals, }; - -// These functions might be split to stream_posix.c. They are referenced by -// axtls os_port.h . -ssize_t mp_stream_posix_write(void *sock_obj, const void *buf, size_t len); -ssize_t mp_stream_posix_read(void *sock_obj, void *buf, size_t len); - -int mp_stream_errno; - -ssize_t mp_stream_posix_write(void *sock_obj, const void *buf, size_t len) { - struct _mp_obj_base_t *o = (struct _mp_obj_base_t *)sock_obj; - const mp_stream_p_t *stream_p = o->type->protocol; - mp_uint_t out_sz = stream_p->write(o, buf, len, &mp_stream_errno); - if (out_sz == MP_STREAM_ERROR) { - return -1; - } else { - return out_sz; - } -} - -ssize_t mp_stream_posix_read(void *sock_obj, void *buf, size_t len) { - struct _mp_obj_base_t *o = (struct _mp_obj_base_t *)sock_obj; - const mp_stream_p_t *stream_p = o->type->protocol; - mp_uint_t out_sz = stream_p->read(o, buf, len, &mp_stream_errno); - if (out_sz == MP_STREAM_ERROR) { - return -1; - } else { - return out_sz; - } -} - #endif // MICROPY_PY_USSL diff --git a/extmod/modwebrepl.c b/extmod/modwebrepl.c index 4f7b924e07..4e3780fa4d 100644 --- a/extmod/modwebrepl.c +++ b/extmod/modwebrepl.c @@ -38,6 +38,7 @@ #include "py/mphal.h" #endif #include "extmod/modwebsocket.h" +#include "genhdr/mpversion.h" #if MICROPY_PY_WEBREPL @@ -57,7 +58,7 @@ struct webrepl_file { char fname[64]; } __attribute__((packed)); -enum { PUT_FILE = 1, GET_FILE, LIST_DIR }; +enum { PUT_FILE = 1, GET_FILE, GET_VER }; enum { STATE_PASSWD, STATE_NORMAL }; typedef struct _mp_obj_webrepl_t { @@ -77,12 +78,6 @@ STATIC char denied_prompt[] = "\r\nAccess denied\r\n"; STATIC char webrepl_passwd[10]; -static inline void close_meth(mp_obj_t stream) { - mp_obj_t dest[2]; - mp_load_method(stream, MP_QSTR_close, dest); - mp_call_method_n_kw(0, 0, dest); -} - STATIC void write_webrepl(mp_obj_t websock, const void *buf, size_t len) { const mp_stream_p_t *sock_stream = mp_get_stream_raise(websock, MP_STREAM_OP_WRITE | MP_STREAM_OP_IOCTL); int err; @@ -117,7 +112,37 @@ STATIC mp_obj_t webrepl_make_new(const mp_obj_type_t *type, size_t n_args, size_ return o; } +STATIC int write_file_chunk(mp_obj_webrepl_t *self) { + const mp_stream_p_t *file_stream = + mp_get_stream_raise(self->cur_file, MP_STREAM_OP_READ | MP_STREAM_OP_WRITE | MP_STREAM_OP_IOCTL); + byte readbuf[2 + 256]; + int err; + mp_uint_t out_sz = file_stream->read(self->cur_file, readbuf + 2, sizeof(readbuf) - 2, &err); + if (out_sz == MP_STREAM_ERROR) { + return out_sz; + } + readbuf[0] = out_sz; + readbuf[1] = out_sz >> 8; + DEBUG_printf("webrepl: Sending %d bytes of file\n", out_sz); + write_webrepl(self->sock, readbuf, 2 + out_sz); + return out_sz; +} + STATIC void handle_op(mp_obj_webrepl_t *self) { + + // Handle operations not requiring opened file + + switch (self->hdr.type) { + case GET_VER: { + static char ver[] = {MICROPY_VERSION_MAJOR, MICROPY_VERSION_MINOR, MICROPY_VERSION_MICRO}; + write_webrepl(self->sock, ver, sizeof(ver)); + self->hdr_to_recv = sizeof(struct webrepl_file); + return; + } + } + + // Handle operations requiring opened file + mp_obj_t open_args[2] = { mp_obj_new_str(self->hdr.fname, strlen(self->hdr.fname), false), MP_OBJ_NEW_QSTR(MP_QSTR_rb) @@ -128,8 +153,6 @@ STATIC void handle_op(mp_obj_webrepl_t *self) { } self->cur_file = mp_builtin_open(2, open_args, (mp_map_t*)&mp_const_empty_map); - const mp_stream_p_t *file_stream = - mp_get_stream_raise(self->cur_file, MP_STREAM_OP_READ | MP_STREAM_OP_WRITE | MP_STREAM_OP_IOCTL); #if 0 struct mp_stream_seek_t seek = { .offset = self->hdr.offset, .whence = 0 }; @@ -143,24 +166,7 @@ STATIC void handle_op(mp_obj_webrepl_t *self) { if (self->hdr.type == PUT_FILE) { self->data_to_recv = self->hdr.size; } else if (self->hdr.type == GET_FILE) { - byte readbuf[2 + 256]; - int err; - // TODO: It's not ideal that we block connection while sending file - // and don't process any input. - while (1) { - mp_uint_t out_sz = file_stream->read(self->cur_file, readbuf + 2, sizeof(readbuf) - 2, &err); - assert(out_sz != MP_STREAM_ERROR); - readbuf[0] = out_sz; - readbuf[1] = out_sz >> 8; - DEBUG_printf("webrepl: Sending %d bytes of file\n", out_sz); - write_webrepl(self->sock, readbuf, 2 + out_sz); - if (out_sz == 0) { - break; - } - } - - write_webrepl_resp(self->sock, 0); - self->hdr_to_recv = sizeof(struct webrepl_file); + self->data_to_recv = 1; } } @@ -248,17 +254,27 @@ STATIC mp_uint_t _webrepl_read(mp_obj_t self_in, void *buf, mp_uint_t size, int buf_sz += sz; } - DEBUG_printf("webrepl: Writing %lu bytes to file\n", buf_sz); - int err; - mp_uint_t res = mp_stream_write_exactly(self->cur_file, filebuf, buf_sz, &err); - if (err != 0 || res != buf_sz) { - assert(0); + if (self->hdr.type == PUT_FILE) { + DEBUG_printf("webrepl: Writing %lu bytes to file\n", buf_sz); + int err; + mp_uint_t res = mp_stream_write_exactly(self->cur_file, filebuf, buf_sz, &err); + if (err != 0 || res != buf_sz) { + assert(0); + } + } else if (self->hdr.type == GET_FILE) { + assert(buf_sz == 1); + assert(self->data_to_recv == 0); + assert(filebuf[0] == 0); + mp_uint_t out_sz = write_file_chunk(self); + if (out_sz != 0) { + self->data_to_recv = 1; + } } if (self->data_to_recv == 0) { - close_meth(self->cur_file); + mp_stream_close(self->cur_file); self->hdr_to_recv = sizeof(struct webrepl_file); - DEBUG_printf("webrepl: Finished writing file\n"); + DEBUG_printf("webrepl: Finished file operation %d\n", self->hdr.type); write_webrepl_resp(self->sock, 0); } diff --git a/extmod/modwebsocket.c b/extmod/modwebsocket.c index 6354633af4..f46dac1773 100644 --- a/extmod/modwebsocket.c +++ b/extmod/modwebsocket.c @@ -73,11 +73,11 @@ STATIC mp_obj_t websocket_make_new(const mp_obj_type_t *type, size_t n_args, siz if (n_args > 1 && args[1] == mp_const_true) { o->opts |= BLOCKING_WRITE; } - return o; + return MP_OBJ_FROM_PTR(o); } STATIC mp_uint_t websocket_read(mp_obj_t self_in, void *buf, mp_uint_t size, int *errcode) { - mp_obj_websocket_t *self = self_in; + mp_obj_websocket_t *self = MP_OBJ_TO_PTR(self_in); const mp_stream_p_t *stream_p = mp_get_stream_raise(self->sock, MP_STREAM_OP_READ); while (1) { if (self->to_recv != 0) { @@ -219,7 +219,7 @@ no_payload: } STATIC mp_uint_t websocket_write(mp_obj_t self_in, const void *buf, mp_uint_t size, int *errcode) { - mp_obj_websocket_t *self = self_in; + mp_obj_websocket_t *self = MP_OBJ_TO_PTR(self_in); assert(size < 0x10000); byte header[4] = {0x80 | (self->opts & FRAME_OPCODE_MASK)}; int hdr_sz; @@ -280,12 +280,13 @@ STATIC mp_obj_t websocket_close(mp_obj_t self_in) { } STATIC MP_DEFINE_CONST_FUN_OBJ_1(websocket_close_obj, websocket_close); -STATIC const mp_map_elem_t websocket_locals_dict_table[] = { - { MP_OBJ_NEW_QSTR(MP_QSTR_read), (mp_obj_t)&mp_stream_read_obj }, - { MP_OBJ_NEW_QSTR(MP_QSTR_readinto), (mp_obj_t)&mp_stream_readinto_obj }, - { MP_OBJ_NEW_QSTR(MP_QSTR_write), (mp_obj_t)&mp_stream_write_obj }, - { MP_OBJ_NEW_QSTR(MP_QSTR_ioctl), (mp_obj_t)&mp_stream_ioctl_obj }, - { MP_OBJ_NEW_QSTR(MP_QSTR_close), (mp_obj_t)&websocket_close_obj }, +STATIC const mp_rom_map_elem_t websocket_locals_dict_table[] = { + { MP_ROM_QSTR(MP_QSTR_read), MP_ROM_PTR(&mp_stream_read_obj) }, + { MP_ROM_QSTR(MP_QSTR_readinto), MP_ROM_PTR(&mp_stream_readinto_obj) }, + { MP_ROM_QSTR(MP_QSTR_readline), MP_ROM_PTR(&mp_stream_unbuffered_readline_obj) }, + { MP_ROM_QSTR(MP_QSTR_write), MP_ROM_PTR(&mp_stream_write_obj) }, + { MP_ROM_QSTR(MP_QSTR_ioctl), MP_ROM_PTR(&mp_stream_ioctl_obj) }, + { MP_ROM_QSTR(MP_QSTR_close), MP_ROM_PTR(&websocket_close_obj) }, }; STATIC MP_DEFINE_CONST_DICT(websocket_locals_dict, websocket_locals_dict_table); @@ -300,12 +301,12 @@ STATIC const mp_obj_type_t websocket_type = { .name = MP_QSTR_websocket, .make_new = websocket_make_new, .protocol = &websocket_stream_p, - .locals_dict = (mp_obj_t)&websocket_locals_dict, + .locals_dict = (void*)&websocket_locals_dict, }; -STATIC const mp_map_elem_t websocket_module_globals_table[] = { - { MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_websocket) }, - { MP_OBJ_NEW_QSTR(MP_QSTR_websocket), (mp_obj_t)&websocket_type }, +STATIC const mp_rom_map_elem_t websocket_module_globals_table[] = { + { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_websocket) }, + { MP_ROM_QSTR(MP_QSTR_websocket), MP_ROM_PTR(&websocket_type) }, }; STATIC MP_DEFINE_CONST_DICT(websocket_module_globals, websocket_module_globals_table); diff --git a/extmod/vfs_fat.c b/extmod/vfs_fat.c index df5cbb0d62..a691ee062c 100644 --- a/extmod/vfs_fat.c +++ b/extmod/vfs_fat.c @@ -90,6 +90,14 @@ STATIC mp_obj_t fat_vfs_remove(mp_obj_t vfs_in, mp_obj_t path_in) { } STATIC MP_DEFINE_CONST_FUN_OBJ_2(fat_vfs_remove_obj, fat_vfs_remove); +STATIC mp_obj_t fat_vfs_rmdir(mp_obj_t vfs_in, mp_obj_t path_in) { + // TODO: Currently just redirects to fat_vfs_remove(), which are + // backed by the same underlying FatFs function. Should at least + // check that path is actually a dir. + return fat_vfs_remove(vfs_in, path_in); +} +STATIC MP_DEFINE_CONST_FUN_OBJ_2(fat_vfs_rmdir_obj, fat_vfs_rmdir); + STATIC mp_obj_t fat_vfs_rename(mp_obj_t vfs_in, mp_obj_t path_in, mp_obj_t path_out) { (void)vfs_in; const char *old_path = mp_obj_str_get_str(path_in); @@ -247,6 +255,7 @@ STATIC const mp_rom_map_elem_t fat_vfs_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_open), MP_ROM_PTR(&fat_vfs_open_obj) }, { MP_ROM_QSTR(MP_QSTR_listdir), MP_ROM_PTR(&fat_vfs_listdir_obj) }, { MP_ROM_QSTR(MP_QSTR_mkdir), MP_ROM_PTR(&fat_vfs_mkdir_obj) }, + { MP_ROM_QSTR(MP_QSTR_rmdir), MP_ROM_PTR(&fat_vfs_rmdir_obj) }, { MP_ROM_QSTR(MP_QSTR_chdir), MP_ROM_PTR(&fat_vfs_chdir_obj) }, { MP_ROM_QSTR(MP_QSTR_getcwd), MP_ROM_PTR(&fat_vfs_getcwd_obj) }, { MP_ROM_QSTR(MP_QSTR_remove), MP_ROM_PTR(&fat_vfs_remove_obj) }, diff --git a/lib/axtls b/lib/axtls -Subproject c2690dc100f272cb01375dcf2f2be7105d0fb7c +Subproject 67d27df4b5d097e146599fc4fb160a2adcbf563 diff --git a/lib/berkeley-db-1.xx b/lib/berkeley-db-1.xx -Subproject 7b31cae3c0892a7fca2089cb0354a4920c98c41 +Subproject 78a4787948bb80cbfafcfd7910f95f61a4dd0d4 diff --git a/lib/embed/abort_.c b/lib/embed/abort_.c new file mode 100644 index 0000000000..a6df8143c3 --- /dev/null +++ b/lib/embed/abort_.c @@ -0,0 +1,7 @@ +#include <py/runtime.h> + +NORETURN void abort_(void); + +NORETURN void abort_(void) { + nlr_raise(mp_obj_new_exception_msg(&mp_type_RuntimeError, "abort() called")); +} diff --git a/lib/utils/pyexec.c b/lib/utils/pyexec.c index ddecb6a8d8..411ee969d6 100644 --- a/lib/utils/pyexec.c +++ b/lib/utils/pyexec.c @@ -40,7 +40,7 @@ #include "irq.h" #include "usb.h" #endif -#include "readline.h" +#include "lib/mp-readline/readline.h" #include "lib/utils/pyexec.h" #include "genhdr/mpversion.h" diff --git a/minimal/uart_extra.c b/lib/utils/stdout_helpers.c index 9b70f74867..6efe10b92c 100644 --- a/minimal/uart_extra.c +++ b/lib/utils/stdout_helpers.c @@ -4,7 +4,7 @@ #include "py/mphal.h" /* - * Extra UART functions + * Extra stdout functions * These can be either optimized for a particular port, or reference * implementation below can be used. */ diff --git a/logo/1bit-logo.png b/logo/1bit-logo.png Binary files differnew file mode 100644 index 0000000000..42927f3140 --- /dev/null +++ b/logo/1bit-logo.png diff --git a/minimal/Makefile b/minimal/Makefile index 6f37afd8c1..02096f9f0e 100644 --- a/minimal/Makefile +++ b/minimal/Makefile @@ -45,7 +45,7 @@ LIBS = SRC_C = \ main.c \ uart_core.c \ - uart_extra.c \ + lib/utils/stdout_helpers.c \ lib/utils/printf.c \ lib/utils/pyexec.c \ lib/libc/string0.c \ diff --git a/minimal/mpconfigport.h b/minimal/mpconfigport.h index c8f9838650..165b058794 100644 --- a/minimal/mpconfigport.h +++ b/minimal/mpconfigport.h @@ -16,6 +16,7 @@ #define MICROPY_MEM_STATS (0) #define MICROPY_DEBUG_PRINTERS (0) #define MICROPY_ENABLE_GC (1) +#define MICROPY_GC_ALLOC_THRESHOLD (0) #define MICROPY_REPL_EVENT_DRIVEN (0) #define MICROPY_HELPER_REPL (1) #define MICROPY_HELPER_LEXER_UNIX (0) diff --git a/mpy-cross/Makefile b/mpy-cross/Makefile index da4d4646a7..3f99566209 100644 --- a/mpy-cross/Makefile +++ b/mpy-cross/Makefile @@ -37,19 +37,24 @@ endif ifeq ($(UNAME_S),Darwin) CC = clang # Use clang syntax for map file -LDFLAGS_ARCH = -Wl,-map,$@.map +LDFLAGS_ARCH = -Wl,-map,$@.map -Wl,-dead_strip else # Use gcc syntax for map file -LDFLAGS_ARCH = -Wl,-Map=$@.map,--cref +LDFLAGS_ARCH = -Wl,-Map=$@.map,--cref -Wl,--gc-sections endif LDFLAGS = $(LDFLAGS_MOD) $(LDFLAGS_ARCH) -lm $(LDFLAGS_EXTRA) -LDFLAGS += -Wl,--gc-sections # source files SRC_C = \ main.c \ gccollect.c \ +ifeq ($(OS),Windows_NT) + ifeq (,$(findstring MSYS,$(UNAME_S))) + SRC_C += windows/fmode.c + endif +endif + OBJ = $(PY_O) OBJ += $(addprefix $(BUILD)/, $(SRC_C:.c=.o)) diff --git a/mpy-cross/main.c b/mpy-cross/main.c index 4393d00ff0..5e5bc73304 100644 --- a/mpy-cross/main.c +++ b/mpy-cross/main.c @@ -35,6 +35,9 @@ #include "py/runtime.h" #include "py/gc.h" #include "py/stackctrl.h" +#ifdef _WIN32 +#include "windows/fmode.h" +#endif // Command line options, with their defaults STATIC uint emit_opt = MP_EMIT_OPT_NONE; @@ -185,6 +188,9 @@ MP_NOINLINE int main_(int argc, char **argv) { gc_init(heap, heap + heap_size); mp_init(); +#ifdef _WIN32 + set_fmode_binary(); +#endif mp_obj_list_init(mp_sys_path, 0); mp_obj_list_init(mp_sys_argv, 0); diff --git a/mpy-cross/mpconfigport.h b/mpy-cross/mpconfigport.h index 031c350116..b3f7d9c3cd 100644 --- a/mpy-cross/mpconfigport.h +++ b/mpy-cross/mpconfigport.h @@ -87,6 +87,10 @@ #ifdef __LP64__ typedef long mp_int_t; // must be pointer size typedef unsigned long mp_uint_t; // must be pointer size +#elif defined ( __MINGW32__ ) && defined( _WIN64 ) +#include <stdint.h> +typedef __int64 mp_int_t; +typedef unsigned __int64 mp_uint_t; #else // These are definitions for machines where sizeof(int) == sizeof(void*), // regardless for actual size. @@ -115,6 +119,8 @@ typedef const void *machine_const_ptr_t; // must be of pointer size // We need to provide a declaration/definition of alloca() #ifdef __FreeBSD__ #include <stdlib.h> +#elif defined( _WIN32 ) +#include <malloc.h> #else #include <alloca.h> #endif @@ -152,6 +152,12 @@ void gc_init(void *start, void *end) { // allow auto collection MP_STATE_MEM(gc_auto_collect_enabled) = 1; + #if MICROPY_GC_ALLOC_THRESHOLD + // by default, maxuint for gc threshold, effectively turning gc-by-threshold off + MP_STATE_MEM(gc_alloc_threshold) = (size_t)-1; + MP_STATE_MEM(gc_alloc_amount) = 0; + #endif + #if MICROPY_PY_THREAD mp_thread_mutex_init(&MP_STATE_MEM(gc_mutex)); #endif @@ -294,6 +300,9 @@ STATIC void gc_sweep(void) { void gc_collect_start(void) { GC_ENTER(); MP_STATE_MEM(gc_lock_depth)++; + #if MICROPY_GC_ALLOC_THRESHOLD + MP_STATE_MEM(gc_alloc_amount) = 0; + #endif MP_STATE_MEM(gc_stack_overflow) = 0; MP_STATE_MEM(gc_sp) = MP_STATE_MEM(gc_stack); // Trace root pointers. This relies on the root pointers being organised @@ -405,6 +414,15 @@ void *gc_alloc(size_t n_bytes, bool has_finaliser) { size_t start_block; size_t n_free = 0; int collected = !MP_STATE_MEM(gc_auto_collect_enabled); + + #if MICROPY_GC_ALLOC_THRESHOLD + if (!collected && MP_STATE_MEM(gc_alloc_amount) >= MP_STATE_MEM(gc_alloc_threshold)) { + GC_EXIT(); + gc_collect(); + GC_ENTER(); + } + #endif + for (;;) { // look for a run of n_blocks available blocks @@ -456,6 +474,10 @@ found: void *ret_ptr = (void*)(MP_STATE_MEM(gc_pool_start) + start_block * BYTES_PER_BLOCK); DEBUG_printf("gc_alloc(%p)\n", ret_ptr); + #if MICROPY_GC_ALLOC_THRESHOLD + MP_STATE_MEM(gc_alloc_amount) += n_blocks; + #endif + GC_EXIT(); // zero out the additional bytes of the newly allocated blocks diff --git a/py/mkrules.mk b/py/mkrules.mk index b77f8d600f..3cf0e30584 100644 --- a/py/mkrules.mk +++ b/py/mkrules.mk @@ -127,7 +127,7 @@ lib: $(OBJ) $(AR) rcs libmicropython.a $^ clean: - $(RM) -rf $(BUILD) + $(RM) -rf $(BUILD) $(CLEAN_EXTRA) .PHONY: clean print-cfg: diff --git a/py/modgc.c b/py/modgc.c index d68ff7e6c0..976fb89980 100644 --- a/py/modgc.c +++ b/py/modgc.c @@ -83,6 +83,25 @@ STATIC mp_obj_t gc_mem_alloc(void) { } MP_DEFINE_CONST_FUN_OBJ_0(gc_mem_alloc_obj, gc_mem_alloc); +#if MICROPY_GC_ALLOC_THRESHOLD +STATIC mp_obj_t gc_threshold(size_t n_args, const mp_obj_t *args) { + if (n_args == 0) { + if (MP_STATE_MEM(gc_alloc_threshold) == (size_t)-1) { + return MP_OBJ_NEW_SMALL_INT(-1); + } + return mp_obj_new_int(MP_STATE_MEM(gc_alloc_threshold) * MICROPY_BYTES_PER_GC_BLOCK); + } + mp_int_t val = mp_obj_get_int(args[0]); + if (val < 0) { + MP_STATE_MEM(gc_alloc_threshold) = (size_t)-1; + } else { + MP_STATE_MEM(gc_alloc_threshold) = val / MICROPY_BYTES_PER_GC_BLOCK; + } + return mp_const_none; +} +MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(gc_threshold_obj, 0, 1, gc_threshold); +#endif + STATIC const mp_rom_map_elem_t mp_module_gc_globals_table[] = { { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_gc) }, { MP_ROM_QSTR(MP_QSTR_collect), MP_ROM_PTR(&gc_collect_obj) }, @@ -91,6 +110,9 @@ STATIC const mp_rom_map_elem_t mp_module_gc_globals_table[] = { { MP_ROM_QSTR(MP_QSTR_isenabled), MP_ROM_PTR(&gc_isenabled_obj) }, { MP_ROM_QSTR(MP_QSTR_mem_free), MP_ROM_PTR(&gc_mem_free_obj) }, { MP_ROM_QSTR(MP_QSTR_mem_alloc), MP_ROM_PTR(&gc_mem_alloc_obj) }, + #if MICROPY_GC_ALLOC_THRESHOLD + { MP_ROM_QSTR(MP_QSTR_threshold), MP_ROM_PTR(&gc_threshold_obj) }, + #endif }; STATIC MP_DEFINE_CONST_DICT(mp_module_gc_globals, mp_module_gc_globals_table); diff --git a/py/mpconfig.h b/py/mpconfig.h index aec5d40826..3808df7430 100644 --- a/py/mpconfig.h +++ b/py/mpconfig.h @@ -107,6 +107,12 @@ #define MICROPY_ALLOC_GC_STACK_SIZE (64) #endif +// Support automatic GC when reaching allocation threshold, +// configurable by gc.threshold(). +#ifndef MICROPY_GC_ALLOC_THRESHOLD +#define MICROPY_GC_ALLOC_THRESHOLD (1) +#endif + // Number of bytes to allocate initially when creating new chunks to store // interned string data. Smaller numbers lead to more chunks being needed // and more wastage at the end of the chunk. Larger numbers lead to wasted @@ -516,6 +522,12 @@ typedef double mp_float_t; #define MICROPY_STREAMS_NON_BLOCK (0) #endif +// Whether to provide stream functions with POSIX-like signatures +// (useful for porting existing libraries to MicroPython). +#ifndef MICROPY_STREAMS_POSIX_API +#define MICROPY_STREAMS_POSIX_API (0) +#endif + // Whether to call __init__ when importing builtin modules for the first time #ifndef MICROPY_MODULE_BUILTIN_INIT #define MICROPY_MODULE_BUILTIN_INIT (0) @@ -584,6 +596,11 @@ typedef double mp_float_t; #define MICROPY_PY_ASYNC_AWAIT (1) #endif +// Issue a warning when comparing str and bytes objects +#ifndef MICROPY_PY_STR_BYTES_CMP_WARN +#define MICROPY_PY_STR_BYTES_CMP_WARN (0) +#endif + // Whether str object is proper unicode #ifndef MICROPY_PY_BUILTINS_STR_UNICODE #define MICROPY_PY_BUILTINS_STR_UNICODE (0) @@ -594,6 +611,11 @@ typedef double mp_float_t; #define MICROPY_PY_BUILTINS_STR_CENTER (0) #endif +// Whether str.partition()/str.rpartition() method provided +#ifndef MICROPY_PY_BUILTINS_STR_PARTITION +#define MICROPY_PY_BUILTINS_STR_PARTITION (0) +#endif + // Whether str.splitlines() method provided #ifndef MICROPY_PY_BUILTINS_STR_SPLITLINES #define MICROPY_PY_BUILTINS_STR_SPLITLINES (0) @@ -1051,6 +1073,11 @@ typedef double mp_float_t; #define MP_NOINLINE __attribute__((noinline)) #endif +// Modifier for functions which should be always inlined +#ifndef MP_ALWAYSINLINE +#define MP_ALWAYSINLINE __attribute__((always_inline)) +#endif + // Condition is likely to be true, to help branch prediction #ifndef MP_LIKELY #define MP_LIKELY(x) __builtin_expect((x), 1) diff --git a/py/mpstate.h b/py/mpstate.h index 281795773f..439ed66066 100644 --- a/py/mpstate.h +++ b/py/mpstate.h @@ -76,6 +76,11 @@ typedef struct _mp_state_mem_t { // you can still allocate/free memory and also explicitly call gc_collect. uint16_t gc_auto_collect_enabled; + #if MICROPY_GC_ALLOC_THRESHOLD + size_t gc_alloc_amount; + size_t gc_alloc_threshold; + #endif + size_t gc_last_free_atb_index; #if MICROPY_PY_GC_COLLECT_RETVAL diff --git a/py/nlrthumb.c b/py/nlrthumb.c index a61c73c036..08a71ac7d6 100644 --- a/py/nlrthumb.c +++ b/py/nlrthumb.c @@ -29,6 +29,8 @@ #if (!defined(MICROPY_NLR_SETJMP) || !MICROPY_NLR_SETJMP) && (defined(__thumb2__) || defined(__thumb__) || defined(__arm__)) +#undef nlr_push + // We only need the functions here if we are on arm/thumb, and we are not // using setjmp/longjmp. // @@ -71,7 +73,7 @@ __attribute__((naked)) unsigned int nlr_push(nlr_buf_t *nlr) { return 0; // needed to silence compiler warning } -unsigned int nlr_push_tail(nlr_buf_t *nlr) { +__attribute__((used)) unsigned int nlr_push_tail(nlr_buf_t *nlr) { nlr_buf_t **top = &MP_STATE_THREAD(nlr_top); nlr->prev = *top; *top = nlr; diff --git a/py/nlrx64.S b/py/nlrx64.S index ad2b66fdb2..caea35de2b 100644 --- a/py/nlrx64.S +++ b/py/nlrx64.S @@ -37,8 +37,10 @@ #if defined(__APPLE__) && defined(__MACH__) #define NLR_TOP (_mp_state_ctx + NLR_TOP_OFFSET) +#define MP_THREAD_GET_STATE _mp_thread_get_state #else #define NLR_TOP (mp_state_ctx + NLR_TOP_OFFSET) +#define MP_THREAD_GET_STATE mp_thread_get_state #endif // offset of nlr_top within mp_state_thread_t structure @@ -87,7 +89,7 @@ _nlr_push: movq %rdi, NLR_TOP(%rip) # stor new nlr_buf (to make linked list) #else movq %rdi, %rbp # since we make a call, must save rdi in rbp - callq mp_thread_get_state # get mp_state_thread ptr into rax + callq MP_THREAD_GET_STATE # get mp_state_thread ptr into rax movq NLR_TOP_TH_OFF(%rax), %rsi # get thread.nlr_top (last nlr_buf) movq %rsi, (%rbp) # store it movq %rbp, NLR_TOP_TH_OFF(%rax) # store new nlr_buf (to make linked list) @@ -117,7 +119,7 @@ _nlr_pop: movq (%rax), %rax # load prev nlr_buf movq %rax, NLR_TOP(%rip) # store prev nlr_buf (to unlink list) #else - callq mp_thread_get_state # get mp_state_thread ptr into rax + callq MP_THREAD_GET_STATE # get mp_state_thread ptr into rax movq NLR_TOP_TH_OFF(%rax), %rdi # get thread.nlr_top (last nlr_buf) movq (%rdi), %rdi # load prev nlr_buf movq %rdi, NLR_TOP_TH_OFF(%rax) # store prev nlr_buf (to unlink list) @@ -150,7 +152,7 @@ nlr_jump: movq %rax, NLR_TOP(%rip) # store prev nlr_buf (to unlink list) #else movq %rdi, %rbp # put return value in rbp - callq mp_thread_get_state # get thread ptr in rax + callq MP_THREAD_GET_STATE # get thread ptr in rax movq %rax, %rsi # put thread ptr in rsi movq %rbp, %rax # put return value to rax (for je .fail) movq NLR_TOP_TH_OFF(%rsi), %rdi # get thread.nlr_top in rdi @@ -192,10 +192,16 @@ bool mp_obj_equal(mp_obj_t o1, mp_obj_t o2) { return mp_obj_str_equal(o1, o2); } else { // a string is never equal to anything else - return false; + goto str_cmp_err; } } else if (MP_OBJ_IS_STR(o2)) { // o1 is not a string (else caught above), so the objects are not equal + str_cmp_err: + #if MICROPY_PY_STR_BYTES_CMP_WARN + if (MP_OBJ_IS_TYPE(o1, &mp_type_bytes) || MP_OBJ_IS_TYPE(o2, &mp_type_bytes)) { + mp_warning("Comparison between bytes and str"); + } + #endif return false; } diff --git a/py/objstr.c b/py/objstr.c index e51c371f7b..a6ee617c03 100644 --- a/py/objstr.c +++ b/py/objstr.c @@ -1683,6 +1683,7 @@ STATIC mp_obj_t str_count(size_t n_args, const mp_obj_t *args) { return MP_OBJ_NEW_SMALL_INT(num_occurrences); } +#if MICROPY_PY_BUILTINS_STR_PARTITION STATIC mp_obj_t str_partitioner(mp_obj_t self_in, mp_obj_t arg, mp_int_t direction) { assert(MP_OBJ_IS_STR_OR_BYTES(self_in)); mp_obj_type_t *self_type = mp_obj_get_type(self_in); @@ -1732,6 +1733,7 @@ STATIC mp_obj_t str_partition(mp_obj_t self_in, mp_obj_t arg) { STATIC mp_obj_t str_rpartition(mp_obj_t self_in, mp_obj_t arg) { return str_partitioner(self_in, arg, -1); } +#endif // Supposedly not too critical operations, so optimize for code size STATIC mp_obj_t str_caseconv(unichar (*op)(unichar), mp_obj_t self_in) { @@ -1875,8 +1877,10 @@ MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(str_rstrip_obj, 1, 2, str_rstrip); MP_DEFINE_CONST_FUN_OBJ_KW(str_format_obj, 1, mp_obj_str_format); MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(str_replace_obj, 3, 4, str_replace); MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(str_count_obj, 2, 4, str_count); +#if MICROPY_PY_BUILTINS_STR_PARTITION MP_DEFINE_CONST_FUN_OBJ_2(str_partition_obj, str_partition); MP_DEFINE_CONST_FUN_OBJ_2(str_rpartition_obj, str_rpartition); +#endif MP_DEFINE_CONST_FUN_OBJ_1(str_lower_obj, str_lower); MP_DEFINE_CONST_FUN_OBJ_1(str_upper_obj, str_upper); MP_DEFINE_CONST_FUN_OBJ_1(str_isspace_obj, str_isspace); @@ -1915,11 +1919,13 @@ STATIC const mp_rom_map_elem_t str8_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_format), MP_ROM_PTR(&str_format_obj) }, { MP_ROM_QSTR(MP_QSTR_replace), MP_ROM_PTR(&str_replace_obj) }, { MP_ROM_QSTR(MP_QSTR_count), MP_ROM_PTR(&str_count_obj) }, + #if MICROPY_PY_BUILTINS_STR_PARTITION { MP_ROM_QSTR(MP_QSTR_partition), MP_ROM_PTR(&str_partition_obj) }, { MP_ROM_QSTR(MP_QSTR_rpartition), MP_ROM_PTR(&str_rpartition_obj) }, -#if MICROPY_PY_BUILTINS_STR_CENTER + #endif + #if MICROPY_PY_BUILTINS_STR_CENTER { MP_ROM_QSTR(MP_QSTR_center), MP_ROM_PTR(&str_center_obj) }, -#endif + #endif { MP_ROM_QSTR(MP_QSTR_lower), MP_ROM_PTR(&str_lower_obj) }, { MP_ROM_QSTR(MP_QSTR_upper), MP_ROM_PTR(&str_upper_obj) }, { MP_ROM_QSTR(MP_QSTR_isspace), MP_ROM_PTR(&str_isspace_obj) }, diff --git a/py/objstringio.c b/py/objstringio.c index abd4e835e8..eb2e516bb3 100644 --- a/py/objstringio.c +++ b/py/objstringio.c @@ -75,13 +75,18 @@ STATIC mp_uint_t stringio_write(mp_obj_t o_in, const void *buf, mp_uint_t size, (void)errcode; mp_obj_stringio_t *o = MP_OBJ_TO_PTR(o_in); check_stringio_is_open(o); - mp_uint_t remaining = o->vstr->alloc - o->pos; - if (size > remaining) { + mp_int_t remaining = o->vstr->alloc - o->pos; + mp_uint_t org_len = o->vstr->len; + if ((mp_int_t)size > remaining) { // Take all what's already allocated... o->vstr->len = o->vstr->alloc; // ... and add more vstr_add_len(o->vstr, size - remaining); } + // If there was a seek past EOF, clear the hole + if (o->pos > org_len) { + memset(o->vstr->buf + org_len, 0, o->pos - org_len); + } memcpy(o->vstr->buf + o->pos, buf, size); o->pos += size; if (o->pos > o->vstr->len) { @@ -90,6 +95,33 @@ STATIC mp_uint_t stringio_write(mp_obj_t o_in, const void *buf, mp_uint_t size, return size; } +STATIC mp_uint_t stringio_ioctl(mp_obj_t o_in, mp_uint_t request, uintptr_t arg, int *errcode) { + (void)errcode; + mp_obj_stringio_t *o = MP_OBJ_TO_PTR(o_in); + switch (request) { + case MP_STREAM_SEEK: { + struct mp_stream_seek_t *s = (struct mp_stream_seek_t*)arg; + mp_uint_t ref = 0; + switch (s->whence) { + case 1: // SEEK_CUR + ref = o->pos; + break; + case 2: // SEEK_END + ref = o->vstr->len; + break; + } + o->pos = ref + s->offset; + s->offset = o->pos; + return 0; + } + case MP_STREAM_FLUSH: + return 0; + default: + *errcode = MP_EINVAL; + return MP_STREAM_ERROR; + } +} + #define STREAM_TO_CONTENT_TYPE(o) (((o)->base.type == &mp_type_stringio) ? &mp_type_str : &mp_type_bytes) STATIC mp_obj_t stringio_getvalue(mp_obj_t self_in) { @@ -148,6 +180,8 @@ STATIC const mp_rom_map_elem_t stringio_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_readall), MP_ROM_PTR(&mp_stream_readall_obj) }, { MP_ROM_QSTR(MP_QSTR_readline), MP_ROM_PTR(&mp_stream_unbuffered_readline_obj) }, { MP_ROM_QSTR(MP_QSTR_write), MP_ROM_PTR(&mp_stream_write_obj) }, + { MP_ROM_QSTR(MP_QSTR_seek), MP_ROM_PTR(&mp_stream_seek_obj) }, + { MP_ROM_QSTR(MP_QSTR_flush), MP_ROM_PTR(&mp_stream_flush_obj) }, { MP_ROM_QSTR(MP_QSTR_close), MP_ROM_PTR(&stringio_close_obj) }, { MP_ROM_QSTR(MP_QSTR_getvalue), MP_ROM_PTR(&stringio_getvalue_obj) }, { MP_ROM_QSTR(MP_QSTR___enter__), MP_ROM_PTR(&mp_identity_obj) }, @@ -159,12 +193,14 @@ STATIC MP_DEFINE_CONST_DICT(stringio_locals_dict, stringio_locals_dict_table); STATIC const mp_stream_p_t stringio_stream_p = { .read = stringio_read, .write = stringio_write, + .ioctl = stringio_ioctl, .is_text = true, }; STATIC const mp_stream_p_t bytesio_stream_p = { .read = stringio_read, .write = stringio_write, + .ioctl = stringio_ioctl, }; const mp_obj_type_t mp_type_stringio = { diff --git a/py/objstrunicode.c b/py/objstrunicode.c index c6c775d109..8444a26892 100644 --- a/py/objstrunicode.c +++ b/py/objstrunicode.c @@ -116,7 +116,14 @@ STATIC mp_obj_t uni_unary_op(mp_uint_t op, mp_obj_t self_in) { // be capped to the first/last character of the string, depending on is_slice. const byte *str_index_to_ptr(const mp_obj_type_t *type, const byte *self_data, size_t self_len, mp_obj_t index, bool is_slice) { - (void)type; + // All str functions also handle bytes objects, and they call str_index_to_ptr(), + // so it must handle bytes. + if (type == &mp_type_bytes) { + // Taken from objstr.c:str_index_to_ptr() + mp_uint_t index_val = mp_get_index(type, self_len, index, is_slice); + return self_data + index_val; + } + mp_int_t i; // Copied from mp_get_index; I don't want bounds checking, just give me // the integer as-is. (I can't bounds-check without scanning the whole @@ -142,26 +149,29 @@ const byte *str_index_to_ptr(const mp_obj_type_t *type, const byte *self_data, s } } ++s; - } else if (!i) { - return self_data; // Shortcut - str[0] is its base pointer } else { // Positive indexing, correspondingly, counts from the start of the string. // It's assumed that negative indexing will generally be used with small // absolute values (eg str[-1], not str[-1000000]), which means it'll be // more efficient this way. - for (s = self_data; true; ++s) { + s = self_data; + while (1) { + // First check out-of-bounds if (s >= top) { if (is_slice) { return top; } nlr_raise(mp_obj_new_exception_msg_varg(&mp_type_IndexError, "string index out of range")); } + // Then check completion + if (i-- == 0) { + break; + } + // Then skip UTF-8 char + ++s; while (UTF8_IS_CONT(*s)) { ++s; } - if (!i--) { - return s; - } } } return s; @@ -236,11 +246,13 @@ STATIC const mp_rom_map_elem_t struni_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_format), MP_ROM_PTR(&str_format_obj) }, { MP_ROM_QSTR(MP_QSTR_replace), MP_ROM_PTR(&str_replace_obj) }, { MP_ROM_QSTR(MP_QSTR_count), MP_ROM_PTR(&str_count_obj) }, + #if MICROPY_PY_BUILTINS_STR_PARTITION { MP_ROM_QSTR(MP_QSTR_partition), MP_ROM_PTR(&str_partition_obj) }, { MP_ROM_QSTR(MP_QSTR_rpartition), MP_ROM_PTR(&str_rpartition_obj) }, -#if MICROPY_PY_BUILTINS_STR_CENTER + #endif + #if MICROPY_PY_BUILTINS_STR_CENTER { MP_ROM_QSTR(MP_QSTR_center), MP_ROM_PTR(&str_center_obj) }, -#endif + #endif { MP_ROM_QSTR(MP_QSTR_lower), MP_ROM_PTR(&str_lower_obj) }, { MP_ROM_QSTR(MP_QSTR_upper), MP_ROM_PTR(&str_upper_obj) }, { MP_ROM_QSTR(MP_QSTR_isspace), MP_ROM_PTR(&str_isspace_obj) }, @@ -20,8 +20,11 @@ INC += -I../lib INC += -I../lib/netutils ifeq ($(MICROPY_PY_USSL),1) -CFLAGS_MOD += -DMICROPY_PY_USSL=1 -I../lib/axtls/ssl -I../lib/axtls/crypto -I../lib/axtls/config -LDFLAGS_MOD += -L../lib/axtls/_stage -laxtls +CFLAGS_MOD += -DMICROPY_PY_USSL=1 +ifeq ($(MICROPY_SSL_AXTLS),1) +CFLAGS_MOD += -DMICROPY_SSL_AXTLS=1 -I../lib/axtls/ssl -I../lib/axtls/crypto -I../lib/axtls/config +LDFLAGS_MOD += -Lbuild -laxtls +endif endif #ifeq ($(MICROPY_PY_LWIP),1) @@ -66,7 +69,7 @@ endif ifeq ($(MICROPY_PY_BTREE),1) BTREE_DIR = lib/berkeley-db-1.xx -CFLAGS_MOD += -D__DBINTERFACE_PRIVATE=1 +CFLAGS_MOD += -D__DBINTERFACE_PRIVATE=1 -Dmpool_error=printf -Dabort=abort_ -Dvirt_fd_t=mp_obj_t "-DVIRT_FD_T_HEADER=<py/obj.h>" INC += -I../$(BTREE_DIR)/PORT/include SRC_MOD += extmod/modbtree.c SRC_MOD += $(addprefix $(BTREE_DIR)/,\ @@ -205,7 +208,7 @@ PY_O_BASENAME = \ ../extmod/machine_pinbase.o \ ../extmod/machine_pulse.o \ ../extmod/machine_i2c.o \ - ../extmod/modussl.o \ + ../extmod/modussl_axtls.o \ ../extmod/modurandom.o \ ../extmod/modwebsocket.o \ ../extmod/modwebrepl.o \ @@ -218,6 +221,7 @@ PY_O_BASENAME = \ ../extmod/vfs_fat_lexer.o \ ../extmod/vfs_fat_misc.o \ ../extmod/moduos_dupterm.o \ + ../lib/embed/abort_.o \ # prepend the build destination prefix to the py object files PY_O = $(addprefix $(PY_BUILD)/, $(PY_O_BASENAME)) diff --git a/py/stream.c b/py/stream.c index 4fcc151dca..473eb96904 100644 --- a/py/stream.c +++ b/py/stream.c @@ -267,12 +267,24 @@ void mp_stream_write_adaptor(void *self, const char *buf, size_t len) { mp_stream_write(MP_OBJ_FROM_PTR(self), buf, len, MP_STREAM_RW_WRITE); } -STATIC mp_obj_t stream_write_method(mp_obj_t self_in, mp_obj_t arg) { +STATIC mp_obj_t stream_write_method(size_t n_args, const mp_obj_t *args) { mp_buffer_info_t bufinfo; - mp_get_buffer_raise(arg, &bufinfo, MP_BUFFER_READ); - return mp_stream_write(self_in, bufinfo.buf, bufinfo.len, MP_STREAM_RW_WRITE); + mp_get_buffer_raise(args[1], &bufinfo, MP_BUFFER_READ); + size_t max_len = (size_t)-1; + size_t off = 0; + if (n_args == 3) { + max_len = mp_obj_get_int_truncated(args[2]); + } else if (n_args == 4) { + off = mp_obj_get_int_truncated(args[2]); + max_len = mp_obj_get_int_truncated(args[3]); + if (off > bufinfo.len) { + off = bufinfo.len; + } + } + bufinfo.len -= off; + return mp_stream_write(args[0], (byte*)bufinfo.buf + off, MIN(bufinfo.len, max_len), MP_STREAM_RW_WRITE); } -MP_DEFINE_CONST_FUN_OBJ_2(mp_stream_write_obj, stream_write_method); +MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(mp_stream_write_obj, 2, 4, stream_write_method); STATIC mp_obj_t stream_write1_method(mp_obj_t self_in, mp_obj_t arg) { mp_buffer_info_t bufinfo; @@ -465,6 +477,17 @@ STATIC mp_obj_t stream_tell(mp_obj_t self) { } MP_DEFINE_CONST_FUN_OBJ_1(mp_stream_tell_obj, stream_tell); +STATIC mp_obj_t stream_flush(mp_obj_t self) { + const mp_stream_p_t *stream_p = mp_get_stream_raise(self, MP_STREAM_OP_IOCTL); + int error; + mp_uint_t res = stream_p->ioctl(self, MP_STREAM_FLUSH, 0, &error); + if (res == MP_STREAM_ERROR) { + nlr_raise(mp_obj_new_exception_arg1(&mp_type_OSError, MP_OBJ_NEW_SMALL_INT(error))); + } + return mp_const_none; +} +MP_DEFINE_CONST_FUN_OBJ_1(mp_stream_flush_obj, stream_flush); + STATIC mp_obj_t stream_ioctl(size_t n_args, const mp_obj_t *args) { const mp_stream_p_t *stream_p = mp_get_stream_raise(args[0], MP_STREAM_OP_IOCTL); @@ -487,3 +510,63 @@ STATIC mp_obj_t stream_ioctl(size_t n_args, const mp_obj_t *args) { return mp_obj_new_int(res); } MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(mp_stream_ioctl_obj, 2, 3, stream_ioctl); + +#if MICROPY_STREAMS_POSIX_API +/* + * POSIX-like functions + * + * These functions have POSIX-compatible signature (except for "void *stream" + * first argument instead of "int fd"). They are useful to port existing + * POSIX-compatible software to work with MicroPython streams. + */ + +// errno-like variable. If any of the functions below returned with error +// status, this variable will contain error no. +int mp_stream_errno; + +ssize_t mp_stream_posix_write(mp_obj_t stream, const void *buf, size_t len) { + mp_obj_base_t* o = (mp_obj_base_t*)MP_OBJ_TO_PTR(stream); + const mp_stream_p_t *stream_p = o->type->protocol; + mp_uint_t out_sz = stream_p->write(stream, buf, len, &mp_stream_errno); + if (out_sz == MP_STREAM_ERROR) { + return -1; + } else { + return out_sz; + } +} + +ssize_t mp_stream_posix_read(mp_obj_t stream, void *buf, size_t len) { + mp_obj_base_t* o = (mp_obj_base_t*)MP_OBJ_TO_PTR(stream); + const mp_stream_p_t *stream_p = o->type->protocol; + mp_uint_t out_sz = stream_p->read(stream, buf, len, &mp_stream_errno); + if (out_sz == MP_STREAM_ERROR) { + return -1; + } else { + return out_sz; + } +} + +off_t mp_stream_posix_lseek(mp_obj_t stream, off_t offset, int whence) { + const mp_obj_base_t* o = (mp_obj_base_t*)MP_OBJ_TO_PTR(stream); + const mp_stream_p_t *stream_p = o->type->protocol; + struct mp_stream_seek_t seek_s; + seek_s.offset = offset; + seek_s.whence = whence; + mp_uint_t res = stream_p->ioctl(stream, MP_STREAM_SEEK, (mp_uint_t)(uintptr_t)&seek_s, &mp_stream_errno); + if (res == MP_STREAM_ERROR) { + return -1; + } + return seek_s.offset; +} + +int mp_stream_posix_fsync(mp_obj_t stream) { + mp_obj_base_t* o = (mp_obj_base_t*)MP_OBJ_TO_PTR(stream); + const mp_stream_p_t *stream_p = o->type->protocol; + mp_uint_t res = stream_p->ioctl(stream, MP_STREAM_FLUSH, 0, &mp_stream_errno); + if (res == MP_STREAM_ERROR) { + return -1; + } + return res; +} + +#endif diff --git a/py/stream.h b/py/stream.h index b0f45e2f02..33d85e823c 100644 --- a/py/stream.h +++ b/py/stream.h @@ -27,6 +27,7 @@ #define __MICROPY_INCLUDED_PY_STREAM_H__ #include "py/obj.h" +#include "py/mperrno.h" #define MP_STREAM_ERROR ((mp_uint_t)-1) @@ -57,6 +58,7 @@ MP_DECLARE_CONST_FUN_OBJ(mp_stream_write_obj); MP_DECLARE_CONST_FUN_OBJ(mp_stream_write1_obj); MP_DECLARE_CONST_FUN_OBJ(mp_stream_seek_obj); MP_DECLARE_CONST_FUN_OBJ(mp_stream_tell_obj); +MP_DECLARE_CONST_FUN_OBJ(mp_stream_flush_obj); MP_DECLARE_CONST_FUN_OBJ(mp_stream_ioctl_obj); // these are for mp_get_stream_raise and can be or'd together @@ -80,13 +82,20 @@ mp_uint_t mp_stream_rw(mp_obj_t stream, void *buf, mp_uint_t size, int *errcode, #define mp_stream_write_exactly(stream, buf, size, err) mp_stream_rw(stream, (byte*)buf, size, err, MP_STREAM_RW_WRITE) #define mp_stream_read_exactly(stream, buf, size, err) mp_stream_rw(stream, buf, size, err, MP_STREAM_RW_READ) +void mp_stream_write_adaptor(void *self, const char *buf, size_t len); + +#if MICROPY_STREAMS_POSIX_API +// Functions with POSIX-compatible signatures +ssize_t mp_stream_posix_write(mp_obj_t stream, const void *buf, size_t len); +ssize_t mp_stream_posix_read(mp_obj_t stream, void *buf, size_t len); +off_t mp_stream_posix_lseek(mp_obj_t stream, off_t offset, int whence); +int mp_stream_posix_fsync(mp_obj_t stream); +#endif + #if MICROPY_STREAMS_NON_BLOCK -// TODO: This is POSIX-specific (but then POSIX is the only real thing, -// and anything else just emulates it, right?) #define mp_is_nonblocking_error(errno) ((errno) == EAGAIN || (errno) == EWOULDBLOCK) #else #define mp_is_nonblocking_error(errno) (0) #endif #endif // __MICROPY_INCLUDED_PY_STREAM_H__ -void mp_stream_write_adaptor(void *self, const char *buf, size_t len); diff --git a/qemu-arm/Makefile b/qemu-arm/Makefile index d09282a1e8..31ba6baa26 100644 --- a/qemu-arm/Makefile +++ b/qemu-arm/Makefile @@ -53,9 +53,6 @@ OBJ_TEST += $(addprefix $(BUILD)/, $(SRC_TEST_C:.c=.o)) OBJ_TEST += $(addprefix $(BUILD)/, $(SRC_S:.s=.o)) OBJ_TEST += $(BUILD)/tinytest.o -# we can't use LTO for nlrthumb.c because it contains inline assembler -$(PY_BUILD)/nlrthumb.o: CFLAGS += -fno-lto - all: run run: $(BUILD)/firmware.elf diff --git a/stmhal/boards/STM32F429DISC/mpconfigboard.h b/stmhal/boards/STM32F429DISC/mpconfigboard.h index e6c2fc9ef6..f2cf6c1422 100644 --- a/stmhal/boards/STM32F429DISC/mpconfigboard.h +++ b/stmhal/boards/STM32F429DISC/mpconfigboard.h @@ -29,8 +29,8 @@ #define MICROPY_HW_UART2_PINS (GPIO_PIN_8 | GPIO_PIN_9) // I2C busses -#define MICROPY_HW_I2C1_SCL (pin_A8) -#define MICROPY_HW_I2C1_SDA (pin_C9) +#define MICROPY_HW_I2C3_SCL (pin_A8) +#define MICROPY_HW_I2C3_SDA (pin_C9) // SPI busses //#define MICROPY_HW_SPI1_NSS (pin_A4) diff --git a/stmhal/mpconfigport.h b/stmhal/mpconfigport.h index 29e4bd8d87..c2a2cbc1f1 100644 --- a/stmhal/mpconfigport.h +++ b/stmhal/mpconfigport.h @@ -235,9 +235,9 @@ static inline mp_uint_t disable_irq(void) { // garbage-collected heap. For completeness, emulate C heap via // GC heap. Note that MicroPython core never uses malloc() and friends, // so these defines are mostly to help extension module writers. -#define malloc gc_alloc -#define free gc_free -#define realloc gc_realloc +#define malloc(n) m_malloc(n) +#define free(p) m_free(p) +#define realloc(p, n) m_realloc(p, n) // see stm32f4XX_hal_conf.h USE_USB_FS & USE_USB_HS // at the moment only USB_FS is supported diff --git a/stmhal/spi.c b/stmhal/spi.c index 387fc174c4..fbc5f9aa48 100644 --- a/stmhal/spi.c +++ b/stmhal/spi.c @@ -172,12 +172,15 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { const pyb_spi_obj_t *self; const pin_obj_t *pins[4]; + pins[0] = NULL; if (0) { #if defined(MICROPY_HW_SPI1_SCK) } else if (spi->Instance == SPI1) { self = &pyb_spi_obj[0]; + #if defined(MICROPY_HW_SPI1_NSS) pins[0] = &MICROPY_HW_SPI1_NSS; + #endif pins[1] = &MICROPY_HW_SPI1_SCK; pins[2] = &MICROPY_HW_SPI1_MISO; pins[3] = &MICROPY_HW_SPI1_MOSI; @@ -187,7 +190,9 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { #if defined(MICROPY_HW_SPI2_SCK) } else if (spi->Instance == SPI2) { self = &pyb_spi_obj[1]; + #if defined(MICROPY_HW_SPI2_NSS) pins[0] = &MICROPY_HW_SPI2_NSS; + #endif pins[1] = &MICROPY_HW_SPI2_SCK; pins[2] = &MICROPY_HW_SPI2_MISO; pins[3] = &MICROPY_HW_SPI2_MOSI; @@ -197,7 +202,9 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { #if defined(MICROPY_HW_SPI3_SCK) } else if (spi->Instance == SPI3) { self = &pyb_spi_obj[2]; + #if defined(MICROPY_HW_SPI3_NSS) pins[0] = &MICROPY_HW_SPI3_NSS; + #endif pins[1] = &MICROPY_HW_SPI3_SCK; pins[2] = &MICROPY_HW_SPI3_MISO; pins[3] = &MICROPY_HW_SPI3_MOSI; @@ -207,7 +214,9 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { #if defined(MICROPY_HW_SPI4_SCK) } else if (spi->Instance == SPI4) { self = &pyb_spi_obj[3]; + #if defined(MICROPY_HW_SPI4_NSS) pins[0] = &MICROPY_HW_SPI4_NSS; + #endif pins[1] = &MICROPY_HW_SPI4_SCK; pins[2] = &MICROPY_HW_SPI4_MISO; pins[3] = &MICROPY_HW_SPI4_MOSI; @@ -217,7 +226,9 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { #if defined(MICROPY_HW_SPI5_SCK) } else if (spi->Instance == SPI5) { self = &pyb_spi_obj[4]; + #if defined(MICROPY_HW_SPI5_NSS) pins[0] = &MICROPY_HW_SPI5_NSS; + #endif pins[1] = &MICROPY_HW_SPI5_SCK; pins[2] = &MICROPY_HW_SPI5_MISO; pins[3] = &MICROPY_HW_SPI5_MOSI; @@ -227,7 +238,9 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { #if defined(MICROPY_HW_SPI6_SCK) } else if (spi->Instance == SPI6) { self = &pyb_spi_obj[5]; + #if defined(MICROPY_HW_SPI6_NSS) pins[0] = &MICROPY_HW_SPI6_NSS; + #endif pins[1] = &MICROPY_HW_SPI6_SCK; pins[2] = &MICROPY_HW_SPI6_MISO; pins[3] = &MICROPY_HW_SPI6_MOSI; @@ -239,7 +252,7 @@ void spi_init(SPI_HandleTypeDef *spi, bool enable_nss_pin) { return; } - for (uint i = (enable_nss_pin ? 0 : 1); i < 4; i++) { + for (uint i = (enable_nss_pin && pins[0] ? 0 : 1); i < 4; i++) { mp_hal_gpio_set_af(pins[i], &GPIO_InitStructure, AF_FN_SPI, (self - &pyb_spi_obj[0]) + 1); } diff --git a/teensy/mpconfigport.h b/teensy/mpconfigport.h index 867c4e6527..05c040d415 100644 --- a/teensy/mpconfigport.h +++ b/teensy/mpconfigport.h @@ -100,14 +100,6 @@ __attribute__(( always_inline )) static inline mp_uint_t disable_irq(void) { #define MICROPY_BEGIN_ATOMIC_SECTION() disable_irq() #define MICROPY_END_ATOMIC_SECTION(state) enable_irq(state) -// There is no classical C heap in bare-metal ports, only Python -// garbage-collected heap. For completeness, emulate C heap via -// GC heap. Note that MicroPython core never uses malloc() and friends, -// so these defines are mostly to help extension module writers. -#define malloc gc_alloc -#define free gc_free -#define realloc gc_realloc - // We need to provide a declaration/definition of alloca() #include <alloca.h> diff --git a/tests/basics/bytes_compare2.py b/tests/basics/bytes_compare2.py index 02516de93a..8959da3ae7 100644 --- a/tests/basics/bytes_compare2.py +++ b/tests/basics/bytes_compare2.py @@ -1,6 +1,5 @@ print(b"1" == 1) print(b"123" == bytearray(b"123")) -print(b"123" == "123") print(b'123' < bytearray(b"124")) print(b'123' > bytearray(b"122")) print(bytearray(b"23") in b"1234") diff --git a/tests/basics/bytes_compare3.py b/tests/basics/bytes_compare3.py new file mode 100644 index 0000000000..4b9cbd5b3a --- /dev/null +++ b/tests/basics/bytes_compare3.py @@ -0,0 +1,9 @@ +# Based on MicroPython config option, comparison of str and bytes +# or vice versa may issue a runtime warning. On CPython, if run as +# "python3 -b", only comparison of str to bytes issues a warning, +# not the other way around (while exactly comparison of bytes to +# str would be the most common error, as in sock.recv(3) == "GET"). +# Update: the issue above with CPython apparently happens in REPL, +# when run as a script, both lines issue a warning. +print("123" == b"123") +print(b"123" == "123") diff --git a/tests/basics/bytes_compare3.py.exp b/tests/basics/bytes_compare3.py.exp new file mode 100644 index 0000000000..7b117b9056 --- /dev/null +++ b/tests/basics/bytes_compare3.py.exp @@ -0,0 +1,4 @@ +######## +False +######## +False diff --git a/tests/basics/bytes_find.py b/tests/basics/bytes_find.py index 434669a901..75ef9796cd 100644 --- a/tests/basics/bytes_find.py +++ b/tests/basics/bytes_find.py @@ -21,3 +21,6 @@ print(b"0000".find(b'-1', 3)) print(b"0000".find(b'1', 3)) print(b"0000".find(b'1', 4)) print(b"0000".find(b'1', 5)) + +# Non-ascii values (make sure not treated as unicode-like) +print(b"\x80abc".find(b"a", 1)) diff --git a/tests/basics/bytes_partition.py b/tests/basics/bytes_partition.py index 3868a81a55..7d3ffaaaaa 100644 --- a/tests/basics/bytes_partition.py +++ b/tests/basics/bytes_partition.py @@ -1,3 +1,10 @@ +try: + str.partition +except AttributeError: + print("SKIP") + import sys + sys.exit() + print(b"asdf".partition(b'g')) print(b"asdf".partition(b'a')) print(b"asdf".partition(b's')) diff --git a/tests/basics/string_partition.py b/tests/basics/string_partition.py index fe0070a658..b3b2f0907d 100644 --- a/tests/basics/string_partition.py +++ b/tests/basics/string_partition.py @@ -1,3 +1,10 @@ +try: + str.partition +except AttributeError: + print("SKIP") + import sys + sys.exit() + print("asdf".partition('g')) print("asdf".partition('a')) print("asdf".partition('s')) diff --git a/tests/basics/string_rpartition.py b/tests/basics/string_rpartition.py index 656121c94d..84e0031fb0 100644 --- a/tests/basics/string_rpartition.py +++ b/tests/basics/string_rpartition.py @@ -1,3 +1,10 @@ +try: + str.partition +except AttributeError: + print("SKIP") + import sys + sys.exit() + print("asdf".rpartition('g')) print("asdf".rpartition('a')) print("asdf".rpartition('s')) diff --git a/tests/extmod/btree1.py b/tests/extmod/btree1.py index 11acd7c98f..c96cce92d6 100644 --- a/tests/extmod/btree1.py +++ b/tests/extmod/btree1.py @@ -1,11 +1,15 @@ try: import btree + import uio except ImportError: print("SKIP") import sys sys.exit() -db = btree.open(None) +#f = open("_test.db", "w+b") +f = uio.BytesIO() +db = btree.open(f) + db[b"foo3"] = b"bar3" db[b"foo1"] = b"bar1" db[b"foo2"] = b"bar2" @@ -57,3 +61,9 @@ print(list(db.values())) for k in db: print(k) + +print("foo1", "foo1" in db) +print("foo2", "foo2" in db) + +db.close() +f.close() diff --git a/tests/extmod/btree1.py.exp b/tests/extmod/btree1.py.exp index a266d7acfc..2983a09874 100644 --- a/tests/extmod/btree1.py.exp +++ b/tests/extmod/btree1.py.exp @@ -30,3 +30,5 @@ KeyError b'bar1' b'foo1' b'foo3' +foo1 True +foo2 False diff --git a/tests/extmod/machine1.py b/tests/extmod/machine1.py index af4dacd03a..433a180376 100644 --- a/tests/extmod/machine1.py +++ b/tests/extmod/machine1.py @@ -1,7 +1,10 @@ # test machine module try: - import machine + try: + import umachine as machine + except ImportError: + import machine except ImportError: print("SKIP") import sys diff --git a/tests/extmod/machine_mem.py b/tests/extmod/machine_mem.py deleted file mode 100644 index 7d8a9ac01e..0000000000 --- a/tests/extmod/machine_mem.py +++ /dev/null @@ -1,16 +0,0 @@ -# This test requires root privilege, so is usually skipped -# It also assumes x86 legacy hardware (with Video BIOS present). - -try: - import machine -except ImportError: - print("SKIP") - import sys - sys.exit() - -try: - print(hex(machine.mem16[0xc0000])) -except OSError: - print("SKIP") - import sys - sys.exit() diff --git a/tests/extmod/machine_mem.py.exp b/tests/extmod/machine_mem.py.exp deleted file mode 100644 index 371f8fa44d..0000000000 --- a/tests/extmod/machine_mem.py.exp +++ /dev/null @@ -1 +0,0 @@ -0xaa55 diff --git a/tests/extmod/machine_pinbase.py b/tests/extmod/machine_pinbase.py index 07a489a596..5e82823ec0 100644 --- a/tests/extmod/machine_pinbase.py +++ b/tests/extmod/machine_pinbase.py @@ -1,10 +1,16 @@ try: - from umachine import PinBase + import umachine as machine except ImportError: - from machine import PinBase + import machine +try: + machine.PinBase +except AttributeError: + print("SKIP") + import sys + sys.exit() -class MyPin(PinBase): +class MyPin(machine.PinBase): def __init__(self): print("__init__") diff --git a/tests/io/bytesio_ext.py b/tests/io/bytesio_ext.py new file mode 100644 index 0000000000..30d79fcfcd --- /dev/null +++ b/tests/io/bytesio_ext.py @@ -0,0 +1,19 @@ +# Extended stream operations on io.BytesIO +try: + import uio as io +except ImportError: + import io + +a = io.BytesIO() +print(a.seek(8)) +a.write(b"123") +print(a.getvalue()) + +print(a.seek(0, 1)) + +print(a.seek(-1, 2)) +a.write(b"0") +print(a.getvalue()) + +a.flush() +print(a.getvalue()) diff --git a/tests/io/write_ext.py b/tests/io/write_ext.py new file mode 100644 index 0000000000..f4ed0d3685 --- /dev/null +++ b/tests/io/write_ext.py @@ -0,0 +1,25 @@ +import uio + +try: + uio.BytesIO +except AttributeError: + import sys + print('SKIP') + sys.exit() + +buf = uio.BytesIO() + +buf.write(b"foo", 2) +print(buf.getvalue()) + +buf.write(b"foo", 100) +print(buf.getvalue()) + +buf.write(b"foobar", 1, 3) +print(buf.getvalue()) + +buf.write(b"foobar", 1, 100) +print(buf.getvalue()) + +buf.write(b"foobar", 100, 100) +print(buf.getvalue()) diff --git a/tests/io/write_ext.py.exp b/tests/io/write_ext.py.exp new file mode 100644 index 0000000000..0f9c6bfbc1 --- /dev/null +++ b/tests/io/write_ext.py.exp @@ -0,0 +1,5 @@ +b'fo' +b'fofoo' +b'fofoooob' +b'fofooooboobar' +b'fofooooboobar' diff --git a/tests/run-tests b/tests/run-tests index 02791896b5..32d334a008 100755 --- a/tests/run-tests +++ b/tests/run-tests @@ -26,11 +26,35 @@ def rm_f(fname): if os.path.exists(fname): os.remove(fname) + +# unescape wanted regex chars and escape unwanted ones +def convert_regex_escapes(line): + cs = [] + escape = False + for c in str(line, 'utf8'): + if escape: + escape = False + cs.append(c) + elif c == '\\': + escape = True + elif c in ('(', ')', '[', ']', '{', '}', '.', '*', '+', '^', '$'): + cs.append('\\' + c) + else: + cs.append(c) + # accept carriage-return(s) before final newline + if cs[-1] == '\n': + cs[-1] = '\r*\n' + return bytes(''.join(cs), 'utf8') + + def run_micropython(pyb, args, test_file): + special_tests = ('micropython/meminfo.py', 'basics/bytes_compare3.py') + is_special = False if pyb is None: # run on PC - if test_file.startswith(('cmdline/', 'feature_check/')) or test_file == 'micropython/meminfo.py': + if test_file.startswith(('cmdline/', 'feature_check/')) or test_file in special_tests: # special handling for tests of the unix cmdline program + is_special = True # check for any cmdline options needed for this test args = [MICROPYTHON] @@ -80,63 +104,6 @@ def run_micropython(pyb, args, test_file): except subprocess.CalledProcessError: return b'CRASH' - # unescape wanted regex chars and escape unwanted ones - def convert_regex_escapes(line): - cs = [] - escape = False - for c in str(line, 'utf8'): - if escape: - escape = False - cs.append(c) - elif c == '\\': - escape = True - elif c in ('(', ')', '[', ']', '{', '}', '.', '*', '+', '^', '$'): - cs.append('\\' + c) - else: - cs.append(c) - # accept carriage-return(s) before final newline - if cs[-1] == '\n': - cs[-1] = '\r*\n' - return bytes(''.join(cs), 'utf8') - - # convert parts of the output that are not stable across runs - with open(test_file + '.exp', 'rb') as f: - lines_exp = [] - for line in f.readlines(): - if line == b'########\n': - line = (line,) - else: - line = (line, re.compile(convert_regex_escapes(line))) - lines_exp.append(line) - lines_mupy = [line + b'\n' for line in output_mupy.split(b'\n')] - if output_mupy.endswith(b'\n'): - lines_mupy = lines_mupy[:-1] # remove erroneous last empty line - i_mupy = 0 - for i in range(len(lines_exp)): - if lines_exp[i][0] == b'########\n': - # 8x #'s means match 0 or more whole lines - line_exp = lines_exp[i + 1] - skip = 0 - while i_mupy + skip < len(lines_mupy) and not line_exp[1].match(lines_mupy[i_mupy + skip]): - skip += 1 - if i_mupy + skip >= len(lines_mupy): - lines_mupy[i_mupy] = b'######## FAIL\n' - break - del lines_mupy[i_mupy:i_mupy + skip] - lines_mupy.insert(i_mupy, b'########\n') - i_mupy += 1 - else: - # a regex - if lines_exp[i][1].match(lines_mupy[i_mupy]): - lines_mupy[i_mupy] = lines_exp[i][0] - else: - #print("don't match: %r %s" % (lines_exp[i][1], lines_mupy[i_mupy])) # DEBUG - pass - i_mupy += 1 - if i_mupy >= len(lines_mupy): - break - output_mupy = b''.join(lines_mupy) - else: # a standard test try: @@ -159,6 +126,45 @@ def run_micropython(pyb, args, test_file): # canonical form for all ports/platforms is to use \n for end-of-line output_mupy = output_mupy.replace(b'\r\n', b'\n') + if is_special or test_file in special_tests: + # convert parts of the output that are not stable across runs + with open(test_file + '.exp', 'rb') as f: + lines_exp = [] + for line in f.readlines(): + if line == b'########\n': + line = (line,) + else: + line = (line, re.compile(convert_regex_escapes(line))) + lines_exp.append(line) + lines_mupy = [line + b'\n' for line in output_mupy.split(b'\n')] + if output_mupy.endswith(b'\n'): + lines_mupy = lines_mupy[:-1] # remove erroneous last empty line + i_mupy = 0 + for i in range(len(lines_exp)): + if lines_exp[i][0] == b'########\n': + # 8x #'s means match 0 or more whole lines + line_exp = lines_exp[i + 1] + skip = 0 + while i_mupy + skip < len(lines_mupy) and not line_exp[1].match(lines_mupy[i_mupy + skip]): + skip += 1 + if i_mupy + skip >= len(lines_mupy): + lines_mupy[i_mupy] = b'######## FAIL\n' + break + del lines_mupy[i_mupy:i_mupy + skip] + lines_mupy.insert(i_mupy, b'########\n') + i_mupy += 1 + else: + # a regex + if lines_exp[i][1].match(lines_mupy[i_mupy]): + lines_mupy[i_mupy] = lines_exp[i][0] + else: + #print("don't match: %r %s" % (lines_exp[i][1], lines_mupy[i_mupy])) # DEBUG + pass + i_mupy += 1 + if i_mupy >= len(lines_mupy): + break + output_mupy = b''.join(lines_mupy) + return output_mupy def run_tests(pyb, tests, args): diff --git a/tests/unicode/unicode_subscr.py b/tests/unicode/unicode_subscr.py new file mode 100644 index 0000000000..a2f434de58 --- /dev/null +++ b/tests/unicode/unicode_subscr.py @@ -0,0 +1,23 @@ +a = '¢пр' + +print(a[0], a[0:1]) +print(a[1], a[1:2]) +print(a[2], a[2:3]) +try: + print(a[3]) +except IndexError: + print("IndexError") +print(a[3:4]) + +print(a[-1]) +print(a[-2], a[-2:-1]) +print(a[-3], a[-3:-2]) +try: + print(a[-4]) +except IndexError: + print("IndexError") +print(a[-4:-3]) + +print(a[0:2]) +print(a[1:3]) +print(a[2:4]) diff --git a/unix/Makefile b/unix/Makefile index 2e0cbfd35a..cb9a999f08 100644 --- a/unix/Makefile +++ b/unix/Makefile @@ -263,6 +263,8 @@ freedos: MICROPY_PY_JNI=0 # build an interpreter for coverage testing and do the testing +# btree module is disabled, because with extra warnings enabled, +# building berkeleydb errors out coverage: $(MAKE) COPT="-O0" MICROPY_PY_BTREE=0 CFLAGS_EXTRA='-DMP_CONFIGFILE="<mpconfigport_coverage.h>" -fprofile-arcs -ftest-coverage -Wdouble-promotion -Wformat -Wmissing-declarations -Wmissing-prototypes -Wold-style-definition -Wpointer-arith -Wshadow -Wsign-compare -Wuninitialized -Wunused-parameter -DMICROPY_UNIX_COVERAGE' LDFLAGS_EXTRA='-fprofile-arcs -ftest-coverage' BUILD=build-coverage PROG=micropython_coverage @@ -308,11 +310,14 @@ libffi: ../configure $(CROSS_COMPILE_HOST) --prefix=$$PWD/out --disable-structs CC="$(CC)" CXX="$(CXX)" LD="$(LD)" CFLAGS="-Os -fomit-frame-pointer -fstrict-aliasing -ffast-math -fno-exceptions"; \ make install-exec-recursive; make -C include install-data-am -axtls: ../lib/axtls/README +axtls: $(BUILD)/libaxtls.a + +$(BUILD)/libaxtls.a: ../lib/axtls/README | $(OBJ_DIRS) cd ../lib/axtls; cp config/upyconfig config/.config cd ../lib/axtls; make oldconfig -B cd ../lib/axtls; make clean cd ../lib/axtls; make all CC="$(CC)" LD="$(LD)" + cp ../lib/axtls/_stage/libaxtls.a $@ ../lib/axtls/README: @echo "You cloned without --recursive, fetching submodules for you." diff --git a/unix/fdfile.h b/unix/fdfile.h new file mode 100644 index 0000000000..8e8e97c795 --- /dev/null +++ b/unix/fdfile.h @@ -0,0 +1,41 @@ +/* + * This file is part of the Micro Python project, http://micropython.org/ + * + * The MIT License (MIT) + * + * Copyright (c) 2013, 2014 Damien P. George + * Copyright (c) 2016 Paul Sokolovsky + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "py/obj.h" + +#ifndef __MICROPY_INCLUDED_UNIX_FILE_H__ +#define __MICROPY_INCLUDED_UNIX_FILE_H__ + +typedef struct _mp_obj_fdfile_t { + mp_obj_base_t base; + int fd; +} mp_obj_fdfile_t; + +extern const mp_obj_type_t mp_type_fileio; +extern const mp_obj_type_t mp_type_textio; + +#endif // __MICROPY_INCLUDED_UNIX_FILE_H__ diff --git a/unix/file.c b/unix/file.c index a7620e079e..203a5a3abe 100644 --- a/unix/file.c +++ b/unix/file.c @@ -36,6 +36,7 @@ #include "py/stream.h" #include "py/builtin.h" #include "py/mphal.h" +#include "fdfile.h" #if MICROPY_PY_IO @@ -43,11 +44,6 @@ #define fsync _commit #endif -typedef struct _mp_obj_fdfile_t { - mp_obj_base_t base; - int fd; -} mp_obj_fdfile_t; - #ifdef MICROPY_CPYTHON_COMPAT STATIC void check_fd_is_open(const mp_obj_fdfile_t *o) { if (o->fd < 0) { @@ -105,29 +101,30 @@ STATIC mp_uint_t fdfile_write(mp_obj_t o_in, const void *buf, mp_uint_t size, in STATIC mp_uint_t fdfile_ioctl(mp_obj_t o_in, mp_uint_t request, uintptr_t arg, int *errcode) { mp_obj_fdfile_t *o = MP_OBJ_TO_PTR(o_in); - if (request == MP_STREAM_SEEK) { - struct mp_stream_seek_t *s = (struct mp_stream_seek_t*)arg; - off_t off = lseek(o->fd, s->offset, s->whence); - if (off == (off_t)-1) { - *errcode = errno; - return MP_STREAM_ERROR; + check_fd_is_open(o); + switch (request) { + case MP_STREAM_SEEK: { + struct mp_stream_seek_t *s = (struct mp_stream_seek_t*)arg; + off_t off = lseek(o->fd, s->offset, s->whence); + if (off == (off_t)-1) { + *errcode = errno; + return MP_STREAM_ERROR; + } + s->offset = off; + return 0; } - s->offset = off; - return 0; - } else { - *errcode = EINVAL; - return MP_STREAM_ERROR; + case MP_STREAM_FLUSH: + if (fsync(o->fd) < 0) { + *errcode = errno; + return MP_STREAM_ERROR; + } + return 0; + default: + *errcode = EINVAL; + return MP_STREAM_ERROR; } } -STATIC mp_obj_t fdfile_flush(mp_obj_t self_in) { - mp_obj_fdfile_t *self = MP_OBJ_TO_PTR(self_in); - check_fd_is_open(self); - fsync(self->fd); - return mp_const_none; -} -STATIC MP_DEFINE_CONST_FUN_OBJ_1(fdfile_flush_obj, fdfile_flush); - STATIC mp_obj_t fdfile_close(mp_obj_t self_in) { mp_obj_fdfile_t *self = MP_OBJ_TO_PTR(self_in); close(self->fd); @@ -228,7 +225,7 @@ STATIC const mp_rom_map_elem_t rawfile_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_write), MP_ROM_PTR(&mp_stream_write_obj) }, { MP_ROM_QSTR(MP_QSTR_seek), MP_ROM_PTR(&mp_stream_seek_obj) }, { MP_ROM_QSTR(MP_QSTR_tell), MP_ROM_PTR(&mp_stream_tell_obj) }, - { MP_ROM_QSTR(MP_QSTR_flush), MP_ROM_PTR(&fdfile_flush_obj) }, + { MP_ROM_QSTR(MP_QSTR_flush), MP_ROM_PTR(&mp_stream_flush_obj) }, { MP_ROM_QSTR(MP_QSTR_close), MP_ROM_PTR(&fdfile_close_obj) }, { MP_ROM_QSTR(MP_QSTR___enter__), MP_ROM_PTR(&mp_identity_obj) }, { MP_ROM_QSTR(MP_QSTR___exit__), MP_ROM_PTR(&fdfile___exit___obj) }, diff --git a/unix/modsocket.c b/unix/modsocket.c index cd68b20a45..56bab3494a 100644 --- a/unix/modsocket.c +++ b/unix/modsocket.c @@ -62,12 +62,14 @@ #define MICROPY_SOCKET_EXTRA (0) +// This type must "inherit" from mp_obj_fdfile_t, i.e. matching subset of +// fields should have the same layout. typedef struct _mp_obj_socket_t { mp_obj_base_t base; int fd; } mp_obj_socket_t; -STATIC const mp_obj_type_t usocket_type; +const mp_obj_type_t mp_type_socket; // Helper functions #define RAISE_ERRNO(err_flag, error_val) \ @@ -80,7 +82,7 @@ static inline mp_obj_t mp_obj_from_sockaddr(const struct sockaddr *addr, socklen STATIC mp_obj_socket_t *socket_new(int fd) { mp_obj_socket_t *o = m_new_obj(mp_obj_socket_t); - o->base.type = &usocket_type; + o->base.type = &mp_type_socket; o->fd = fd; return o; } @@ -374,7 +376,7 @@ STATIC const mp_stream_p_t usocket_stream_p = { .write = socket_write, }; -STATIC const mp_obj_type_t usocket_type = { +const mp_obj_type_t mp_type_socket = { { &mp_type_type }, .name = MP_QSTR_socket, .print = socket_print, @@ -550,7 +552,7 @@ STATIC MP_DEFINE_CONST_FUN_OBJ_1(mod_socket_sockaddr_obj, mod_socket_sockaddr); STATIC const mp_rom_map_elem_t mp_module_socket_globals_table[] = { { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_usocket) }, - { MP_ROM_QSTR(MP_QSTR_socket), MP_ROM_PTR(&usocket_type) }, + { MP_ROM_QSTR(MP_QSTR_socket), MP_ROM_PTR(&mp_type_socket) }, { MP_ROM_QSTR(MP_QSTR_getaddrinfo), MP_ROM_PTR(&mod_socket_getaddrinfo_obj) }, { MP_ROM_QSTR(MP_QSTR_inet_pton), MP_ROM_PTR(&mod_socket_inet_pton_obj) }, { MP_ROM_QSTR(MP_QSTR_inet_ntop), MP_ROM_PTR(&mod_socket_inet_ntop_obj) }, diff --git a/unix/moduselect.c b/unix/moduselect.c index 13cb3f1fa8..38f8d11ed8 100644 --- a/unix/moduselect.c +++ b/unix/moduselect.c @@ -38,6 +38,9 @@ #include "py/objlist.h" #include "py/objtuple.h" #include "py/mphal.h" +#include "fdfile.h" + +extern const mp_obj_type_t mp_type_socket; // Flags for poll() #define FLAG_ONESHOT (1) @@ -51,10 +54,23 @@ typedef struct _mp_obj_poll_t { struct pollfd *entries; } mp_obj_poll_t; +STATIC int get_fd(mp_obj_t fdlike) { + int fd; + // Shortcut for fdfile compatible types + if (MP_OBJ_IS_TYPE(fdlike, &mp_type_fileio) || MP_OBJ_IS_TYPE(fdlike, &mp_type_socket)) { + mp_obj_fdfile_t *fdfile = MP_OBJ_TO_PTR(fdlike); + fd = fdfile->fd; + } else { + fd = mp_obj_get_int(fdlike); + } + return fd; +} + /// \method register(obj[, eventmask]) STATIC mp_obj_t poll_register(size_t n_args, const mp_obj_t *args) { mp_obj_poll_t *self = MP_OBJ_TO_PTR(args[0]); - int fd = mp_obj_get_int(args[1]); + int fd = get_fd(args[1]); + mp_uint_t flags; if (n_args == 3) { flags = mp_obj_get_int(args[2]); @@ -95,7 +111,7 @@ MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(poll_register_obj, 2, 3, poll_register); STATIC mp_obj_t poll_unregister(mp_obj_t self_in, mp_obj_t obj_in) { mp_obj_poll_t *self = MP_OBJ_TO_PTR(self_in); struct pollfd *entries = self->entries; - int fd = mp_obj_get_int(obj_in); + int fd = get_fd(obj_in); for (int i = self->len - 1; i >= 0; i--) { if (entries->fd == fd) { entries->fd = -1; @@ -113,7 +129,7 @@ MP_DEFINE_CONST_FUN_OBJ_2(poll_unregister_obj, poll_unregister); STATIC mp_obj_t poll_modify(mp_obj_t self_in, mp_obj_t obj_in, mp_obj_t eventmask_in) { mp_obj_poll_t *self = MP_OBJ_TO_PTR(self_in); struct pollfd *entries = self->entries; - int fd = mp_obj_get_int(obj_in); + int fd = get_fd(obj_in); for (int i = self->len - 1; i >= 0; i--) { if (entries->fd == fd) { entries->events = mp_obj_get_int(eventmask_in); diff --git a/unix/mpconfigport.h b/unix/mpconfigport.h index 06c4edc1e2..c449e7316f 100644 --- a/unix/mpconfigport.h +++ b/unix/mpconfigport.h @@ -63,6 +63,7 @@ #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE) #define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_MPZ) #define MICROPY_STREAMS_NON_BLOCK (1) +#define MICROPY_STREAMS_POSIX_API (1) #define MICROPY_OPT_COMPUTED_GOTO (1) #ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE #define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (1) @@ -72,6 +73,7 @@ #define MICROPY_PY_DESCRIPTORS (1) #define MICROPY_PY_BUILTINS_STR_UNICODE (1) #define MICROPY_PY_BUILTINS_STR_CENTER (1) +#define MICROPY_PY_BUILTINS_STR_PARTITION (1) #define MICROPY_PY_BUILTINS_STR_SPLITLINES (1) #define MICROPY_PY_BUILTINS_MEMORYVIEW (1) #define MICROPY_PY_BUILTINS_FROZENSET (1) @@ -111,7 +113,7 @@ #define MICROPY_PY_URE (1) #define MICROPY_PY_UHEAPQ (1) #define MICROPY_PY_UHASHLIB (1) -#if MICROPY_PY_USSL +#if MICROPY_PY_USSL && MICROPY_SSL_AXTLS #define MICROPY_PY_UHASHLIB_SHA1 (1) #endif #define MICROPY_PY_UBINASCII (1) @@ -119,6 +121,7 @@ #ifndef MICROPY_PY_USELECT #define MICROPY_PY_USELECT (1) #endif +#define MICROPY_PY_WEBSOCKET (1) #define MICROPY_PY_MACHINE (1) #define MICROPY_PY_MACHINE_PULSE (1) #define MICROPY_MACHINE_MEM_GET_READ_ADDR mod_machine_mem_get_addr @@ -138,6 +141,7 @@ // names in exception messages (may require more RAM). #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_DETAILED) #define MICROPY_WARNINGS (1) +#define MICROPY_PY_STR_BYTES_CMP_WARN (1) // Define to 1 to use undertested inefficient GC helper implementation // (if more efficient arch-specific one is not available). @@ -300,3 +304,6 @@ void mp_unix_mark_exec(void); #ifndef _DIRENT_HAVE_D_INO #define _DIRENT_HAVE_D_INO (1) #endif + +// For debugging purposes, make printf() available to any source file. +#include <stdio.h> diff --git a/unix/mpconfigport.mk b/unix/mpconfigport.mk index 9f826661a9..f7912141af 100644 --- a/unix/mpconfigport.mk +++ b/unix/mpconfigport.mk @@ -28,6 +28,7 @@ MICROPY_PY_FFI = 1 # ussl module requires axtls MICROPY_PY_USSL = 1 +MICROPY_SSL_AXTLS = 1 # jni module requires JVM/JNI MICROPY_PY_JNI = 0 diff --git a/unix/mpconfigport_minimal.h b/unix/mpconfigport_minimal.h index 2ab95c67af..054e82c243 100644 --- a/unix/mpconfigport_minimal.h +++ b/unix/mpconfigport_minimal.h @@ -34,6 +34,7 @@ #define MICROPY_ALLOC_PARSE_CHUNK_INIT (64) #define MICROPY_ALLOC_PATH_MAX (PATH_MAX) #define MICROPY_ENABLE_GC (1) +#define MICROPY_GC_ALLOC_THRESHOLD (0) #define MICROPY_ENABLE_FINALISER (0) #define MICROPY_STACK_CHECK (0) #define MICROPY_COMP_CONST (0) diff --git a/unix/mpthreadport.c b/unix/mpthreadport.c index e5cfe7a669..663d3a5dee 100644 --- a/unix/mpthreadport.c +++ b/unix/mpthreadport.c @@ -134,11 +134,14 @@ void mp_thread_start(void) { } void mp_thread_create(void *(*entry)(void*), void *arg, size_t *stack_size) { - // default stack size is 8k machine-words, minimum is 2k + // default stack size is 8k machine-words if (*stack_size == 0) { *stack_size = 8192 * BYTES_PER_WORD; - } else if (*stack_size < 2048 * BYTES_PER_WORD) { - *stack_size = 2048 * BYTES_PER_WORD; + } + + // minimum stack size is set by pthreads + if (*stack_size < PTHREAD_STACK_MIN) { + *stack_size = PTHREAD_STACK_MIN; } // set thread attributes @@ -163,7 +166,8 @@ void mp_thread_create(void *(*entry)(void*), void *arg, size_t *stack_size) { } // adjust stack_size to provide room to recover from hitting the limit - *stack_size -= 1024 * BYTES_PER_WORD; + // this value seems to be about right for both 32-bit and 64-bit builds + *stack_size -= 8192; // add thread to linked list of all threads thread_t *th = malloc(sizeof(thread_t)); diff --git a/windows/README b/windows/README index ca26c46293..4b383eaed3 100644 --- a/windows/README +++ b/windows/README @@ -52,5 +52,5 @@ backend which has the look&feel of a standard Unix console: For more info, see https://www.winehq.org/docs/wineusr-guide/cui-programs If built without line editing and history capabilities -(MICROPY_USE_READLINE=0), the resulting binary can be built using standard +(MICROPY_USE_READLINE=0), the resulting binary can be run using the standard "wine" tool. diff --git a/windows/fmode.c b/windows/fmode.c new file mode 100644 index 0000000000..33ba24ed1f --- /dev/null +++ b/windows/fmode.c @@ -0,0 +1,49 @@ +/* + * This file is part of the MicroPython project, http://micropython.org/ + * + * The MIT License (MIT) + * + * Copyright (c) 2013-2016 Damien P. George + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "fmode.h" +#include "py/mpconfig.h" +#include <fcntl.h> +#include <stdlib.h> + +// Workaround for setting file translation mode: we must distinguish toolsets +// since mingw has no _set_fmode, and altering msvc's _fmode directly has no effect +STATIC int set_fmode_impl(int mode) { +#ifndef _MSC_VER + _fmode = mode; + return 0; +#else + return _set_fmode(mode); +#endif +} + +void set_fmode_binary(void) { + set_fmode_impl(O_BINARY); +} + +void set_fmode_text(void) { + set_fmode_impl(O_TEXT); +} diff --git a/windows/fmode.h b/windows/fmode.h new file mode 100644 index 0000000000..23d6d3d54d --- /dev/null +++ b/windows/fmode.h @@ -0,0 +1,38 @@ +/* + * This file is part of the MicroPython project, http://micropython.org/ + * + * The MIT License (MIT) + * + * Copyright (c) 2013-2016 Damien P. George + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef __MICROPY_INCLUDED_WINDOWS_FMODE_H__ +#define __MICROPY_INCLUDED_WINDOWS_FMODE_H__ + +// Treat files opened by open() as binary. No line ending translation is done. +void set_fmode_binary(void); + +// Treat files opened by open() as text. +// When reading from the file \r\n will be converted to \n. +// When writing to the file \n will be converted into \r\n. +void set_fmode_text(void); + +#endif |
