This repository provides a general framework to deploy smarthome nodes based on micropython and communicating using mqtt. The framework is easy to use and extend with custom components/scripts.
Many SmartHome frameworks are based on C(++) which makes extending them more difficult. In micropython environment there are a lot less frameworks and often have a difficult documentation or None and are either complex or do not have enough features. So I decided to build one myself with the following points in mind:
The project is tested on ESP32 and ESP8266 microcontrollers. It also supports the unix port.
The ESP8266 works reliable and resilient and is useable 24/7 as it recovers from any crashes/freezes using a software watchdog. Before micropython 1.11 there was a bug that could freeze uasyncio for 1h 11min that made the software watchdog neccesary.
The device doesn't have much RAM but it's enough for multiple bigger components and many small ones.
Official ESP32 port is supported since version 4.1.0 and loboris fork is officially unsupported since there hasn't been any updates and it doesn't support uasyncio v3.0, therefore all code for it got removed too.
Since version 5.0.0 the unix port is supported. There are no calls to machine or network in the base code of the framework. The usable components are limited at the moment as no GPIO support is available. However system calls work which gives a lot of options, e.g. to use the raspberry-remote library to control RF433Mhz devices. Basic GPIO support is planned to be implemented but will still be slow and not a C-module.
To clone this repository run
git clone https://github.com/kevinkk525/pysmartnode cd pysmartnode git submodule update --init --recursive --remote
To update run inside the repository directory
git pull git submodule update --init --recursive --remote
You should have the latest micropython firmware and include the directory "pysmartnode" as frozen bytecode into your firmware. (Put it in the "module" directory before building the firmware) On ESP32 frozen bytecode is not neccessary but should be considered if not using psram.
Alternatively a prebuilt firmware for the esp8266 can be used that should be attached to every release on github. On esp32 the precompiled .mpy files can be used that are also attached to every github release.
The esp32 could build the project but that is very slow and therefore it is recommended to use .mpy files.
Required external modules are:
All required modules are in this repository and don't need to be aquired manually.
Just put the directory
external_modules into your
modules directory before building the firmware or run the correct script in section
The micropython_mqtt_as directory is a submodule to my repository so this will be updated automatically.
The included components can be found in the directory "pysmartnode/components".
Their description and configuration will soon be added to the wiki but for now you can check the files itself.
They contain an example configuration of the component and some description. The example configuration however is in hjson, which is easier than json and has the possibility to have comments in it. This format is used in the SmartServer (see 4.1.), if you want to put the configuration onto your controller, you have to convert it to json.
Every loaded component will be published as log.info to "/log/info/<device_id>" with the version of the module.
Components can be and do anything. There is a basic API and base class which helps with homeassistant mqtt discovery and the basic API.
Sensors all have a similar API to have a standardized usage through the sensor base class. The sensor base class makes developing sensors very easy as it takes care of mqtt discovery, reading and publishing intervals and the standardized API. All sensors have a common API:
Sensor_types are in definitions but can be custom, those are only the ones supported by Homeassistant.
This should make working with different types of sensors easier. If you are e.g. building a heating controller and need a temperature from some sensor, you can just connect any sensor and provide the heating code with that sensor by configuration.
Switch components can be integrated similarily easy using the switch base class.
Templates for how to use the components can be found in templates.
Home-Assistant has the feature to automatically discover mqtt components if they provide the needed configuration using mqtt. Since version 5.0.0 all components can now make use of the MQTT-Discovery.
To make building new components easier, there is a template. It should help understanding how the configuration can be read and mqtt subscriptions and publishing values works.
Besides the general components, components can mostly be divided in sensors and switches, which have a slightly different API. There are templates for both types of components, sensors and switches. There is also a Pushbutton template which is just a simpler version of the switch.
A more detailed documentation of the basic components can be found in COMPONENTS.md
To run the pysmartnode on boot, just use the included "main.py".
The environment specific configuration is done within the "config.py" file that has to be put in the root directory of your microcontroller (or if frozen in "modules"). Copy the "config_example.py" to "config.py" and change it according to your needs. In this file you have to specify the WIFI and MQTT parameters of your home environment. There are also some optional parameters to configure base modules and behaviour.
The project configuration is done in the file config.py which should be created by copying the config_example.py as config.py. If you have a filesystem, copy it onto the device or put it as frozen bytecode in your modules directory.
In config.py only those configurations have to be provided, that overwrite the default values found in config_base.
The basic configuration options are:
Optional configurations for the network are:
Platform dependent options are
A few additional options define some constants:
The configuration of all components used on a microcontroller can be configured in two different ways:
To configure the used components it is possible to run SmartServer in your network, which holds the configuration of each microcontroller and publishes it over MQTT when requested. This makes it very easy to change configurations but makes the SmartHome less secure as the configuration is not fixed on the device itself. The received configuration is stored locally to be able to recover from power outages even if the SmartServer is offline. For storing offline the method 126.96.36.199. is used on the ESP32 and 188.8.131.52. is used on the ESP8266 as these are the preffered methods. Another benefit of using the SmartServer is that it collects the log messages that this framework publishes over MQTT.
Local configuration is only used if either no configuration could be received from the SmartServer or receiving the configuration is disabled.
This configuration can be used if you plan on freezing the component configuration or do not have a filesystem activated to save some RAM.
If the components.py module has a COMPONENTS dictionary, this will be used just like the received dictionary from SmartServer. Don't save it as a .json string but as a python dictionary.
Alternatively you can use this module to register components as you see fit or add your own functions to the asyncio loop. See the template for the components.py file in the template directory here.
The basic configuration options of a component are when using SmartServer or COMPONENTS dictionary in components.py:
The tools directory contains a lot of scripts I use to synchronize the project files with the build directory of the microcontroller and for building and flashing firmwares. They are heavily specialized to my environment (my home path e.g.). These can be run in a linux shell (windows wsl too). I'm not that skilled in making these very generally usable but I keep them in the repository in case someone finds them useful or even contributes better ones. (Would be very happy about that!)
A small overview of the directory structure:
I'd like to provide a small flowchart which makes it easier to understand how this project works and which file is called when. It shows which modules are imported by which files first (as some are imported in different files of course) in a temporal flow. Only the most important modules are shown, not util modules.
This project is licensed under the MIT License - see the LICENSE.md file for details.