An open-source software solution for situational awareness from a network of video and audio sources. Utilizing Home Assistant, addons, the LINUX Foundation Open Horizon edge fabric, and edge AI services, the system enables personal AI on low-cost devices; integrating object detection and classification into a dashboard of daily activity.
Start-to-finish takes about thirty (30) minutes with a broadband connection. There are options to consider; a non-executable example script may be utilized to specify commonly used options. Please edit the example script for your environment.
The following two (2) sets of commands will install motion-ai
on the following types of hardware:
arm
); 2GB recommendedamd64
); 2GB, 2vCPU recommendedarm64
); 4GB requiredThe initial configuration presumes a locally attached camera on /dev/video0
. Reboot the system after completion; for example:
sudo apt update -qq -y
sudo apt install -qq -y make git curl jq
git clone http://github.com/dcmartin/motion-ai
cd ~/motion-ai
sudo ./sh/get.motion-ai.sh
sudo reboot
When the system reboots install the official MQTT broker (aka core-mosquitto
) and Motion Classic (aka motion-video0
) add-ons using the Home Assistant Add-on Store (n.b. Motion Classic add-on may be accessed by adding the repository http://github.com/dcmartin/hassio-addons to the Add-On Store.
Select, install, configure and start each add-on (see below). When both add-ons are running, return to the command-line and start the AI's. After the MQTT and Motion Classic addons have started, run the make restart
command to synchroize the Home Assistant configuration with the Motion Classic add-on, for example:
cd ~/motion-ai
make restart
Once the system has rebooted it will display a default view; note the image below is of a configured system:
A more detailed interface is provided to administrators only, and includes both summary and detailed views for the system, including access to NetData and the motion add-on Web interface.
Install the MQTT and Motion Classic add-ons from the Add-On Store and configure and start; add the repository https://github.com/dcmartin/hassio-addons to the Add-On Store to install Motion Classic.
The Motion Classic configuration includes many options, most which typically do not need to be changed. The group
is provided to segment a network of devices (e.g. indoor vs. outdoor); the device
determines the MQTT identifier for publishing; the client
determines the MQTT identifier for subscribing; timezone
should be local to installation.
Note: No capital letters [A-Z], spaces, hyphens (-), or other special characters may be utilized for any of the following identifiers:
group
- The collection of devicesdevice
- The identifier for the hardware devicename
- The name of the cameraThe cameras
section is a listing (n.b. hence the -
) and provide information for both the motion detection as well as the front-end Web interface. The name
,type
, and w3w
attributes are required. The top
, left
, and icon
attributes are optional and are used to locate the camera on the overview image. The width
, height
, and other attributes are optional and are used for motion detection.
...
group: motion
device: raspberrypi
client: raspberrypi
timezone: America/Los_Angeles
cameras:
- name: local
type: local
w3w: []
top: 50
left: 50
icon: webcam
width: 640
height: 480
framerate: 10
minimum_motion_frames: 30
event_gap: 60
threshold: 1000
- name: network
type: netcam
w3w:
- what
- three
- words
icon: door
netcam_url: 'rtsp://192.168.1.224/live'
netcam_userpass: 'username:password'
width: 640
height: 360
framerate: 5
event_gap: 30
threshold_percent: 2
Return to the command-line, change to the installation directory, and run the following commands to start the AI's; for example:
cd ~/motion-ai
./sh/yolo4motion.sh
./sh/face4motion.sh
./sh/alpr4motion.sh
These commands only need to be run once; the AI's will automatically restart whenever the system is rebooted.
The overview image is used to display the location of camera icons specified in the add-on (n.b. top
and left
percentages). The mode may be local
, indicating that a local image file should be utilized; the default is overview.jpg
in the www/images/
directory. The other modes utilize the Google Maps API; they are:
hybrid
roadmap
satellite
terrain
The zoom value scales the images generated by Google Maps API; it does not apply to local
images.
The motion-ai
solution is composed of two primary components:
Home Assistant add-ons:
motion
- add-on for Home Assistant - captures images and video of motion (n.b. motion-project.github.io)MQTT
- messaging brokerFTP
- optional, only required for ftpd
type camerasOpen Horizon AI services:
yolo4motion
- object detection and classification
face4motion
- face detection
alpr4motion
- license plate detection and classification
pose4motion
- human pose estimation
Data may be saved locally and processed to produce historical graphs as well as exported for analysis using other tools (e.g. time-series database InfluxDB and analysis front-end Grafana). Data may also be processed using Jupyter notebooks.
CPU only
amd64
- Intel/AMD 64-bit virtual machines and devicesaarch64
- ARMv8 64-bit devicesarmv7
- ARMv7 32-bit devices (e.g. RaspberryPi 3/4)GPU accelerated
aarch64
- with nVidia GPUamd64
- with nVida GPUarmv7
- with Google Coral Tensor Processing Unitarmv7
- with Intel/Movidius Neural Compute Stick v2Installation is performed in five (5) steps; see detailed instructions. The software has been tested on the following devices:
tegra
)Recommended components:
ncs2
)This configuration includes dual OLED displays to provide display of annotations text and image, as well as a USB-attached camera (n.b. Playstation3 PS/Eye camera). The Intel/NCS2 implemtation is still in alpha mode and not in the master
branch.
Low-cost computing (e.g. RaspberryPi, nVidia Jetson Nano, Intel NUC) as well as hardware accelerators (e.g. Google Coral TPU, Intel Movidius Neural Compute Stick v2) provide the opportunity to utilize artificial intelligence in the privacy and safety of a home or business.
To provide for multiple operational scenarios and use-cases (e.g. the elder's activities of daily living (ADL)), the platform is relatively agnostic toward AI models or hardware and more dependent on system availability for development and testing.
An AI's prediction quality is dependent on the variety, volume, and veracity of the training data (n.b. see Understanding AI, as the underlying deep, convolutional, neural-networks -- and other algorithms -- must be trained using information that represents the scenario, use-case, and environment; better predictions come from better information.
The Motion Ã👁 system provides a personal AI incorporating both a wide variety artificial intelligence, machine learning, and statistical models as well as a closed-loop learning cycle (n.b. see Building a Better Bot); increasing the volume, variety, and veracity of the corpus of knowledge.
This system may be used to build solutions for various operational scenarios (e.g. monitoring the elderly to determine patterns of daily activity and alert care-givers and loved ones when aberrations occur); see the Age-At-Home project for more information; example below:
Releases are based on Semantic Versioning, and use the format
of MAJOR.MINOR.PATCH
. In a nutshell, the version will be incremented
based on the following:
MAJOR
: Incompatible or major changes.MINOR
: Backwards-compatible new features and enhancements.PATCH
: Backwards-compatible bugfixes and package updates.David C Martin ([email protected])
netcam
or local
camera and let me knowAdd motion-ai
as upstream to your repository:
git remote add upstream http://github.com/dcmartin/motion-ai.git
Please make sure you keep your fork up to date by regularly pulling from upstream.
git fetch upstream
git merge upstream/master
CLOC
Language | files | blank | comment | code |
---|---|---|---|---|
YAML | 308 | 1709 | 1636 | 27069 |
JSON | 1198 | 1 | 0 | 3399 |
Bourne Shell | 48 | 511 | 279 | 2540 |
Markdown | 10 | 310 | 0 | 1139 |
Jupyter Notebook | 1 | 0 | 1020 | 927 |
make | 3 | 135 | 81 | 866 |
Python | 6 | 51 | 36 | 299 |
HTML | 1 | 19 | 1 | 89 |
-------- | -------- | -------- | -------- | -------- |
SUM: | 1575 | 2736 | 3053 | 36328 |