Awesome Open Source
Awesome Open Source

Jarvis Home Automation Build Status

Jarvis is a Home Assistant ( implementation with wake word detection, SMS commands, and a lot of automation control. While the Conversation component does it's job, it's currently a bit limited and without wake word detection it was almost useless to me. I also have tried the AlexaPi implementation as well as using Amazon Alexa products. However, creating custom intents was not as straight forward as I would have liked for quick and easy creation. Between wake word and API.AI for speech handling, it appears to work rather well. I have also decoupled the TTS services to make it easier to use that functionality elsewhere.

Firewall Note: The Jarvis firewall is setup so that only local clients and one specific public IP can access. The specific IP that can access Jarvis is a front end server with Apache as a reverse proxy and encryption. Any other external clients will connect to the front end server outside of the network that then connects direct to Jarvis.

Note: SMS commands will require a basic level of experience with PHP.

P.S.: This system is far from perfect, if you make some enhancements or have any ideas please feel free to share them. If you find any bugs you would like to report, you can do so via the following link: Keep in mind, generally any Home Assistant related issues should be raised on their side. If you are having a configuration problem, I would be glad to help.

You will need to read this section completely before using this software to ensure that everything works as designed. There is no other documentation other than this README file. Please note: this software comes with no guaranteed support. Furthermore, this software is licensed under the Apache License, Version 2.0. License text is provided in the LICENSE file.

Front End Integrations

  • Meteorologisk Institutt
  • Hue
  • Speedtest
  • Apple iCloud
  • Minecraft
  • Roku
  • uPnP


This software makes a couple of assumptions, I will list them below and you can either update/install what is needed on your system or make the required changes. Even though the software is open source, some services currently are utilizing paid services. The main service is Wit AI Speech. While there are free alternatives for speech-to-text services. I have found them to be less than desirable. Luckily, most have a free tier available to use.

For everything to work, you will need the following API keys:

If you want SSL you can purchase and install a certificate. However, I find it much easier to sign up for a free Cloud Flare ( plan and encrypt your external site that way.

Note: I am going to include my current configuration and layout as an example. The systemd services are included in the resources folder if you wish to edit them. The installer will take care of setting this up for you.

Configuration Files

  • /opt/jarvis/configuration.yaml
  • /opt/jarvis/secrets.yaml
  • /opt/jarvis/stt/config/*.yaml
  • /opt/jarvis/web/webapi/sms/config.php

If you utilize the install script it will set you up with ready to edit configuration files. Otherwise, you can copy the .example files located in each primary directory.


This assumes you are setting up Home Assistant from scratch.

  1. Download the Raspian Lite image, and burn to an SD card for your Raspberry Pi.
  2. Boot your Raspberry Pi and set the initial system configuration.
    1. Login with the default user pi and password raspberry.
    2. Change the password with the command "passwd" and change to something other than the default.
    3. Run the program "sudo raspi-config" to initially setup your Pi.
      1. Hostname menu, set the hostname you wish to have your Pi appear as on the network.
      2. Interfacing Options menu, enable at a minimum SSH for remote access.
      3. Advanced Settings, set the memory split to 16 and audio to force 3.5mm.
      4. Reboot your Pi.
  3. Navigate to /opt "cd /opt" on your Raspberry Pi.
  4. Checkout the latest Jarvis program:
    1. Install git: sudo apt-get install git
      1. If this command fails, you may need to run: sudo apt-get update
    2. Then run: sudo git clone jarvis
  5. Update permissions: sudo chown -fR pi:pi jarvis
  6. Navigate to /opt/jarvis "cd /opt/jarvis" and run
  7. Configure AWS CLI for Amazon Polly TTS: aws configure --profile adminuser
    1. You will need to obtain your AWS Access Key, and Secret Key before you begin
    2. Default Region Name: us-east-1
    3. Default Output Format: text
  8. Edit the configuration options in /opt/jarvis/secrets.yaml
  9. Setup API Access
    1. API.AI
      1. Navigate to
      2. Create a new a new Agent
      3. Go to the Agent Settings, make note of your Client Access Token
      4. Navigate to Export and Import, and RESTORE FROM ZIP: /opt/jarvis/resources/
    2. Amazon AWS
    3. Wit AI Speech
  10. Edit the configuration options in /opt/jarvis/stt/config/config.yaml
  11. Update /opt/jarvis/configuration.yaml
  12. Update SMS Commands
  13. Edit the PHP file /opt/jarvis/web/webapi/sms/config.php
  14. Update the commands you wish to execute in functions.php
  15. Update Pianobar (Pandora) configuration
  16. Edit /home/pi/.config/pianobar/config and update email/password for Pandora to work
  17. Update the floorplans, refer to
  18. Reboot your Raspberry Pi

If all is well, your system will be 100% ready to go. If you have an attached screen/touch screen the main panel will appear after all systems boot. If not, navigate to http://[raspberry pi IP]:8123.


Once installed you can use the following system services to restart/control the entire system.

Restart Everything:

sudo service jarvis* restart

Restart Home Assistant:

sudo service jarvis-base restart

Restart Speech to Text:

sudo service jarvis-stt restart

Restart Text to Speech:

sudo service jarvis-tts restart

Restart Kiosk:

sudo service jarvis-web restart

If you get a TLS fingerprint mismatch error with Pianobar/Pandora run this command to get the updated one:

openssl s_client -connect < /dev/null 2> /dev/null | \
    openssl x509 -noout -fingerprint | tr -d ':' | cut -d'=' -f2

Screenshot of Jarvis Home Screenshot of Jarvis Lights Screenshot of Jarvis Climate Screenshot of Jarvis Security Screenshot of Jarvis SMS

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (1,146,154
Raspberry Pi (7,581
Automation (7,038
Iot (5,838
Home Assistant (2,515
Home Automation (1,945
Amazon (1,097
Alexa (798
Internet Of Things (712
Light (675
Speech (632
Home Assistant Config (294
Hue (248
Ifttt (220
Touchscreen (196
Led Controller (154
Stt (90
Hue Lights (83
Garage Door (30
Wake Word Detection (23
Related Projects