Author: Jaimyn Mayer

My home automation system overview (main components)

Post author By Jaimyn Mayer
Post date January 30, 2017
1 Comment on My home automation system overview (main components)

I’ve been working on my home automation system quite a bit recently. So far I’ve automated a few things, namely my work room’s light and power, and my bedroom’s light and fan. When I get around to it I’ll be automating all the outside lights and some more power points. In this post I’ll quickly go over all the components you need for my home automation system and what they do.

The main components are:

MQTT Broker (Server) – this is the “communications mainframe”
Home Assistant Server – a nice interface/automation server
Server Hardware – I use an orange pi zero
Automation hardware – WiFi relays, sensors etc

1) Mosquitto (The MQTT Broker)

Mosquitto is an open source MQTT broker. Now the first thing to explain is what is MQTT? I mentioned it’s like a communications mainframe, well it kind of is. If you aren’t familiar with the publish subscribe concept I suggest you read up on it. However, the main concepts are you have clients that connect to a broker. (MQTT terminology in bold) A client can subscribe or publish a message to a topic. A topic might look like “sensors/kitchen/temperature”. When a client is subscribed to a specific topic, that client will receive all messages published to that topic.

This tutorial by Digital Ocean is a fantastic resource for setting up the basics of an MQTT broker. It’s a great introduction on how to set up a working broker, and how to secure it. However, I reccomend that you go one step further. See how to use ACLs and multiple user accounts to limit what topics each client can subscribe and publish to. This ensures if an account is ever compromised, the damage can be contained.

2) Home Assistant

Home Assistant is an open source, python 3 based home automation hub. They had me at open source and python 3. 😉 It is fairly easy to use but the killer feature is it’s integration. It integrates with everything! I can almost guarantee if you want something integrated it already has a compatible component.

The main components I use are WeMo (curse thee belkin, I should never have wasted my money on a WeMo), chromecast, MQTT, owntracks and pushbullet. I’m still adding more when I have the time so this list will probably be out of date quickly. It’s configured with a single yaml file, which is a little annoying and will definitely get convoluted fast.

To see the ever expanding list of built in components check out this link. In my opinions its the best one out there. It also has by far the nicest looking interface, it comes with a mobile friendly web app.

3) Server Hardware

This title is slightly misleading as you don’t really need a server. I bought a few Orange Pi Zeros and have been pleasantly suprised at how well they work. In Australia, I could get 2 for a similar price as a single raspberry pi zero. The raspberry pi is also only available in quantities of one at the moment. But the main reason I like them better for this purpose is the full size ethernet and USB port on board, and the built in WiFi.

The full sized ethernet port is a big plus, makes it so much easier to work with. However, with great price comes terrible software support. I didn’t feel that comfortable with a dodgy looking baidu cloud/google drive link for the operating system. I ended up trying it on only to find out it wouldn’t boot from any “official” image. However, armbian to the rescue! They support the orange pi zero and it booted up straight away. I could see it on my network so I went ahead and installed mosquitto and home assistant.

Here is the official armbian page and download links for the orange pi zero. I like ubuntu so that what I went ahead with. Note, neither of these operating systems contains a desktop environment so you’ll have to get familiar with the command line.

4) Automation Hardware

A couple years ago I bought a belkin WeMo for my 3D printer; I wanted to be able to turn it off when it was done and I wasn’t home. However, it’s a terrible product. Sorry Belkin, you tried, but it just doesn’t work. It’s slow, laggy and never stays connected properly. The only thing going for it is it’s price tag, oh wait I forgot it’s $70. But at least they make an api available – actually nevermind they don’t do that either. To be fair it was one of the first plug and play WiFi relays around so I’ll give them that.

My favourite piece of hardware at the moment is the sonoff by itead. It needs a bit of hacking to get going. But once you’ve loaded TASMOTA onto it they’re awesome. TASMOTA is basically plug and play; upload, configure and you’re done. Basically, you wire a sonoff in between any device/appliance you want to control wirelessly. I’m renting and have decided not to modify any of the wiring in my house because I value our insurance validity.

NOTE: Do not try this unless you know what you’re doing and you have a qualified electrician look over your work. Something as simple as mixing up the colour of the wire can cause someone in the future to shock, injure or kill themselves! (electrical standards exist for a reason)

What I’ve decided to do is buy some slampher‘s to go on my lights where they fit. You can modify the button and wire one into a blank faceplate. This means you’ve still got a manual butotn but it’s WiFi connected. For all other appliances I’m making up power boards and extension boards with a sonoff built in. This means I can plug anything in and make it WiFi connected – all without voiding my house insurance and upsetting my landlord.

Another option for lightswitches is the sonoff touch. However I don’t like it’s price tag or the fact I legally have to get an electrician to install it for me. I’d rather a slampher and DIY button for a fraction of the cost. That being said, they do look really nice.

That’s my brain dump for tonight. I’ve gone into more detail of my MQTT setup and will eventually do the same for all the other topics. (see what I did there 😉 ) Hopefully this is a good overview of all the major components needed and will point you in the right direction.

Home Automation MQTT

MQTT – How to use ACLs and multiple user accounts

Post author By Jaimyn Mayer
Post date January 30, 2017
4 Comments on MQTT – How to use ACLs and multiple user accounts

I’ve previously written about how awesome MQTT is and how it’s an integral part of my home automation system. This tutorial will show you how to make your MQTT broker more secure. In any sort of information technology you should always use the principle of least privileges. Basically, only give each account the bare minimum access that they actually need.

1) Define Needs

Lets use my home automation system as an example. I’ll limit to just a few clients in this example. I have my automation server (home assistant), a WiFi light (a sonoff) and a light sensor. Lets state the minimum needs of each client:

Home Assistant

Process all MQTT messages to allow full control of devices and to run automation scripts

WiFi light (sonoff)

Subscribes to “cmnd/light/POWER” for control
Subscribes to “cmnd/light/UPDATE” for OTA updates
Publishes to “stat/light/POWER” for status and confirmation

Light Sensor

Subscribes to “cmnd/sensor/kitchen/light” for a manual sensor reading
Subscribes to “cmnd/sensor/kitchen/light/set” to change settings
Subscribes to “cmnd/sensor/UPDATE” for OTA updates
Publishes to “stat/sensor/kitchen/level” with current light level

2) Create User Accounts

We can now see we have three clients with distinct, separate needs. This is a good case where three MQTT user accounts would be beneficial. Let’s say our Light Sensor has a security flaw that accidentally exposes the password, we don’t want to give someone full access to our system!

If you haven’t already, follow Digital Ocean’s tutorial on setting up and securing Mosquitto. To add a new user account is quite easy. Simply run the following command and follow the prompts to enter a password.

sudo mosquitto_passwd /etc/mosquitto/passwd <new-user>

So in total, you should run 3 commands that look something like these, note how I like to obscure the usernames a little. Call me paranoid, but it makes them just a little bit harder to guess.

sudo mosquitto_passwd /etc/mosquitto/passwd homeassist
sudo mosquitto_passwd /etc/mosquitto/passwd sonoffswitch
sudo mosquitto_passwd /etc/mosquitto/passwd lightsense

Note: make sure you use a very secure password for the home assitant account as it will have full access!

3) Tell Mosquitto to use ACLs

Open your mosquitto configuration file:

sudo vim /etc/mosquitto/conf.d/default.conf

Add the following line, specifying where you put your ACL file. (I put mine in the same directory as the passwd file)

acl_file /etc/mosquitto/acl

MQTT has two types of wildcards:

“#” means literally everything and is “recursive” so can only be used on the end of a topic.
1. example “cmnd/light/#” will receive every message from topics that start with “cmnd/light“.
“+” meanse literally everything but only one level, so one or more may be used inside a topic.
1. example “stat/+/POWER” could receive the “POWER” message from every device.

Generate your ACLs

Open your ACL file and add your topics and user accounts to it like below. Note the three types of permissions; there are read, write and readwrite. Wild cards may also be used. Also, please for the sanity of future you document what each one does!

# Give Home Assitant full access to everything
user homeassist
topic readwrite #

# Allow the sonoffs to read/write to cmnd/# and stat/#
user sonoffswitch
topic readwrite cmnd/#
topic readwrite stat/#

# Allows the light sensor to read/write to the sensor topics
user lightsense
topic cmnd/sensor/#
topic stat/sensor/#

Notice how the light sensors have a similar permission level as the sonoffs? The sensors don’t need to read/write to the sonoffs so they shouldn’t be allowed. However, we may want the sonoffs to be able to directly read the sensors.

4) Save and profit ??

Save all the settings and make sure they’re right. Normally I wouldn’t reccomend it, but it may be useful to right down the usernames and passwords of all the accounts. This could save a lot of confusion with all the different accounts. (of course you should destroy the passwords after)

Be sure to restart both home assitant if you’ve got it running and the Mosquitto server so that the changes can be applied.

sudo service mosquitto restart
sudo service homeassistant restart

Let me know how you go or if you run into trouble in the comments below. I couldn’t find much documentation on actually implementing ACLs and multiple user accounts so hopefully this makes it easier for others.

Electronics Home Automation

How to use Espressif SDK functions in the Arduino ESP8266 IDE

Post author By Jaimyn Mayer
Post date January 25, 2017
No Comments on How to use Espressif SDK functions in the Arduino ESP8266 IDE

I found myself wanting to use one of the espressif SDK functions in the ESP8266 Arduino IDE. It took me longer then it should have to work it out so I thought I’d share it! The first step is to make sure you’ve got the ESP8266 library and boards setup properly. (see the documentation) In the sketch you’d like to include the espressif function in, paste the following up the top:

extern "C" {
    #include "user_interface.h"
}

Just replace “user_interface.h” with the name the espressif documentation tells you to. What this does is tells the arduino IDE to include the espressif library. (which is a standard C library) This little trick should also work for any standard C library.

The correct name for each API can be found at the top of the relevant espressif documentation page. (their documentation – choose the latest “NON-OS SDK”) Here is an example of the force sleep api:

Electronics Power Wall

Make a 10 way Li-Ion battery charging station for $10

Post author By Jaimyn Mayer
Post date December 5, 2016
6 Comments on Make a 10 way Li-Ion battery charging station for $10

So my DIY Powerwall (like the Tesla one) project is moving along quite nicely. I’ve just finished my mass charging station design and I’m quite happy with it. This design is a lot simpler/easier to make (for most people) than my normal designs. It doesn’t require any laser cutting because the laser cutter at HSBNE (brisbane hackerspace) was out of action briefly. It still requires 3D printed parts but a bit of extra work means you can use the normal 18650 holders.

The Components

1) Ply wood mounting plates.

I used two rectangular plywood plates that I cut out by hand. (yes, I didn’t use a laser cutter 😉 ) The top one is for mounting everything to, whilst the bottom one is to allow for a level bottom and to hide all the wiring. Cut out the two plates so you can fit everything on with some room around the edges for the wiring.

2) 18650 Chargers (TP4056)

These are one of, if not the cheapest Li-ion/LiPo battery chargers that exist. It’s an all in one board with through holes for 5v in and a battery output. They will charge any Li-ion or LiPo battery up to full from either a USB connection or 5v to the two pads either side of the USB plug. The cheapest place to get them I found was Aliexpress. (I got mine for 30c each on the 11/11 sale!)

3) Computer power supply

Any generic 5v power supply will work. However, you’ll need quite a lot of current to charge more than a couple of batteries at once. Old (or new) computer power supplies are perfect. Most can supply around 30 amps on the 5v rail. Each battery needs about 1 amp so this means it should be good for about 25-30 batteries at once.

Tech Tip: Some power supplies only watch the 12v line for voltage drop, so if there is no load the 5v rail could drop below 4v! The easiest way to fix this is to put a load on the 12v rail. I find putting a few 12 fans to cool the chargers down helps. See below for optional monitor.

4) Wiring, connectors and (optional) fan

You’ll need a bunch of wiring and a connector if you want to easily disconnect it from the power supply. I find old network cables are great for this job. A single 5m network cable has about 40m of wiring in it! The wires are tiny and can carry enough current for this job, making them the perfect fit. Solder one wire to each + and – near the USB connector, join them all up and use some thicker wire to join the bunch to a connector. Connect the fan to the 12v rail and glue it onto the side. If you get a small bit of cardboard/wood you can direct the flow a little better.

5) Voltage Monitor (optional)

I bought a few of these voltage monitors off eBay. They are super cheap and easy to wire, simply put the red wire on the positive side and black on the negative side of whatever you’re measuring. It even powers itself from the same source. This is great for watching the voltage of the power supply. I have mine on the 5v rail of the power supply so you can keep an eye on it. Cheap Voltage Monitor (eBay link)

Once it’s all put together this is what it looks like. Quite a good little package I think, and really cheap at about $1 per battery slot. (assuming you have access to a 3D printer, box of power supplies, network cables and connectors like me 😉 )

Uncategorised

sonoff WiFi relays

The sonoff WiFi relays have arrived. I ended up buying ten of them and 3 motion sensors. My first impression is that they’re tiny and solid. They’re much smaller than I thought, which is a good thing! The case they come in is perfect for mounting inline with something and neatly hides the exposed wires. For comparison, you can see my old LG G4 phone next to it.

On the inside

On the inside, they look pretty good. The soldering is done well and the gaps between the mains traces is reassuring. As you can also see from the picture below there are a few header pins. These are the programming pins. Itead has been nice enough to breakout the programming pins into headers to make it easier to reprogram with your own code.

Reliability

I’ve currently had one set up on my desk lamp for the last couple of days. It has been rock solid and hasn’t experienced any drop outs or glitches. This was running their stock firmware which allowed me to connect it to their app. Although I have no intention of continuing to use their app it is miles ahead of the Belkin system. For example, switching it on or off happens via the internet almost instantaneously. However the Belkin’s system sometimes takes 10 seconds!

Uncategorised

How to make an autonomous car (code included)

Post author By Jaimyn Mayer
Post date October 8, 2016
56 Comments on How to make an autonomous car (code included)

I’ve just finished a recent side project with my friend Kendrick. (his GitHub) We built an autonomous car that you can teach how to drive, then it drives around by itself. I did all of the hardware/arduino software and Kendrick did all of the machine learning software. He called his project Suiron and it’s available on GitHub here. The code running on the arduino is called car-controller and is available on my GitHub here.

Now that you’ve got the links feel free to go and have a look. Work through the code and try to figure out how it works. I’ll try to briefly cover the software here but my main focus will be on the hardware. One thing to note are the open source licenses, all my stuff is GPL and Kendrick’s Suiron project is MIT.

This post is more intended as an overview of how the whole thing works. If I get time I might turn it into a tutorial on how to get it working yourself.

Before we begin here is a short video of it in action.

Now onto the fun stuff! How does it work?

The Hardware

These are the main components used.

1) Remote Control Car – we used this car (link) but anything of similar size will work. As long as it has a standard ESC and Steering Servo. It comes with a remote control, battery and charger to start with. I recommend buying a new remote control system. (link 5 below)

2) Inten NUC – The raspberry pi doesn’t really have enough power and is arm based. An x86 based processor like the i5 in our NUC is much easier to use for machine learning purposes. The exact one you use doesn’t matter.

3) Battery for NUC – A standard laptop battery bank was used to power it. This one from kogan gives about 6-10 hours of runtime. (link)

4) Logitech C920 Webcam – Buy from any reputable retailer. You could probably use a cheaper model but we had better luck with a C920. (link)

4) Lens filters – if you are operating in any sunlight, you will want a Polarising and ND (Neutral Density) filter. The camera just can’t cope with the harsh sunlight and shadows so these filters help bring the conditions into something much better. A variable ND is great as it let’s you adjust the “darkness” level.

5) Radio control system – if you intend on doing lots of this stuff then get an FrSky TARANIS. You won’t be disappointed. Otherwise, a turnigy 9XR will work just as good. Make sure you get a receiver too if it isn’t included.

6) You’ll also need an arduino. I like the Arduino Nano’s because they are super cheap and have on board USB.

If you want some specific help choosing components just leave a comment.

I won’t go into details on how to wire everything as this isn’t a tutorial. However, If you need some help drop a comment below. I suggest you learn how an ESC (electronic speed controller) works together with a motor, receiver, servo and battery. This is a standard setup on normal remote control cars. Once you understand that you should look at arduino’s and how to use them to blink lights and read inputs. Read through the arduino code and the wiring should be pretty self explanatory.

How it all fits together

It’s up to you how you put everything together. I recommend trying to keep everything as low as possible for better stability when driving. The webcam needs to be mounted up high so it has a better chance of seeing the lane that it’s in. I just used a square bit of balsa wood as it’s really light and strong, then glued the webcam to it. Instead of explaining exactly how I mounted everything I’ll dump a few pictures here. All the white things are 3D printed, but you could easily do it without a 3D printer.

Polarising/ND Filter

The importance of a polarising filter cannot be underestimated. It reduces reflections and the harsh glare sometimes encountered. In the image below (credit) you can see how much of a difference a polarising filter can make. Now water is a bit of an extreme example, but I chose that picture so it’s easier to demonstrate the difference. In realty, where we’re operating the difference won’t be so obvious.

The neutral density filter is equally or more important than the polarising filter. The ND filter is basically like sun glasses for the webcam. The webcam doesn’t like really harsh light so it reduces the intensity of it without interfering with the image to much. The picture below (credit wikipedia) shows how much better the right ND filter can make an image in harsh light.

I suggest making the lens filters removable as it will make the image to dark in lower lighting situations. For example, it was perfect mid day but much to dark a few hours later just before dusk. I made a simple mount that just uses an alligator clip to hold the filters in place. The filters are both glued together then onto a small 3D printed right angle mount.

The Arduino

The diagram below shows how everything is hooked up. Basically the arduino is the “brains of the hardware”. It reads in the values from the R/C receiver (bottom left) and then decides what to do based on the mode channel. Dig through the arduino code (link) and see exactly how. Basically there are 3 modes, manual, autonomous and emergency stop.

In manual mode the arduino reads in the steering and motor values and passes it straight to the motor and steering servo. In this mode with the right flag enabled, it also sends back over UART what those values are every time it receives a character. (every time it receives prevents the serial buffer getting full and “lagging”) In autonomous mode the arduino reads inputs over UART from the NUC. In this mode it receives two messages; steer,x and motor,x where x is the value you want to set it to. It then writes those outputs to the steering servo or motor. Finally, emergency stop kills the motor output and straightens the steering servo. This emergency stop overrides any sort of manual or autonomous control.

The Machine Learning Part

This isn’t my expertise so I’ll briefly summarise what it’s doing. (not really how it’s doing it, no one really knows) We used a library called Tensor Flow. It’s an Open Source machine learning library published by Google. It’s open source and released under an Apache license. It has a nice python and a “no nonsense” C++ api.

Collecting data

This is a really short summary of the whole process. Each time a video frame is recorded Suiron (software on the NUC) asks car-controller (software on arduino) what the human operator is doing. Remember, in manual mode the human operator is driving the car around. Car-controller responds by sending the current steering and motor values back to Suiron. Suiron takes these values and saves them along with a processed version of the frame.

This process happens at about 30Hz (or 30 times per second) for as long as you record data. In the final model, we used about 20 minutes worth of training data. That is 20 minutes of continuously driving around the track. It may not seem like a lot but it’s repetitive very quickly. 😉 In reality, 20 minutes is no where near enough data. It works great on this particular track with similar lighting conditions but would likely fail if the conditions changed to much.

Training data

Again, I’m not an exert at this but I’ll try to briefly explain how the training works. Convolutional Neural Networks (CNNs) are weird in the way they work. It’s impossible to know exactly how or why a CNN works. Basically, we’re giving Tensor Flow the frame and two numbers. (steering and motor) Then we’re asking it to work out how the frame relates to those two numbers. After giving it hundreds of thousands of examples (frames) it can try to generalise a model.

Because of the amount of computing power required it takes a very long time to train a good model. Due to the type of calculations it has to do, Tensor Flow runs much faster on a dedicated GPU. With only 20 minutes of data our model took half a day to train properly. The training took place in a desktop with a borrowed GTX980, a GPU that’s towards the higher end of consumer graphics cards.

Using the model

You can see it in action in the gif below. The blue line is what the model thinks it should do, the green line is what I actually did when I was steering it. Note that this data was not included in the training set, this is to ensure the model works with other data.

Once it has been trained we can then use the model. Basically, what happens is we collect just a frame from the webcam. Then we pass it to Tensor Flow and ask it to run it through the model. The model then spits out what it thinks our two values should be, one for steering and one for throttle. At the moment the throttle is unused and it runs at a constant speed. However we thought we’d include it just in case we wanted to use it in the future.

Update: Clive from hobbyhelp.com reached out to me after seeing this. He’s got a pretty cool “Ultimate beginners guide to RC cars” article on his website here. I recommend checking it out if you want to get started doing something similar to this project.

Uncategorised

sonoff home automation products

Post author By Jaimyn Mayer
Post date October 3, 2016
No Comments on sonoff home automation products

I’ve discovered a really cool product that is cheaper to buy than what I was making them for. Plus it looks a lot neater and is probably safer seeing as I’m not a qualified electrician. 😉

These sonoff “smart switches” (link) are exactly what my home automation system is looking for. Basically, the sonoff switches has a mains to 3.3v regulator, relay, ESP8266 and a button/LED all on board. For about $5. The manufacturer has even broken out the serial pins so it’s easy to upload your own code. I’ve bought about 10 of these little devices after hearing great reviews about them from the internet.

I intend on automating as much as I can with my home. I’m going to make all the automation switches MQTT compliant which make it easier to expand and/or change things around later. I’m going to be making a personal companion (much like Siri or Alexa) that can answer useful questions and do some cool things around the house. Eventually, small remote control modules likely running raspberry pi zero’s will be placed around the house so you can pick one up and ask the house to do things.

I’ll post an update when the sonoff modules arrive and post heaps of pictures!

Uncategorised

What is free Software?

I strongly believe in the philosophy of open source and free software. Most of the projects and code I publish on this site and my GitHub are released under the GNU GPL v3 or later license. What is this GNU GPL you say? Well, it’s a type of software license you must abide by. If you’ve ever downloaded a program that asks you to accept something, it’s likely the license agreement. The GNU GPL is great, it lets anyone do anything they want with the software, as long as they pass along the same freedoms.

The GNU GPL v3 logo

Free software generally has no price attached to it. This means you can download and use the software at no direct monetary cost. However, the greatest benefit is having the ability to modify the code and make changes. This allows you to improve the software and release an even better version for other people to use. This may include adding new features, or fixing problems like bugs and security flaws.

That was a quick overview of what “free” software is and why I love it so much. For more information and some great reading I suggest checking out the GNU project’s website by clicking the link: www.gnu.org

Uncategorised

First post!

Hey!

This is my first blog post. The aim is to fill this blog with heaps of projects and lots of random tips and tricks that I find during my day to day hacking. I will also list all the hardware/software I make and make it available for download and/or purchase. Thanks for visiting! 🙂