aquova.net Github's logo Mastodon's logo Blue Sky's logo Backloggd's logo Pico-8's logo The RSS logo

My Smart Home, and How It Got Here

Smart home devices have become all the rage in recent years, and as a nerd, it was inevitable that I would end up with an increasing number myself. These range from things like light bulbs and lamps to temperature and weather sensors to smart TVs and vacuums. The mainstream method to managing these devices is to purchase a hub device from one of the large tech enterprises and lock yourself into their ecosystem. This has the advantage of ease of setup and use, with a guarantee that everything will more or less work as expected. However, you gain this convenience at not only the loss of your own privacy, but also by losing the ability to include any gadget into your ecosystem. You're tied into what your Apple Nestlexa allows you to use, and you're beholden to how they do things.

I've spent a good deal of time playing around with some of the more DIY solutions, and I've settled on an infrastructure that maintains as much functionality locally as possible while allowing for customization and interconnectivity. Some of this will likely be familiar to those of you who have set up your own smart home infrastructure before, but for those who are looking for more seamless solutions, this may give you some ideas.

A Word Against Wi-Fi

Before diving into my own stack, I want to begin by discussing the different hardware options. It's almost a given that for maximum utility your smart devices will need to communicate wirelessly. This allows for remote placement around the home while still remaining fully featured. There are several different protocols for communicating wirelessly between devices, with the most widespread being over Wi-Fi.

For those first wanting to dip their toes into smart devices, getting one with Wi-Fi capabilities is perhaps the obvious choice. Nearly everyone has a wireless router these days, which means that a dedicated -- and expensive -- smart home hub isn't required. These are far and wide the most common devices, even my local supermarket has a number of Wi-Fi capable light bulbs available. These are precisely how I started getting into smart devices, and while this is a great option if you simply want a few RGB lights around the house, they do not scale up well at all.

Firstly, there's an issue with control. By not requiring a dedicated hub, many Wi-Fi devices are instead administered by an accompanying smart phone app. Again, this is fine for only a few devices, but once you start adding many to your network, you begin to pollute your Wi-Fi network. Many of these devices often want their own static IP address, which complicates setup further. As you branch out, you'll often also want devices from different manufacturers, which each have their own dedicated app. If you have a lamp plugged into a smart plug from Company A but light bulbs from Company B, there's no way to control them together without some third party control system, defeating the purpose of avoiding buying a hub. Finally, but perhaps most severely, there is the issue with your Wi-Fi router having internet access. Even ignoring the (considerable) privacy implications of this, performing a simple action with these devices is much more intensive than it needs to be. A message is sent from your phone to your router, which then travels the internet to a cloud server somewhere, parses the request, then sends the appropriate response all the back to your home and finally to your appliance. What should be a simple local operation now relies on an external infrastructure you hope never has an outage or is deprecated. This isn't to say that I have no Wi-Fi smart devices on my network, but the ones I do are devices that I would give network access for other purposes, such controls for accessing my smart TV, which is on the network already for accessing streaming services. The extra smart functionality they offer isn't their main purpose, but is an added bonus made available to them being on the network.

If not Wi-Fi, then what do I use instead? There are a handful of two-way, local, wireless communication protocols widely supported today. Two of these, Zigbee and Z-Wave, offer very similar features and functionality. They both operate from a local device -- either a smart hub or in my case, a USB dongle. These have no access to the open internet and thus don't require communication with an external server to work. They also can bounce signals between compatible devices to more remote ones. This is very useful for smart devices located on the other side of the home as the hub, beyond the reach of where antenna alone might be able to reach. There are some differences between the two protocols though. Zigbee operates on the 2.4 GHz band, while Z-Wave is around 800 MHz. This means that Zigbee can potentially interfere with your 2.4 GHz Wi-Fi network, if you have one. I've also been told that Zigbee devices tend to be cheaper, but Z-Wave devices have a more rigid specification, meaning there's less compatibility issues than with Zigbee. I've gone with Zigbee for my home, mostly because there seems to be a much larger range of devices that use it compared to Z-Wave. I should also mention a third protocol, Matter, which is backed by many of the major smart home players. This protocol has been touted as being a potential unified protocol for all devices, with all the benefits of the others, but at the time of writing this, very few devices seem to support it.

Sonoff Zigbee Dongle, plugged into my server

I mentioned that I use Zigbee devices for two-way communication devices, but there are some items that only require a single direction of information. These are things like weather or temperature sensors, which need to provide their measurements to the home interface, but don't need to (and can't) receive any commands from the hub itself. I'll discuss more how I came to this setup, but it turns out that a large number of these sensors all operate on a standard 433 MHz frequency. There is a small community around reverse engineering and parsing the packets sent my these devices, and so if you equip yourself with a radio antenna-to-USB interface, you can pick up any sensor data broadcast in the general area. This is honestly the most hands-off of all the different smart devices I incorporate.

The Brains of the Operation

So, equipped with the idea that I want to prefer local protocols like Zigbee or the 433 MHz radio transmissions, we now need some way to control and receive all the devices around the home. Enter Home Assistant. Home Assistant is a community-developed software suite that can interface with a wide variety of different device manufacturers, which they refer to as "integrations". These integrations can be controlled through a customizable dashboard equipped with standard interfaces such as buttons, graphs, gauges, etc. It strives to be the open source alternative to your typical smart home hub, but not locked into devices from any single ecosystem. Much like how those hubs must be run continuously somewhere in the home, Home Assistant must too live and run somewhere around your household. I personally run it on my home server (read: a computer in my office closet that I never turn off), and the only other hardware attached to it is that which I previously mentioned -- a USB dongle for Zigbee support and the 433 MHz radio antenna.

With each new device from a new manufacturer, I first find and add its respective integration, then pair the device itself. Once connected, you can add UI controls to your personal dashboard, which can be further organized into different tabs. For example, we purchased a robotic vacuum cleaner several years ago, one of the few devices connected to our Wi-Fi network. By installing its integration and connecting it to Home Assistant, a UI "card" can be added to a dashboard that offers various controls, such as starting and stopping a vacuum cleaning cycle.

The story is a little different for Zigbee devices. Despite owning devices from different manufacturers, all of my Zigbee compatible devices, which make up the majority of my smart items, are all controlled via the same integration. There are actually two options here, both of which I've used in the past. The previous service I used is the Zigbee Home Automation (ZHA) which is a Home Assistant integration. This allows for the pairing and management of all your Zigbee devices directly from within Home Assistant. You may wonder why you'd ever want to manage your Zigbee devices outside of Home Assistant, but I've done just that by switching to Zigbee2MQTT, a separate service that also runs on my home server.

Zigbee2MQTT does exactly what its name implies, converts Zigbee communications to and from the MQTT protocol. MQTT, if you aren't familiar, is a protocol that answers the question "how do I send a message between two of my computers?". Devices connect to a central MQTT server (or "broker", as they call it), which again my home server serves as. They can then subscribe to one or more "topics". When other devices publish a message, they do so under a specific topic, and other clients will only parse the included payloads of topics they're interested in.

MQTT is a cornerstone technology for my smart home. It allows any device, even running software I've written myself, to send or receive messages from other devices, which greatly expands the amount of interconnectivity. If you're going to be living solely inside of Home Assistant, this extra layer of translating Zigbee communications to MQTT is overkill, but if you wish to incorporate devices that aren't supported by Home Assistant or Zigbee, this opens the door.

Perhaps not surprising, I use MQTT as the method to connect my third class of devices, those communicating over the 433 MHz band. As I mentioned before, this protocol is used commonly by one-way devices, such as my balcony weather gauge, or a number of battery operated temperature sensors I have placed around the home. I use a program called rtl_433 which can decode the signals sent by a number of common consumer devices. One of the options it supports is to export those decoded signals to an MQTT server, which is how I connect those devices into my larger ecosystem.

As a bit of background, I began investigating this solution after receiving my outdoor weather station as a Christmas gift. As is common, it came with a small LCD screen which displayed the measurements taken by the device. This is enough for most people, but by this point I had a Home Assistant setup, and wanted to forward that information so I didn't have to walk to a specific shelf to read the weather. The officially supported solution is to connect the LCD display to your Wi-Fi network, provide it with an API key to a service such as Weather Underground, and then connect Weather Underground to Home Assistant to fetch that data. I actually used this method for sometime, but was already frustrated with it. Not only because it's comically overcomplicated, but it was rather flaky to boot. I had begun to be resigned to my fate when I learned of these projects and decided to try it out for myself. All it required was the purchase of a standard antenna with USB connection, and with the installation of the software I was able to cut out the middleman entirely. Not only do I not even need the LCD screen anymore, but the web API limited updates (understandably) to about once every 15 minutes. It turns out the sensor actually broadcasts updated about once every 20 seconds, meaning I was now getting much more granular measurements than before.

We've already covered four services running on the server, how about a fifth? The last piece of software I use is called Node Red. Node Red provides some of the glue that connects the different devices together. Want to turn some lights on at a specific time of day? Node Red can provide that. Different devices are represented as different nodes, and by piping nodes from one to another, data can be converted as required by the eventual recipient. A lot of what Node Red does could probably be done within Home Assistant, but I find its interface much more intuitive to use (although I'm told Home Assistant's has improvement since I last tried it). Here is where I handle the most complicated connections, things that can't be handled with a simple UI button. This includes triggering lights at certain times of day, controlling large groups of objects, utilizing motion sensors, and perhaps the most surprisingly complex of all, providing functionality to an iOS shortcut button.

To summarize, my home server is running five different services which together connect all the individual devices together in a single place.

These together create the backbone of my smart home. It might seem a bit complicated, and it probably is compared to the vast majority of setups, but it allows devices to work together with the smallest amount of friction. Rather than deal with the app and interface of every single manufacturer, everything is channeled into these few places, where everything can be handled and controlled in the same way.

The Devices Themselves

This is all well and good, but what exactly do I do with all this software? It doesn't do me much good to have all this set up if I don't actually have anything to control. We'll begin with what everyone begins with -- lighting. Perhaps the most bang for your buck smart device are those which control the lights, and I too am no stranger to this. However, after some iteration in my own home, I've completely removed any smart light bulbs from this setup. They're one of the most commonly found appliances, and are among the easiest to install -- after all, pretty much anyone can screw in a light bulb -- but I find them among the least useful to use. If you're setting up your own home and looking for tips, I hope this piece of advice is the one thing you take away from this article: always have a manual backup to everything. The problem with smart light bulbs is that the socket they're screwed into must always be receiving power for them to work. If you wander into the kitchen in the middle of the night without your smart phone, you're out of luck. You have to resort to flicking the switch anyway which on some bulbs will illuminate them, but also cause them to lose their connection. Instead, I recommend looking into controllable plugs with manual buttons. You can instead plug lamps or other devices into them, they're just as easy to set up as a light bulb, and if even if you don't have your phone, you can just reach down and press the button to provide power.

I've taken this a step further and replaced a good number of the light switches in my home with smart ones. This only requires a basic understanding of your home wiring and safety, but is still orders of magnitude more complex than screwing in a light bulb, so I understand it's not for everyone. However, by replacing the light switches themselves, it's the most seamless way to control the lighting in a room. Many guests (and landlords) don't even notice that the switches have been replaced. You receive all the benefits of a traditional switch with the extra benefits of remote control. I've also learned that nothing makes people happier than being able to control a ceiling fan without getting up.

All of the outlet plugs and smart switches are controlled over Zigbee, and consist of the vast majority of the devices I have around. The only other Zigbee devices I have are an Ikea motion sensor -- which, via Node Red, I use to control the lights in my office -- and a lone temperature sensor I was given. Most of my temperature sensors are instead communicating over 433 MHz radio signals. I've mentioned the outdoor weather sensor, which provides temperature, humidity, and wind speed (or it did, the most recent winter did a number on it). Scattered around the house are some battery operated sensors which only provide temperature and humidity. I don't do anything with this data except display it, although it would be possible to use their inputs to control humidifiers or thermostats if I had one.

Next are the devices which are already connected via Wi-Fi, which I figured I might as well connect to Home Assistant since they're there. We use some HDMI connected Roku boxes to give access to streaming services, and their integration allows Home Assistant to emulate a remote control, as well as displaying the currently playing content. I mentioned the robot vacuum before, and it too can be controlled via Home Assistant. It's missing some of the more advance functionality available in its own app, but we never used any of that to begin with. I also have a Raspberry Pi running Octoprint to help control my 3D printer, and when running it updates Home Assistant with the elapsed and estimated times, although it just displays invalid information most of the time.

Conclusion

That, in an overly verbose nutshell, is how I've configured my smart home. Compared to some people, it might seem a little barebones, but I think we have a pretty high density of controllable devices per square foot. It's taken a few years of experimenting with different services and devices to get here, but I feel the infrastructure I have now is robust enough to handle any new devices. Perhaps most importantly, it guides any new purchasing decisions I want to make towards how well I know they'll work with everything else in the ecosystem, which in the end yields a better experience when I do make a decision. Hopefully this has been marginally useful or interesting, and can be helpful for those of you who like the idea of a smart home, but are frustrated dealing with a loose collection of uncooperative devices.