Jun 20, 2024 8 min read

How many technologies does it take to open your blinds?

Art by Clelia Rella

Welcome to the June edition of the Open Home Foundation newsletter, the place to learn about the latest and greatest things for your smart home that improve its privacy, choice, and sustainability.

The Open Home newsletter is written by Paulus Schoutsen - President of the Open Home Foundation, founder of Home Assistant, and Nabu Casa. Was this email forwarded to you? Subscribe here!

All of a sudden it's the end of June, summer holidays are about to start, kids are off, and AI is all the rage. This month, we wrote about our approach to AI and why we are focusing on enabling experimentation with cloud and local LLMs (more on that below).

I have been experimenting with AI in my home, and it's been a lot of fun! Below is a video where, after a shower, I tell Assist that I'm coming out of the bathroom and ask if it can prepare my room. It closes my blinds and turns on my light.

Assist controlling my MotionBlinds blinds using ChatGPT

On the outside, it seems simple: ask for something, and the home responds. But what you are actually seeing is the next evolution of the smart home. Where multiple technologies work together seamlessly and local data is being harnessed to intelligently understand the user's needs.

We’re experimenting with the smart home of the future today. Sure, you may see something similar in product demos from big tech, but theirs could be years off (or may never happen), ours is here today.

So how does this work behind the scenes?

It starts with a voice command - My voice experience is powered by Assist, Home Assistant's built-in modular voice assistant. I leverage Home Assistant Cloud for Speech-to-Text and Text-to-Speech, which I connected to ChatGPT with our new feature that allows AI to control Home Assistant. In the video, I use our app to quickly open the Assist dialog using a widget. Assist works on many different devices, but widgets are one of the newer ways, and are available on Android, WearOS, and iPhone, while Apple Watch support is coming soon.

How Home Assistant controls my blinds using Matter over Thread.

Next, devices connect over open standards - The blinds are Motionblinds and can be controlled via the Matter protocol sent over the Thread radio standard. I use a Home Assistant Connect ZBT-1 (formerly SkyConnect) as a Thread Border Router to allow these devices to talk to Home Assistant. My hub is not close enough to my bedroom to allow a stable Thread connection, so I'm leveraging the protocol's ability to use multiple routers. Connecting my blinds and lights to a HomePod Mini, which is also a Thread Border Router, which connects to Home Assistant.

Finally, the devices take action - My Motionblinds close, lights turn on, and my neighbors don't have to see me in a towel. In this one example, you have Home Assistant connecting seven different devices and services. To the end user, it feels like one, which is the power of interconnectivity, open standards, and a platform built around choice.

Technology is awesome and we're coming to a point where all of this more or less works. There is room for improvement, especially around getting Thread credentials shared between products from different vendors, and the AI I used still runs in the cloud. It's here today to experiment with and gives a great look at what is possible, where the smart home is headed, and what experiences we need to polish.

If you want to see what's new for Assist, check out our upcoming Voice - Chapter 7 livestream on June 26

ESPHome is growing up

We did a fun live stream with the ESPHome team to talk about the latest features, a new approach to version updates, and the specs of our upcoming voice hardware.

One feature that should be a game-changer is the improvements we made to Over-the-Air (OTA) updates. This allows makers to provide firmware for the projects they share or sell, and users can keep these devices up-to-date without needing to use the ESPHome dashboard or build the firmware themselves. This will open the project up to so many more novice users.

Check out the stream as 2024.6.0 this is a big update. It recaps a lot of the recent developments bringing new functionality for power-user, while making ESPHome more approachable and easy to use. Also worth checking out is the recent update to the ESPHome logo and website - https://esphome.io/

Raspberry Pi lists on the London Stock Exchange

On June 11, our beloved Raspberry Pi went public and I have mixed feelings about this. On one hand, Raspberry Pi has been making and delivering great products with excellent documentation and support for older models. They have said that access to the public market will enable them to build more of the products we love, faster. On the other hand, they now have a lot of shareholders who will eventually want a return on their investment.

A photo of people celebrating surrounded by confetti with a dark cube in the middle of the crowd with the text "Market Open". — The London Stock Exchange when it listed Raspberry Pi Holdings.

This is a balancing act that will constantly pull on leadership moving forward. This could go well for a long time, especially as Eben, the founder of Raspberry Pi, remains in charge. But what about the next CEO?

The best way to deal with such a balancing act is not to start it. That's why Home Assistant is owned by the Open Home Foundation. On top of that, Nabu Casa - the commercial partner of the OHF that funds development - has no investors or debt. We are our own bosses and only answer to our users.

A Roadmap for Home Assistant

Recently Madelena, who leads the Product & Design team for Home Assistant, released two important blogs. The first explains why and how we built our roadmaps, and the other details our first roadmap. Our roadmap is not an exhaustive list of features and dates, but a strategy and direction for the continuous development of the Home Assistant project.

It has taken months of research, which included listening to the community (yes we read all the comments) and talking to users—specifically, not just the users who admin Home Assistant but the people who actually live in these homes. We are using this to identify problems and gaps, and then setting goals to address them. With the ultimate goal of Home Assistant being the best smart home platform.

[Home Assistant Roadmap 2024 Midyear Update — There are also roadmaps for Cloud and open standards, along with more on our research.

These roadmaps are intended to bring the community together and focus it on solving our common problems. Our Year of the Voice brought in so many new experts in the field to support Assist development. These roadmaps will have a similar broader effect on driving forward our open home vision - we want you to help shape our roadmap and get involved today.

Our vision for AI

Connected to these roadmaps was laying out how we see AI in the home. My above example of using ChatGPT to control my blinds is not for everyone. Yes, there is a lot of hype everywhere for AI, but there are real benefits it can bring to the home. We believe Home Assistant is uniquely positioned to be the smart home platform for AI. Local data, user control and powerful customization allows us to do things not possible with any of the other big platforms.

It is still too early for mass adoption, but we want to give users the choice to experiment with this technology today. In our 2024.6 release, we brought new features that will let users experiment with AI controlling their homes, starting with OpenAI or Google AI.

We've also been working with NVIDIA to help develop a local AI solution, as an alternative to cloud-based LLMs that are currently expensive and not ideal from a privacy perspective. NVIDIA has made amazing progress already, and their research on using multiple AI agents is really worth reading about.

Home Assistant 2024.6

2024.6: Dipping our toes in the world of AI using LLMs

This release brought some big developments for AI in the home... and I promise I'll stop talking about AI now! It also included some nice quality-of-life improvements:

For Assist we improved media player commands, allowing you to play, pause, skip tracks, and set the volume with voice commands.
Dashboards gained the ability to show sections conditionally, for instance you can hide sections from specific users or on certain devices. This also works for cards with the new Visibility tab.
You can now set a background 🖼️ for your dashboards in the UI.
And much more!

Community highlights

DIY smart speaker

r/ekognaG has made a 3D-printed voice speaker for music and voice assist via ESPHome. It has an S3 inside, along with two pretty big speakers, a volume knob, and mics. I think a little speaker fabric would complete the look 😉.

Try before you build ESPHome devices

Take the guesswork out of building a new device, and virtually simulate your smart home creations. @the-mentor has built an ESPHome device simulator to help simplify development.

The droid your looking for

r/Realistic-Ad1877 has built a ESPHome-powered Lo-La59 Star Wars Droid, and this modified Disney Shoulder Pal lights up and makes noises. When it's in the home its noises and lights can be automated.

In other news

Sonos removed the sentence "Sonos does not and will not sell personal data about our customers." from their privacy statement. My relationship status with Sonos has changed to "it's complicated". (Louis Rossmann, YouTube)
Alexa's AI-based version is in trouble. Great deep dive on Amazon's struggle to create an LLM-powered version – despite 10,000 employees working on Alexa. (Sharon Goldman, Fortune)
Feature purge continues as Alexa removes shopping list integrations. By the time Alexa runs on LLMs, will there be any features left to support? (Jennifer Pattison Tuohy, The Verge)

Forward to a friend, sharing is caring.

Anything else? Hit reply to send us feedback or say hello. We read everything!