Voice controlling things with a smartphone and Raspberry Pi

I am not a big fan of voice control, but this particular project turned out quite practical. What I wanted to achieve is to pick up my phone, speak out a command and so control connected things like my TV, heat pump and so forth. I decided on using my phone so that I could speak out the command from anywhere, anytime.

To achieve this I opted to use Android’s in-build voice actions, in particular the ‘note to self’ function. When you speak ‘note to self’ followed by some message, Android will send you an email (from yourself) with subject ‘Note to self’ and transcribed message as body. Audio file is also available as attachment. What I need is to watch out for such messages, parse them out and take action. IMAP is the way to go for push email notifications, so I enabled it for my Gmail account. I have set up a filter that will put all messages from myself and with subject ‘Note to self’ to a folder called ‘Voice’.

I used Chris Kirkham’s python IMAP push notifications script to start with and modified it to call my own notify script with the email message body as parameter. My version is available on GitHub. Basically the script will log in to my GMail account and keep an eye on the folder ‘Voice’. If new message comes in there, it will start another script, ‘notify.py‘ with email’s body as parameter, that will do the rest of the work. Using IMAP ensures immediate push message, there is only 3-4 seconds delay between speaking out the voice command and the script actually picking it up.

I have the script run in the background by running:

nohup >/dev/null 2>&1 /home/pi/IMAPPush.py myemailaddress@gmail.com mypassword &

The notifications script can be extended as needed, I now have it catch the following voice commands:

‘TV off‘ voice command switches my TV off. I use that when we are in the kitchen for example and I am too lazy to go to the living room and switch off the TV. It uses the LG python remote control script that I described in this post.

‘AC on/off, set temperature’ to control my Daikin heat pump with the RESTful API that I have on my DIY Daikin controller.

Air conditioner OFF

‘Internet deny,allow‘ to deny/allow Internet access for my teenager son. We have agreement to only use Internet after all homework is done, the script now ssh-es to my DD-WRT router and allows/denies his access (blocked by MAC address) whenever I speak out the command.

Another fun feature that I have is a universal Google Spreadsheet lifelogger using the gspread python library and based on Adafruit’s code. The script kicks in when I speak a command that looks like  ‘log dimension1 dimension2 value’ for example:

Log Martin weight 92.5
Log Toyota mileage 55444
Log blood sugar 123.45
Log blood pressure 345.6

..and so forth whatever you can think of into a Google Spreadsheet called ‘Data’. It is auto timestamped for you:

This way one can collect any sort of data simply by speaking it out. Using the top filters, you will be able to capture only dimensions of interest for further analysis, run pivot reports and do graphs.

I’ll probably add some sort of confirmation email once a voice command is accepted and executed, it can easily be linked to Facebook, Twitter or many other social networks using IFTT.

It is practically quite easy from here on, you can turn on/off relays, request sensor state with voice and so forth.