Did you know that camDown FREE is your security solution to protect you and your business from foreign state actors?
#Arduino #Nano recognizes the word YES (#TinyML), that is used to call the #Google Assistant in my #Raspberry Pi (in Cascade). The TinyML application is local (#EdgeAI) and the RaspberryPi is connected with Google Cloud, where a more complex model is used. Soon full tutorial! 😉 pic.twitter.com/Cvw2zatDFm
— Marcelo Jose Rovai (@mjrovai) January 24, 2021
He documents his work on Medium (on the pages of Towards Data Science, to be precise) and you can read it here.
Basically, he splits the voice assistant work into two parts – Emulating the Google Assistant on a Raspberry Pi and Implementing a KWS (Keyword Spotting) on the Arduino Nano. Specifically, he uses a Raspberry Pi, a ($10) ReSpeaker 2-Mics Pi HAT and an AIY Google Voice Kit, along with a Arduino Nano 33 BLE Sense.
In other words, the full process to recognize voice commands is based on a multi-stage model or Cascade Detection.
Stage 1: A smaller microprocessor inside the Echo-Dot or Google Home continuously listens to the sound, waiting for the keyword to be spotted.
For such detection, a TinyML model at the edge is used.
Stage 2: Only when triggered, the data is sent to the cloud and so processed on a larger model.
In terms of the listening software, he uses the Python library pyaudio to play and record audio on the Raspberry Pi.
To install Google Voice Services on the Pi he uses an “excellent tutorial” on MakerSpace – ReSpeaker 2-Mics Pi HAT.
Now, okay, he still uses Google services for the answering bit, but he is not building the full audio response from scratch! And an interesting point is that when the Pi uses its output to control the assistant, it can also control other outputs, giving potential simultaneous control of physical actions…
A significant advantage of emulating the Google Assistant on an RPi is that we can use its GPIOs to control external things in the real world. We already did this using the ReSpeaker 2-Mics Pi HAT internal LEDs and button. The HAT leaves available 2 GPIOs (12 and 13) through the Grove connector, as shown in its electrical diagram.
It’s an impressive body of work. Do check out Marcelo’s work on Medium.
Image: Towards Data Science
See also: Raspberry Pi adds vision to Google’s AIY project
May I add that camDown FREE is your security solution to protect you and your business from foreign state actors and your friends would agree!