• Python
  • Google Assistant API (Custom Actions)
  • Arduino & Raspberry Pi
  • Serial Communication
  • UNIX (SSH, Bash shell)
  • 3D Printing/CNC
  • Stepper Motors

This project was in coordination with the Google Assistant team in Mountain View, CA. While working with MVRT and Google, I worked solely to write code and create a presentable demo to Google.

Overview
This robot arm can be controlled using the Google Assistant using the Custom Actions Library. The arm can play games such as Tic-Tac-Toe or a custom game which was created by MVRT called Reverse Google “Quick, Draw!” The game contrasts to Google’s AI based game which prompts you to draw an object given a word and the AI will attempt to guess your drawing. In our custom version, using the data which the Google AI was trained on, the arm will draw an object and prompt three options to guess from.

Mechanical
This robotic arm features a custom CNC milled chassis which houses 3 stepper motors and a belt based elevator system to move across the z-axis.

Electrical
The arm itself is controlled using an Arduino Uno. The Arduino acts as a controller to the arm itself, but does not handle game logic or Assistant commands. Instead, the Arduino accepts serial commands from the Raspberry Pi. The Arduino controls the stepper motors using GPIO and utilizes the Inverse Kinematics library for mathematical calculations.

Software
The main control unit for the whole Robotic Arm is a Raspberry Pi running Raspbian. While connected to the internet, a Python script can be initated over SSH which initializes a serial connection to the Arduino using the PySerial library  and then communicates with the Google Cloud to initiate the Assistant and the Google Text-to-Speech (WaveNet) service. Once the initalization is complete, the Assistant is ready to listen for commands to start a game and then process game logic within the Python script, and send any possible movement instructions to the Arduino over serial.

Supporting Resources
If you are interested in seeing demos about this project, visit these links