Interactive Design
Artistic Documentation
In this module, we made interactive designs with our ESP32s, along with a joystick and a button.
The Game
I made a game called "Steal the Color". Hidden in the display is a rectangle. Inside this rectangle, you can change color. Your goal is to navigate around the display using the joystick, find the invisble rectangle, and click the joystick to change color! For added fun, hold the button to cycle through all the colors (and evade the rectangle altogether.)
The Enclosure
The breadboard is encased in cardboard, and decorated with black construction paper. The shape of the joystick and button was awkward, so I cut holes in the cardboard and cut a piece of a pen to press the button. I added three spots of color to mimic the aesthetic of my display. One feature of the enclosure that I didn't expect but felt suited my project was the side I left open for the USBC cable. When the device is plugged in, a bluish light can be seen from within the black construction paper. It reminds me of the hidden rectangles, and adds to the ambiance of my controller.
The Process
Once I began exploring PyGames, the python library I used to make the display, the possibilies for the serial inputs were limitless. I knew I wanted something with color and with motion. I wasn't sure if I wanted a more serious art installation or more of a game, so "Steal the Color" actually had a more serious, artistic origin. The idea was that we, as people move around blindly. We find ourselves trapped in ways we cannot explain, and the only way to escape is to try something different-- like changing color. As I implemented this, I found myself having fun trying to find the rectangle that traps the circle. I ran with that lighthearted feeling, and ended up with a more playful project.
One difficulty in coding this project was how to maintain the state of "stuck", as the code loops repeatedly. Eventually I decided on a global variable called "inside", which, when true, would adjust the position of the circle.
To try this yourself, follow the below technical description, or follow this link to my code repository.
Technical Documentaion
Link to github repository: https://github.com/susannaha8/CS3930-Steal-the-Color
Setting up the Hardware
This project consists of a breadboard, 7 wires, and three main pieces of hardware: an ESP32, a joystick, and a button. Prepare the wires by stripping a bit of the plastic off each end. Plug the pins of the hardware into the breadboard, and connected them further with wires: connect the joystick to the ESP32 with one wire to Ground, one to 3V, and the other three to pins 25, 26, and 27 for X, Y, and Z motion; connect the button with one wire to Ground, and the other to pin 13 for digital input. The ESP32 is powered via a USBC cable.
Flashing Code to the ESP32
After setting up our hardware, flash the code found in `src/main.cpp` to your ESP32. This allows you to recieve serial input from the joystick and the button. In the code, change the pin numbers if you used different GPIO pins when setting up the hardware. You can either write this code in the Arduino application or through the VSCode IDE PlatformIO. Select the port, and set the baudrate to match whatever you value you put in your `src/main.cpp` code (ex. `Serial.begin(115200)`.) To set the baudrate in PlatformIO, in `platformio.ini`, add the line `monitor_speed = 115200`, where 115200 is the baudrate. To check that you are recieving serial input, open the Serial Moniter. As you move the joystick and press the button, you should watch the values change.
Reading Serial Input
Once you confirm you are recieving serial input, you need to retrieve that input so you can parse it. This is done in `reader.py`. First install the python module serial with `pip install pyserial`. Use the line `ser = serial.Serial('/dev/cu.usbserial-546F1153241', 115200)` to tell serial the port and baudrate. `str(ser.readline().strip(), 'ascii')` returns a string of the serial input you are recieving from your hardware; this line is inside an infinite while loop, so you are continuously recieving lines of input. From there, you can use those values as variables that change color, or position, or any other features of your display.
Creating My Display with PyGame
PyGame is a helpful Python library that allows for the display of graphics in games. PyGame's documentation has helpful examples, and provide good explanations for various functions you can use to create simple games:
I built a game that has a moving circle (controlled by the joystick) that gets trapped in an invisble square, and to get out of the square, one must press the button on the joystick. This also has the effect of randomly changing the color of the circle. The button acts as a free pass, changing the color of the cricle continuously, and circumventing the square of doom. Here is a picture of 'Steal the Color' in action: