REXO ; Interactive Art Machine using Sound s— MuW-201
INTRODUCTION :
I created an Interactive Art Machine, which creates digital artwork with various sounds.
I have used Arduino and connected it to Processing using certain libraries. In this project, I used the buttons as a palette, the sound as the paints, and the final digital art as the canvas!
It can be customized to everyone’s interests. For this project, I decided to work with Processing, Arduino, Python(optional), and open processing.
MATERIALS REQUIRED:
- MDF wood
- White opaque Acrylic sheet
- 11x 16mm red(24V): Pushbuttons
- Connecting wires
- Arduino Uno
- Laptop
SOFTWARES REQUIRED:
- Python 3.9
- Open processing
- Arduino IDE
SKETCHES:-
STEP #1: Design the structure you desire on illustrator, for my project I used a dodecahedron. Being a perfectionist, this shape fascinates me a lot because of its structure and complexity. These pieces are then cut using a laser cutter.
Here, my structure has 12 faces, so I cut out 12 MDF faces and 11 Acrylic faces- because one face is the base. A hole is cut into the wood pieces, that's where the button goes. Circles with holes are cut as well- these serve as an elevation for the structure to sit on.
STEP #2: Assemble the parts together. This includes building the dodecahedron, fitting in the push buttons through the voids created on each face of the structure, soldering the buttons, and sticking the opaque acrylic on top of the buttons.
STEP #3: Built a box online using the website: https://makerdesignlab.com/tutorials-tips/online-file-generators-for-laser-cutting/#boxes. This is where I stored the Arduino board and wires.
STEP #4: Connected each button to various sounds. I did this via a simple python code- where Serial and Playsound were imported. This enabled the sounds to each button.
import serial
from playsound import playsound
ser = serial.Serial('COM5')
ser.flushInput()try:
while True:
ser_bytes = ser.readline()
a=int(ser_bytes.decode('utf-8'))if a == 2:
playsound('2.mp3')
elif a == 3:
playsound('3.mp3')
elif a == 4:
playsound('4.mp3')
elif a == 5:
playsound('5.mp3')
elif a == 6:
playsound('6.mp3')
elif a == 7:
playsound('7.mp3')
elif a == 8:
playsound('8.mp3')
elif a == 9:
playsound('9.mp3')
elif a == 10:
playsound('10.mp3')
elif a == 11:
playsound('11.mp3')
elif a == 12:
playsound('12.mp3')except KeyboardInterrupt:
print("Keyboard Interrupt")
Add in your mp3 files in the same files as the python file. Add in the mp3 files into the above code and run the module.
STEP #5: I wanted to link these sounds to a digital drawing; something generative and personal. Down below are a few of the prototypes I created using open processing (p5js & pjs). The Arduino was connected to processing.
This one (below)was one of my favorite prototypes; it drew 30 circles when an even-numbered button was pressed and draws 30 squares when an odd-numbered button is pressed. The shapes were of random sizes and stroke colors with the circles being black and squares being white.
STEP #6: Finally I decided to create a sound visualization as my final output art. This visualization was inspired by my inclination towards symmetry -my P1 project was completely based on a radial line of symmetry and hyperrealistic drawing. Dont forget to add the p5.sound Library
The Rorschach test is a psychological test in which inkblots are recorded and then analyzed using psychological interpretation. I used this as the base for my sound visualization.
This shows how each drawing can have different abstract meanings based on each individual.
var octaves = 4
var fftSmooth = 0.95
var frameSize = 512
var startMhz = 150
var endMhz = 12000
var beatThreshold = 0.4
var bpm = 120var headRadius = 80
var wingSize = 1.3var wingTransparency = .02//0.035
var wingSaturationMultiplier = 0.8
var wingBrightness = bnorm(.5)
var dotSize = 12var strokeTransparency = 0.01
var strokeBaseBrightness = bnorm(.6)var audio, fft, beat
function bnorm(floatVal, low=0, high=255){return Math.round(floatVal*(high - low))}function setup() {
createCanvas(windowWidth, windowHeight)
colorMode(HSB)
background(0)
audio = new p5.AudioIn()
audio.start()fft = new p5.FFT(fftSmooth, frameSize)
fft.setInput(audio)
beat = new p5.PeakDetect(startMhz,endMhz, beatThreshold, 60/(bpm/60))
}function draw() {
let spectrum = fft.analyze()
let scaledSpectrum = splitOctaves(spectrum, octaves)
let volume = max(scaledSpectrum)
beat.update(fft)
let xCenter=width/2, yCenter=height/2
translate(xCenter,yCenter)
{ beginShape()
let hue = map(fft.getCentroid(), startMhz,endMhz, 0,359)
fill(hue, volume*wingSaturationMultiplier, wingBrightness, wingTransparency)
stroke(hue, volume, strokeBaseBrightness - volume/2, strokeTransparency)
strokeWeight(1)
//curveVertex(0,0)
let N = scaledSpectrum.length
let mirrorCopy = Array(N)
for (let i=0; i < N; i++) {
let R = headRadius + wingSize * scaledSpectrum[i]
let x = R * sin(radians(i*180/N +180))
let y = R * cos(radians(i*180/N +180))
drawDot(x,y, i)
mirrorCopy[N-1-i] = [Math.abs(x),y]
}
mirrorCopy.map(([x,y], i)=> drawDot(x,y, N-1-i))
function drawDot(x,y, i){
if( beat.isDetected ){
fill(hue, volume*wingSaturationMultiplier, wingBrightness, bnorm(0.8))
}
strokeWeight(map(scaledSpectrum[i], 0,bnorm(1), 0,dotSize+4))
line(0,0, x,y)
strokeWeight(1)
/*push()
fill(hue, volume*wingSaturationMultiplier, wingBrightness, 0.4)
stroke(hue, volume, strokeBaseBrightness - volume/2, 0.4)
ellipse(x,y, dotSize)
pop()*/
curveVertex(x,y)
}
stroke(hue, volume, strokeBaseBrightness - volume/2, 0.4)
strokeWeight(1)
endShape(CLOSE)
}
}let play = true
function mouseClicked(){
play = !play
if(play) loop(); else noLoop()
}function splitOctaves(spectrum, slicesPerOctave) {
var scaledSpectrum = [];
var len = spectrum.length;var n = slicesPerOctave|| 3;
var nthRootOfTwo = Math.pow(2, 1/n);var lowestBin = slicesPerOctave;var binIndex = len - 1;
var i = binIndex;while (i > lowestBin) {
var nextBinIndex = round( binIndex/nthRootOfTwo );if (nextBinIndex === 1) return;var total = 0;
var numBins = 0;for (i = binIndex; i > nextBinIndex; i--) {
total += spectrum[i];
numBins++;
}var energy = total/numBins;
scaledSpectrum.push(energy);binIndex = nextBinIndex;
}for (var j = i; j > 0; j--) {
scaledSpectrum.push(spectrum[j]);
}return scaledSpectrum;
}function smoothPoint(spectrum, index, numberOfNeighbors) {var neighbors = numberOfNeighbors || 2;
var len = spectrum.length;var val = 0;var indexMinusNeighbors = index - neighbors;
var smoothedPoints = 0;for (var i = indexMinusNeighbors; i < (index+neighbors) && i < len; i++) {
if (typeof(spectrum[i]) !== 'undefined') {
val += spectrum[i];
smoothedPoints++;
}
}val = val/smoothedPoints;return val;
}
FINAL WORKING PROJECT: Here, I present you to REXO!
CONCLUSION
Rexo uses a lot of features and software at the same time, during this phase of building the project. I would like to cancel out python from this project as it only enables the sound and could be avoided if I get the mp3 player component. Arduino has a part that allows us to play the music from REXO itself. Unfortunately, I couldn’t find the piece to arrive at the time in order to finish the project- so hence had to find other ways to play the music. I learnt a lot from this project as it included various new methods which I was completely not used to. I watched a lot of YouTube videos that could guide me through the process and read a few articles. (everything is referenced towards the end)
For the future, I would like to expand the horizon of REXO. Instead of buttons, I want to upgrade to screens that can detect and add more points of interactions. I would also like to add a printing option so that the individual can take home the piece of art they created.
I want my project to evolve in a way that it is personalized, interactive as well as create a communication between two or more individuals. For example: — if more than a person jointly accesses REXO, it exhibits a series of new/weird/fun behaviors. This could create a sense of connection between humans and provoke them to go through the experience with a partner or a group of other people. I want it to be more interactive among human beings too.
REFERENCES:-
https://makezine.com/projects/make-35/advanced-arduino-sound-synthesis/ (ARTICLE)
https://www.youtube.com/watch?v=pGHJj0kxVK8
https://www.youtube.com/watch?v=_4cIv6qLaM8
https://www.youtube.com/watch?v=sqQzIN7G6Oc
https://mrelectrouino.blogspot.com/2019/08/arduino-piano-diy-conductive-paint.html
https://forum.arduino.cc/index.php?topic=135706.0
https://processing.org/discourse/beta/num_1270835482.html
https://processing.org/reference/libraries/serial/Serial.html