A Stress-Reducing Explorable Musical Object by Hannah Rhiannon Lienhard Submitted to the Department of Mechanical Engineering in partial fulfillment of the requirements for the degree of Bachelor of Science in Mechanical Engineering at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2019 @ Massachusetts Institute of Technology 2019. All rights reserved. Signature redacted A u th or ...................... ............ Department of Mechanical Engineering May 10, 2019 Signature redacted Certified by... U ... Tod Machover Muriel R. Cooper Professor of Music and Media Thesis Supervisor Signature redacted A ccep ted by ........................................................... MASSACHUSETTS INSTITUTE Maria OF YangTECHNOLOGYX.. Associate Professor of Mechanical Engineering JUL 16 2019 Undergraduate Officer LIBRARIES -j A Stress-Reducing Explorable Musical Object by Hannah Rhiannon Lienhard Submitted to the Department of Mechanical Engineering on May 10, 2019, in partial fulfillment of the requirements for the degree of Bachelor of Science in Mechanical Engineering Abstract In this thesis, an explorable musical object - which is meant to help reduce anxiety and stress in the user - was designed and built. Using the program Pure Data, output data from an Arduino was manipulated to create varied tones when several pressure, bending, and soft potentiometer sensors were activated. These sensors were embedded in a soft, spherical outer shell made of roto-cast silicone. The final object was fully contained within this outer shell, as a Raspberry Pi computer was used to run Pure Data so it did not have to be plugged in. The sound created was output through a radio transmitter connected to a radio and headphones, again so the instrument could be fully contained within its outer shell. Thesis Supervisor: Tod Machover Title: Muriel R. Cooper Professor of Music and Media 3 Acknowledgments I would like to thank Nicole L'Huillier for her guidance and never-ending support throughout this project and the rest of my time working in the lab. I would also like to thank Tod Machover for his advice and support, and for giving his time to help me make this project a reality. Additional thanks to Graham Yeager, for helping me figure out casting, all the res- idents of the 33rd, for being available to squish things upon request, and for being enthusiastic about my project regardless of the current state, and to Ash Demir and Lauren Schexnayder for being the most supportive witches I've ever met. Finally, I want to thank Jesse Michel, for keeping me optimistic throughout this entire process. 5 6 Contents List of Figures 9 1 Introduction 11 2 Background 13 2.1 Music as a Form of Therapy . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 Sound Sculptures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3 O ther Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3 Mechanical Design 19 3.1 O verall Shape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Arduino and Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3 P ure D ata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4 C asting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.5 Sound D esign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.6 Sensor Mapping and Interactivity . . . . . . . . . . . . . . . . . . . . 25 4 Results and Interactions 27 4.1 Final M odel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2 Human Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 5 Conclusions and Future Work 31 Bibliography 33 7 I, 8 Mil"MMIRWRIMIN"p"n - List of Figures 2-1 'Composition #11' . . . . . . . . . . . . . . . . . . . 1 5 2-2 'Sound Sculpture' . . . . . . . . . . . . . . . . . . . 1 5 2-3 'Ollie' . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 6 2-4 'Music Shaper' . . . . . . . . . . . . . . . . . . . . . . 1 7 3-1 Shape Brainstorming . . . . . . . . . . . . 20 3-2 Concept Brainstorming . . . . . . . . . . . 20 3-3 Interaction Concepts . . . . . . . . . . . . . . . . . . 21 3-4 Breakdown of Pure Data functions . . . . . . . . . . 22 3-5 Objects used to remotely turn on Speakers in Pure Data . . . . . . 23 3-6 Full setup of Pure Data script . . . . . . . 23 4-1 Final Orb . . . . . . . . . . . . . . . . . . 27 4-2 Inner Structure .................. 29 9 10 Introduction Music has been widely adopted as a form of therapy for conditions such as depression and anxiety, and is commonly used by people for self-care in times of need. [1] Physical exercise or movement is also often used to treat depression, as it gives people an outlet for emotions that are otherwise hard to express.[2] Many stress-reducing objects today involve fine motor movement, like fidget spin- ners, stress balls, slime, or meditation balls. These products provide a subtle way for people to let out pent up stress through movement, but don't allow for a more immersive experience of stress reduction. This thesis investigates the ways in which both of these concepts can be integrated into one tool to help reduce stress. By creating an interactive object with both physical and auditory feedback, users can experience multiple forms of stress-relief, facilitating distraction from internal anxieties. The topic of stress relief was chosen because within the MIT community stress is a huge problem. Many students suffer from depression or anxiety, and up to 28% of students will utilize MIT's mental health resources to help treat these conditions throughout their time at MIT.[3] If the student body was able to utilize tools that facilitated relaxation methods such as music therapy, they might be able to increase student well-being and generally reduce stress across MIT's campus. 11 12 Background 2.1 Music as a Form of Therapy Music therapy aims to aid development of a person's communicative or expressive abilities through a combination of sound and musical meditation. There are two main forms of music therapy: receptive music therapy, in which people listen to music, and active music therapy, which allows people to make music. [41 Several studies have been done that look at music therapy's effect on conditions such as depression, anxiety, and seasonal affective disorder, among other things. It has proven to be an effective form of treatment for many of these conditions. One such study looked at the way it can be utilized in elder care facilities for patients with Alzheimer's disease or dementia. It was found that listening to mu- sic calmed patients and made them feel more at ease in situations that had other- wise caused them anxiety or frustration. Music therapy has also been shown to aid Alzheimer's patients in maintaining a personal identity, through exposure to music they might remember from other points in their lives, as it can help with recalling memories associated with a certain piece or style of music.[4] Multiple studies have also been done on music therapy's effect on depression. One study compared two groups of people already undergoing standard care for depression - one group used music therapy in addition to other methods of treatment, and one did not. Through EEG measurements of the people in both groups, it was shown that the group using music therapy demonstrated significantly fewer symptoms of depression 13 after the study than the one that did not.[5] Active music therapy is thought to be this effective because it allows patients to express emotions through both aesthetic and physical experiences, and can invoke positive mood changes. It also encourages increased awareness of one's self and surroundings, which can help patients reconnect with themselves outside of their disorder.[4] Other studies completed in nursing homes have shown the positive effects music therapy can have on people suffering from seasonal affective disorder. It was found that those completing music therapy scored lower on both the self-rating depression scale and the Hamilton depression scale once treatment had ended than the control group. [6] 2.2 Sound Sculptures Interactive sound sculptures are another inspiration for this project. Allowing the ob- server to take an active role in the progression of an artwork after it has been installed is especially of interest. In the scope of this project, this aspect of collaboration is especially important for stress relief, as it gives the user active engagement in that they can control what music is created - which is a form of active music therapy. One example of such a project is Eirik Brandal's 'Composition #11' (Figure 2-1), which is a sound sculpture that incorporates proximity sensors. Observers can move their hands or their whole body to change both the volume and the pitches created by the sculpture - it behaves similarly to a theremin. The sound stops completely when no one is close enough to it to produce sound, so passerby can't completely decipher the work without also engaging with it.[7] Another example is Ryan Edwards' 'Sound Sculpture' (Figure 2-2). This piece consists of large flexible blocks containing speakers, which can be stacked and moved around by observers. By changing the position of the blocks in relation to each other, the music created by them is altered. Users can build new structures with the blocks, and in doing so, change the sounds occurring around them completely.[81 14 Ii "ImiAgi A T V Figure 2-1: This piece by Eirik Brandal, 'Composition #11' is an interactive sound sculpture, that uses proximity sensors to allow observers to play it as they would a theremin. [7] Figure 2-2: This piece by Ryan Edwards, 'Sound Sculpture' is an interactive installa- tion, that allows users to move musical blocks around in relation to each other, thus creating new sounds throughout the space they occupy.[8] 15 2.3 Other Projects A variety of other therapeutic robots and devices have been created to combat de- pression, anxiety, or dementia. One such robot is Ollie, an plush interactive otter that responds to a user's touch. When someone strokes Ollie, it makes soothing sounds, vibrates, or hugs the user's hand to provide positive, supportive feedback (Figure 2-3). It is sized to be approximately the dimensions of a human baby, so the user feels as though they are caring for something that is alive and responsive to love and affection. Ollie was an effective project, but because it was created as a part of a senior design class at MIT, it was never sold as a product on the market. Similar products do exist, such as Paro, a therapeutic robotic seal, but they are sold at prices upwards of $6,000, making them inaccessible to most people looking for relief.[9] Figure 2-3: Ollie is an animatronic otter that is responsive to human touch. It was created as a therapy tool to help patients with dementia.[91 16 Several projects from Tod Machover and the Opera of the Future group at the MIT Media Lab have also looked at how music can be made more accessible to the public. One such project is the "Brain Opera", which was an interactive installation and opera performance. The audience attending would first enter a room called the Mind Forest, which contained several kinds of hyperinstruments they could interact with and play. One of these hyperinstruments was the Rhythm Tree, which was a giant percussive instrument consisting of large sacks with pads attached to them. Up to 50 users could press the pads at a time, creating percussive music. Another was the Melody Easel, which let users draw on a pressure-sensitive screen and outputs a single- line melody based on what was drawn. Once the audience had explored the Mind Forest, their input from the hyperinstruments was incorporated into the performance aspect of the opera, which was also performed on similar hyperinstruments.[10] Figure 2-4: Music Shapers are one of the Music Toys created for Tod Machover's "Toy Symphony" They allow users to manipulate pieces of music through touch.[11] Another project from the Opera of the Future that incorporates accessible music 17 is the Toy Symphony. Instruments called Music Toys were created for this project, which are designed so that anyone can play them, regardless of age or musical back- ground. These included Music Shapers (Figure 2-4), which were squeezable cloth instruments created with conductive embroidery and capacitive sensing. Children playing the Music Shapers could manipulate a piece's timbre, density, or structure - all of which are high-level concepts they would otherwise be unable to experience - just by squeezing the handheld instrument.[12] Though some tools for the improvement of mental health exist, there is no product currently on the market that utilizes music therapy to alleviate stress, anxiety, or depression. The goal of this project has been to create an instrument that combined both active and receptive music therapy to facilitate stress relief. 18 Mechanical Design The desired instrument needed to be a flexible but durable structure, that would respond to a user's touch and squeeze through sounds corresponding to their actions. It had to be able to successfully emit these sounds somehow either through the outer shell, or in another way that made it clear that they were coming from the object itself. The instrument was also supposed to relax the user, so it had to be easy and intuitive to operate, and could not emit sounds that would induce stress in users. A large part of designing this object involved figuring out the unseen details of the inner programs and structure. Though aesthetically the design is simple, a lot of careful work went in to figuring out how to place sensors inside, what those sensors should do, and how they should be interacted with. 3.1 Overall Shape Several shapes were considered for this object, but a simple sphere was decided on because it could maximize both the amount of sensors used and the different kind of sensors used. This is because it has the largest, most uniform surface area of the other shapes considered, making it ideal for embedding a variety of sensors. Other shapes would have constraints on sensor type and amount because of their corners or other irregularities. 19 Figure 3-1: Shape Brainstorming A sphere also allows the object to be more mysterious. If someone is handed an unmarked white sphere to explore, it will be harder for them to keep track of everything they discover than it would be if the shape were something like a pyramid, cube, or any more irregular shape. There are no landmarks on a sphere to remember - aesthetically, the object is the same no matter how you look at it. This is also why the orb was not marked in any way. Indicator marks like those in Figure 3-2 were considered as a way to guide the user through the object, telling them where to press to hear what sounds, but were decided against because it took the thrill of discovery out of the experience. 0 Figure 3-2: Concept Brainstorming When interacting with the object, users were expected to be able to treat the object as they would a stress ball - which is what they were instructed to do during testing. It would be completely compressible and flexible in multiple dimensions, so 20 it could be used without a fear of breaking or over-stretching it. Figure 3-3: Interaction Concepts 3.2 Arduino and Sensors A variety of sensors were needed for this project to create a wide spectrum of possible sound outputs, that could change depending on how the users interacted with the orb. The main characteristics sought out in sensors were: flexibility, which was necessary so they couldn't be felt beneath the shape's outer shell; and a non-requirement for capacitive touch, so the sensors could be activated through the outer material. This led to the choice of three kinds of sensors - soft potentiometers, pressure sensors, and bending sensors. Each of these was connected to an Arduino, where the output data could be configured. In the Arduino script, incoming data from each sensor was mapped to a specific range of values unique to each sensor. This data was then sent to the serial port, where it could be sent on to other programs. By mapping the output data this way, you allow other programs taking in data from the serial port to split up the data by sensor. This way, data from each sensor can be manipulated individually, and tailored to each sensor's exact function - this will be done in the electronic music program Pure Data. 21 3.3 Pure Data Pure Data was chosen as a music software environment because of its ability to communicate with Arduino through the serial port. It inputs the data being output by the sensors, which was previously mapped to specific number ranges in Arduino, with the 'comport' object. This data can then be split based on number range in Pure Data, and then scaled to higher frequency ranges before it's sent to the speakers as sound waves. This structure, which was used for each sensor, is shown below in Figure 3-4. devices(oen1 close( comport on 96fy split3a76 205 moses 190 mod 0 scale'180 205408E osc~ 0 Figure 3-4: Breakdown of the Pure Data functions that act on each sensor. It splits the incoming serial data from the comport object based on frequency, and re-maps it to a range of higher frequencies that are then sent to the speakers. 22 loadbang delay iOO d dsp 1 Figure 3-5: This string of objects sends a 'bang' to the sound functionality in Pure Data, dsp, so it turns on automatically when the program is opened. This is neces- sary when running the program on a Raspberry Pi, because you can not turn it on manually. adba devces( open 10( comport 10 9600 split3~15 45 split3 46 75 split3 76 MI]sli16 splispi3161 split3 176 205 moses 20 moses 68 moses 80 moses 11590 mod 0 mod 0 mod 0 mod mod 9 mod 9 scale 20 55 M2000i scale 50 75 400 500 scale 80 10550 sc0al0e 119 135 600 70e scale 140 175 600 909 scale 18 254 80 0 osc~ 0 osc- 0 osc- 0 osc- 9 osc~ 0 osc- a dac- Figure 3-6: This shows the full setup in Pure Data, where each of the six branches coming from the comport object correspond to one sensor. The full Pure Data script splits up each of the 6 sensors into a range of frequencies, which are then sent to the speakers. To control when sound is on or off, a loop was created with the 'moses' object, that directs the input based on size. If a number is over a certain threshold, sound is let through to the osc object. Otherwise, it resets the number to 0, which is then sent to the speakers, turning them off. 23 To get Pure Data to run remotely, a Raspberry Pi was used, which meant the program had to run independently on startup, since there was no monitor or other controls for the computer. This meant an object had to be added to the script that turned on the speakers (DSP in Pure Data) automatically once the program was loaded (Figure 3-5). It essentially sends a 'bang' to the speakers, telling them to turn on when the program is opened. The delay you see is necessary because of the way audio is configured in Raspberry Pi - without it, Pure Data tries to connect to the speakers before they are fully turned on. Once this was done, a command was added to the startup script on the Raspberry Pi that simply opened and ran the full Pure Data script (Figure 3-6). 3.4 Casting When designing the outer shell, casting seemed like the best option as it would allow for a wider variety of material choice, as well as more flexibility in choosing shape. Once a sphere was chosen as the final shape, casting also made a lot of sense because the shell could then easily be manufactured as one solid, continuous piece. The sphere was manually roto-cast inside a 8 inch hollow plastic sphere. The sphere was cast from silicone, which was chosen for two reasons. For one, it is a very flexible yet durable material, which lends itself to being squished by users repeatedly. It's also very soft to the touch, which makes it pleasant to hold. Silicone was also chosen because it typically has a very quick cure time (about 8 minutes), making it ideal for manual roto-casting. 3.5 Sound Design The sounds used in the orb were selected based on both the way the sensors them- selves behaved, and the way people were expected to react to them. Generally, pitch frequencies were chosen to be in range of 200-800Hz. This was for several reasons. During testing, it was found that high pitched frequencies were more noticeable than 24 much lower ones when played together, and almost seemed to drown them out. Be- cause of this, most pitches on the extreme ends of the frequency ranges were removed. Most of the sensors had ranges spanning about 200Hz, so when activated some pro- duced deeper droning sounds, and some produced higher, more 'cute' sounding pitches that would make the orb sound almost like a purring animal. The combination of these sounds would help the user experience joy while using the orb. 3.6 Sensor Mapping and Interactivity Each sensor's chosen frequency range was tuned to create soothing sounds - generally, sounds found to increase happiness or reduce stress - that varied based on the expected user interactions. For example, the force sensors were all mapped to smaller ranges, because the speed at which people apply pressure in this case is slow - so a smaller range will make the pitch changes more noticeable and in tune with the person's action. It if were a larger range, the pitch would change more rapidly and the resulting sounds would be more chaotic and abrupt. The soft potentiometers, however, have a larger range of frequencies, because they output data based on where they are pressed rather than how hard they are pressed. This allows for a wider variety of pitches to be produced from each potentiometer, and a more noticeable difference in sounds between locations pressed on the sensor. In designing this aspect of the orb, it had to be considered that many of the sensors could easily be activated at once. Because of this, the sounds coming from each had to be somewhat complimentary, but still different. If a user picked up the instrument and poked it on one side and then the other, two somewhat different sounds should be emitted. Otherwise, they might assume the sound is uniform regardless of how they interact with it, and will be less motivated to continue playing with it. By varying the way sensor data is mapped to pitch, the possibility of this happening is mostly eliminated. This way, users can interact with the orb in any way, and always have a different experience that urges them to continue using it. 25 26 W-W Results and Interactions The final outcome of this project largely met the goals initially set for the object. The inner workings of the orb functioned as expected, and the outer shell was able to be a fully sealed ball, allowing users to squeeze it however they desired. ~A/ Figure 4-1: The final prototype of the orb, presented with headphones hooked up to a radio, which received sounds from the orb through a radio transmitter. 27 4.1 Final Model The final model was also largely successful in terms of meeting the technical spec- ifications set out initially. To assemble the entire orb, thought had to be given to how it could maintain its flexible nature while containing the Raspberry Pi, Arduino, battery, and the sensors. Initially, the orb was going to have a rigid inner structure made out of acrylic, but after initial testing it was decided against, because users could feel it underneath the silicone. From here softer structures were considered, mainly using thin flexible foam, however, there still needed to be some rigidity in the structure, because the force sensors had to press against a solid object in order to work correctly. After testing with the completed inner circuitry, it was discovered that the space occupied by the electronics was more than enough rigid structure to allow the sensors to function correctly, so the design was finalized with the thin foam structure. Sound output for the device also went through several iterations before being finalized. Initially, speakers were embedded inside the orb, so sound came directly from the object itself. This version was very effective, because it made it seem more like the object was a living being creating sounds in response to touch. Speakers were moved away from largely because the orb was going to be presented in a busy space with a lot of things going on around it, but for future iterations they will likely be brought back. For the version presented, a radio transmitter was plugged into the headphone jack on the Raspberry Pi, which sent audio to a radio outside the orb. This way, the orb could be fully sealed, and the user could listen through noise-cancelling headphones. 4.2 Human Interactions Limited informal user testing was done with the instrument, but given the nature of the testing no fully conclusive results could be drawn. Most people using the orb had positive feedback - several people mentioned it was relaxing after using it for a short 28 period of time. Generally, more carefully organized user testing has to be completed before concrete conclusions are drawn. -W-SJU 1ilfl1 Figure 4-2: The inner structure of the orb was made out of thin foam, which was flexible enough that users could not feel it, but provided enough rigidity that the force-based sensors had something to push against. 29 30 9..1.- 1, iae: 1.- .i> ,.- .3-, .- - -> ;- .-, p-. .-, ,2. .rspg.-e;.- .-3, i-, a:. .;g: .-,, i1.. -. .--- -.132.- - ,-a s.r:-~y a o.iN tl1- -:e, ,- 'N.a ri.sM 4e- -'1 :.k -=:" -E - -.4. s1'i '-: .n'P - Conclusions and Future Work Though the orb met most of it's desired objectives, there are definitely several ways in which the current prototype could be improved. For one, the inner electronics need to be further refined in terms of durability. Given the nature of the instrument, it is necessary for the rigid objects within to be designed to take a decent amount of force in multiple directions. There were several issues during user testing where various components got unplugged or turned off accidentally by people using the orb. These could be avoided by designing the electronic components more carefully, so there is more of an enclosure surrounding buttons or wires that are currently exposed within the instrument. Another area for improvement has to do with how the inside parts of the orb are accessed in the current prototype. Right now, there is a slit in the side of the silicone shell that can be resealed using superglue, which creates a strong enough seal to keep the orb together while people are using it, but can be easily broken when the instrument needs to be serviced. Though this is effective, it is not ideal. A potential solution would be adding a charging port and an on/off switch to the outside of the shell. This, paired with more durable inner electronics, would eliminate the need to access the inside of the instrument, allowing the orb to be fully enclosed at all times. Future iterations on this project may also explore the possibility of different sizes and shapes for similar stress-reducing instruments. Objects large enough to be hugged or lay down on by the user are especially of interest, because it would further the aspects of stress-reduction that can be utilized by the instrument. Looking into the way users react to the instrument on a more data-based level would also be 31 informative. For example, monitoring a user's heart rate while they interact with the orb would be useful, because the way people react to specific sounds could be tracked, and the orb could be further refined to promote relaxation. 32 Bibliography [1] Robert Nauert PhD. Music Soothes Anxiety, Reduces Pain. Psych Central, 2018. 12] Kirsten Weir. The Excercise Effect. American Psychological Association, 2011. [3] The Office of the Chancellor MIT. The healthy minds study 2015 survey results. Technical report, Massachusetts Institute of Technology, 2015. [4] Damino Laura, Jayez Sylvie, and Saladin Aurore. The Effects of Music Therapy on Anxiety and Depression. Austin Publishing Group, 2015. [5] JAfirg Fachner, Christian Gold, and Jaakko ErkkilAd'. Music Therapy Modulates Fronto-Temporal Activity in Rest-EEG in Depressed Clients. Brain Typography, 2016. [6] Xifang Liu, Xin Niu, Qianjin Feng, and Yaming Liu. Effects of five-element music therapy on elderly people with seasonal affective disorder in a Chinese nursing home. Journal of Traditional Chinese Medicine, 2013. [7] Eirik Brandal. Composition #11. 2016. [8] Ryan Edwards. Sound Sculpture. 2017. [9] Kristina Johnson, Will Kalb, Luke O'Malley, David Orozco, Patricia P Pezzi, Luke Plummer, Doug Sanchez, Andrew Stratton, Emily Yang, Rick Zang, Alix de Monts, Anders Rasmussen, Carrie Liang, Chris Haid, Hazel Zengeni, Jose Lara, Kate O'Connor, Mateo Pena Doll, Nilu Zhao, and Tim Wilczynski. Ollie. Senior project, Massachusetts Institute of Technology, Mechanical Engineering Department, September-December 2013. [10] Scott Wilkinson. Phantom of the Brain Opera. Electronic Musician, 1997. [11] Katherine Taylor. Duet for Composition and Software. The New York Times, 2013. [12] Lisa Scanlon. Toy Symphony Maestro. MIT Technology Review, 2003. 33