TOTO
An Empathy Tool to Reinforce Positive Conversation
Toto is a psycho-reactive flower that responds to human conversation through emotive feedback. It blooms in response to positive phrases and shuts in response to negative ones, to convey the impact words can have on people and things around us.
SUMMARY
CONTEXT
This story shows how I conceptualized, developed and tested an interactive experience, for the College for Creative Studies’ Student Exhibition Opening held in Detroit, Michigan. Design industry professionals, college faculty, students, and their friends and families were expected to attend the exhibition and thus, were my audience.
The brief was to build an interactive installation in 10 weeks, that engaged this audience and conveyed a message that reflected my values as a designer and human being.
My subject was the impact of conversations on humans and other living things and I chose to express the value of positive conversation through this experience.
ROLE
I lead the project with Rinat Sherzer, who was my mentor and helped me refine the experience. I consulted Michael Evans about the code that ran the program. I also had the opportunity to test my idea with renowned American opera and theater director Yuval Sharon during an experience design workshop.
As part of my responsibilities, I conducted research, defined the concept, sketched scenarios, prototyped, user tested the developments and delivered a high fidelity artifact and experience.
CHALLENGES
• Affect human emotion and understanding
• Engage a varied audience in a vast and busy exhibition
• Develop a robotic installation with no prior experience in the programming language used or the hardware mechanics required
TOOLS
Python, Raspberry Pi, Servo motor, Arduino, C#, Potentiometer
PROCESS
My process was cyclical with four major stages, but was flexible and morphed with the demands of the project.
*****
CONCEPTION
Drawing insights from secondary research, I formulated goals and values to direct the project. Important questions rose from this exercise, answers to which helped me build the framework of the experience.
Identifying the Values
I explored interaction design projects that inspired me and observed the common attributes between them. The idea was to capture the values that made them interesting to me in three words. They were - joy, movement and surprise.
Joy was the primary value because of its emotive strength. I used a concept map approach to explore the values in possible relations with the user. This helped me find the opportunity for an interaction - Experiencing joy and sharing it with others.
Defining the Goals
I used secondary research to learn what interactions could help spread positive thoughts and ideas through people. Various studies and research detailed the significant impact words can have on plants, water and the human neurochemistry.
I learnt that negative words stay longer on the human mind than positive words. This was because the chemical cortisol - a stress hormone released in response to negativity, takes longer to metabolize in our body than oxytocin - a happy hormone released in response positivity.
A verbal conversation would thus be a simple yet powerful medium to induce and spread joy through my users. I decided to use it as the medium of interaction and listed my goals for the project.
Defining the goals sparked the right questions and the helped determine the steps that would advance the idea.
Building the Framework
The goals led me to three important questions, answers to which helped build the framework of the experience.
1. Who is the conversation with?
The user had to converse with something they could affect and empathize with. I turned to my research to brainstorm ideas for this object of empathy.
‘Secret Life of Plants’ was about a man’s effort to understand the sentience of plants, where he claimed that plants could perceive human intentions and emotions. The Shy Lights installation by Studio Drift was about capturing a natural mechanism is called ‘nyctinasty’ in certain flowers that allows them to close at night for self defense and energy conservation.
I realized users could easily empathize with a flower as it was a life form. Additionally flowers have behaviors I could use to reflect the impact of negativity and positivity. It could bloom or wilt in response to the quality of the conversation.
2. What is the conversation about?
The conversation had to be relevant to varied age groups and simple to follow. Relevancy was needed, to have an impact on the user. It would make sense for the conversation to be something the user was likely to say to someone.
After discussing with Rinat, we agreed that the best option was to use phrases that were spoken between friends and family members. Instructions could cue users to speak them to the flower, like they spoke to someone they knew.
I developed a prototype using foam core board and tissue paper to validate the idea with 4 users, using my hands to animate the behaviors. Testers expressed joy with the blooming and remorse with the closing of the flower. Some spoke their own phrases when the flower closed, to apologize for being rude. This built my confidence in the idea and I moved to my next question.
3.What is the experience framework?
I sketched a user journey focused on the emotions that needed to be evoked and the interactions that could help achieve that. The plan was to then backtrack towards what was possible in the amount of time and skill available.
Rinat and I discussed if adding layers of other behaviors for the user to discover would enhance the experience. But we concluded that the additional behaviors should be a secondary priority due to time constraints.
I continued refining the scenarios to find the basic structure of the interaction and ended up with what felt like a good starting point.
According to the scenario the flower listened to what was being said, determined the quality of the words and responded by actuating hardware components. This was used to determine how the flower was built.
DEVELOPMENT
The framework evolved over three cycles of rapid prototyping, refining of the experience and user testing. Each cycle uncovered a new set of challenges. Overcoming them made the experience more meaningful and the installation more robust.
CYCLE I
The installation had to last two weeks without breakdowns or errors in the exhibition. I built a software and hardware structure to define the physical and technical constraints and refine the experience accordingly.
Prototyping
I had basic understanding of using C# and Arduino to actuate hardware components like motors, LEDs and sensors. Speaking to Michael Evans helped me find the ingredients to develop the flower’s technical behaviors. I studied examples of mechanically blooming flowers to understand how to build the structure.
Physical Structure
I looked for examples that included physical computing that I was familiar with or could hack and learn quickly. I took inspiration from examples where the bloom of the flower was determined by the position of the servo. I was comfortable working with a servo, but the mechanical movements depended on a lot more than that.
JK Brickworks - https://www.youtube.com/watch?v=rQhdn95jBK0
I built a prototype with cardboard, brass wires and a micro servo to try the mechanics myself. I decided to use an Arduino Uno board and a potentiometer to turn the servo motor as it was the quickest way to test the mechanics and the interaction with users. It allowed me to control the bloom by turning the knob and fake the experience until the program was built.
Software
I initially used the ReSpeaker Raspberry pi hat for speech detection along with Snowboy, a hotword detection library. This combination is generally used to make voice enabled devices, where you can train your own audio models and use them to trigger hardware functions. The benefits of using Snowboy was that it didn't require internet connectivity to run and had low latency.
I used ReSpeaker because I could program it using Node.js, a language I was more familiar to than Python. But the hat took up all the pins on the Pi which meant it would complicate my hardware connections to the servo. I decided to use Python and plug a USB microphone into the Raspberry Pi instead of the hat. This simplified my hardware connections.
I configured a Raspberry Pi to work with Snowboy and tinkered with the dual hotword example. I managed to actuate different sounds when different example words were spoken. Speech detection was successful.
The drawbacks of using a hotword detector was that the audio models could only be a few seconds long which restricted my phrases to 2 - 4 words. I had to reconsider what phrases the flower would listen for.
Refining the experience
Blooming and wilting were the primary behaviors of the flower. But they felt binary and didn't reflect the difficulty of holding on to positive thoughts as opposed to negative ones. This was the key impact I needed to convey to the user.
Using only the servo position as output, I had to explain that it took a lot of effort to make the flower happy, but very little to make it upset.
I designed the conversation such that, three positive phrases opened the flower to a full bloom, but just one negative phrase caused it to close completely.
The phrases would be presented to the users as cards on or around the installation and were selected based on :
• Their ability to evoke emotions
• The ease of speaking them clearly (number of syllables)
• The success level of it’s detection by the software
After testing and refining the phrases against these constraints the ones that checked all the boxes were: "Hello there!", "Can we be friends?", "I love you." and "You’re ugly!"
They followed a basic progression of human interaction in relationships (greet, befriend, love, insult), so the conversation flowed without abrupt change in context.
User Testing
Testing the interaction with users revealed that although the speaking to the flower and having it respond was delightful and intriguing, the experience failed to establish the context. The users were not sure of the purpose of the interaction and thought it was just a cool robotic flower.
I tested the same prototype and interaction with Yuval Sharon and verbally explained to him the context after the conversation. The information that I shared about the impact of words on our neurochemistry made the experience more meaningful to him. He suggested that I include this fact as a part of the messaging displayed for the user as it added more value to the experience. This suggestion was very valuable.
The question that still lingered was - What should the users do with the information? I paid another visit to the goals and values of this project to try and answer that question.
The highest level goal was to invoke feelings of joy in the user and have them spread it to others. The experience so far focused on the experience of joy and not its sharing. This was an opportunity to enhance the interaction.
CYCLE II
Using the technical constraints and insights from the user tests, I returned to refine the experience and make the necessary developments to the physical prototype and the code.
Refining the Experience
Initial ideas for the 'sharing of joy' leaned toward digital sharing using social media, but that restricted my audience. It also involved the use of smart phones which would take users out of the experience both physically and mentally.
An analog approach felt more appropriate in this context. Users could write their own positive messages and share it with others by displaying it in the installation space. Thus encouraging their mind to generate a positive thought and manifest it by physically writing it down, making it a part of the installation.
Iterating Prototype
I tried making the flower look more realistic by using gift tissue paper. I assumed that the soft tissue petals would compress when the structure closed satisfying both aesthetics and function. But when the tissue flower was placed on the disc, the structure could not close completely and the servo became jittery.
I decided to observe the rotation of the disc in the center to use the motion to my advantage instead. I made the flower look like it was twirling open and close by overlaying thin strips of gift tissue and connecting the structure to the disc. I then fixed the flower on a poster board to test the complete experience with instructions.
User Testing
The messaging included an introduction to the flower, instructions for the conversation, underlying context and the cue to write a message. I used the potentiometer to open and close the flower based on the phrases spoken.
The test revealed that the experience was successful in both delivering the message and evoking the right emotions. Users were delightfully surprised by the blooming of the flower and enjoyed the mechanical sounds that the motor made while sweeping back and forth. They often went off script to get Toto’s reaction, but understood the context and shared their positive messages for others to see.
I realized that it would be useful to program a few more phrases for people to speak in order to give them more to play with. I expanded on the prior phrases by adding : “I’m Sorry”, “Are you okay?”, “You’re beautiful” and “Go away!”.
The twirling flower made the servo unstable. Too many structural connections pressured the servo, causing it to dislocate from the bottom. I had to find an alternative to the flower and wedge the servo into the base to keep it steady.
CYCLE III
The final stretch was about making sure all the individual components - the physical structure, the software and the messaging, came together perfectly. After validating the experience one last time, I proceeded to craft the aesthetics and begin set up.
Prototyping
Physical Structure
I built a more sturdy structure using a heavy duty servo, thread, screw and some hard plastic straws. The flower was still a structure of foam core board and bright red plastic straws with pencil markings on it. It could look more like a beautiful flower, but I decided to finalize the interaction and the program before I worked on the aesthetics.
Software
I worked on the code to detect the phrases and set the corresponding servo position and set the bloom level. Once the behaviors were established, I made the script run on startup so the flower activated as soon as the Raspberry Pi was plugged to the power supply.
User Testing
A final user test validated the overall experience and all the key points of interaction. I learnt that the software failed to recognize certain phrases sometimes and required a repeated listening. With too little time to address this through model training, I rewrote the instructions encouraging users to try again in such a case and not be discouraged.
Aesthetics
Wedging the servo into the base meant that the distance between the disc and the base reduced and the structural connections had to be made above the disc rather than below. This made it difficult for me to beautify the flower using gift tissue or any other material without hampering the motion.
I consulted Rinat about this and she suggested that I could paint the flower instead and use the structure as the inspiration. The structure was geometric and symmetrical which made me think of a Mandala, a spiritual symbol of the universe in various cultures, created using varied symmetrical patterns. I decided to use it as an inspiration to paint the structure. After trying a few patterns on my iPad I painted the structure using acrylic colors.
I built a space to house the installation using three pedestals of varying heights and widths. I placed Toto on a pedestal with the instructions, the context statement with cards on another. The third pedestal was where the users were requested to leave their positive messages for others to see.
OUTCOME
IMPACT
Most users immediately empathized with the flower when they caused it to close. Groups of people interacting together generated the most excitement by teasing each other for upsetting the flower or for speaking certain phrases. They also competed as to who Toto listened to more and was thus ‘liked’ more.
Children seemed to enjoy it the most, empathizing with the flower like it was a pet and repeatedly returning to speak with it. It also became a subject of artificial intelligence and security related conversations among some groups of professionals that interacted with it.
I returned to the installation after two weeks to find the pedestal full of positive messages.
REFLECTIONS
Due to lack of time and coding skills, I was unable to program the flower to reset if not interacted with for a while. This was the biggest glitch in the experience as some users approached to an already open flower.
The program was biased towards women or people with a voice that was similar to mine because I trained all the audio models myself. This was the drawback of using Snowboy instead of Natural Language Processing APIs.
The blooming motions could be made slower and smoother to reflect a real flower. But the audience enjoyed the mechanical sounds of the servo caused by the snappy movements and perceived it as an emotional sound instead of a mechanical one.
Lastly, Toto could have been much larger in size and the installation space could have been more aesthetically appealing.
IN CLOSING
This project tested my time management, critical thinking, problem solving and programming skills the most among others. I ran into some difficulties, but keeping my focus on the goals and values helped me get through them.
I also learnt about the significant power conversations have on people through this project. This is knowledge I will always carry with me in the quest of becoming a better human being with every passing day.
All in all I was happy with what I ended up discovering, learning and creating, but there is always room for improvement.
***
Thank you for taking the time to read my story. I'd love to hear what you think about it. Please feel free to send me your comments, suggestions or positive affirmations by clicking the button below.
