Project Details
Tools
IFTTT Platform, MESH Sensors, Processing, Arduino, Sketch, Illustrator
Time
March - April. 2018
Role
Product Designer, UX/UI
Twinkle Stare
Twinkle Stare is my second semester's final studio project at Parsons School of Design. Twinkle Stare is an IoT device that reconnects me in New York and my dog in Taiwan to be able to feel her presence and recreate a moment we share together.
01. Overview
Our lives are comprised of relationships, whether it is your loved ones and friends or acquaintances at work or school. While what we often talk about is that people build strong relationships with one another, it’s also very common to see people grow close bonds with animals, particularly dogs. In fact, dogs have gained the title of being “man’s best friend”. I have been around and owned a dog all my life since elementary school. In my opinion, dogs are some of the most healing and comforting animals in this world. They have helped me gotten through some of the toughest times in my life.
Brainstorm
Last year, enrolling in graduate school in the United States meant a long-distance relationship with my family, as well as my dog. Long-distance with a sick family member is different, harder, and much more unbearable. This sick family member is my dog, Xing Xing, who was diagnosed with bladder cancer two years ago. This whole process was hard for our family because the tumor was malignant and would spread and be unable to contain someday. With original intentions of bringing Xing Xing with me in New York while I study in Parsons, her sickness made this impossible. After coming to New York to study and leaving Xing Xing behind with my parents, I’ve always wished that I could somehow spend more time with her, if not actually be able to be there to take care of her. I wondered, how do I spend more time with her remaining days over long-distance? How do I recreate a moment with Xing Xing over long-distance to constantly remind me and feel that she is with me?
How do I spend more time with my dog's remaining days over long-distance?
02. Research
After brainstorming, I began researching on the different exisiting IoT products out on the market because it is the most common way to reduce the distance between two beings and still be able to communicate through wifi. There have been countless IoT project and devices built for long-distance relationships for people but much less for dogs. This is because of the difference in how our sensory systems are built. Dogs have poor eyesight and are colorblind but have excellent hearing and olfactory system. Dogs can hear 4 times further away and twice as high pitched as humans can. A dog’s brain is 1/10 the size of humans but the part that controls smell is 40 times larger. They have 300 million olfactory receptors in their noses, compared to about six million in ours. As a result, dogs sense the world very differently than we do. It’s not easy for them to feel our presence without us being physically there.
With all the research I have done about IoT objects & dog's sensory system in mind, I also began to do an extensive research on my dog. I listed out all the characteristics of my dog and the interactions I have with my dog. My dog Xing Xing is 11 years old, a Shetland Sheepdog breed with black and white long-coated fur, who is shy, sensitive, and needy. She is very sensitive to sounds compared to most other dogs. As a result, I often play relaxing and soothing music for her when she keeps getting distracted with different sounds, for example, on nights with thunder or fireworks. She also is also a vocal dog and will bark when she hears a sound. Xing Xing constantly needs me to be around in her sight and follows me around everywhere in the house. There are many meaningful interactions we share together. One particular habit is that she loves to lay on her doll to rest her head while watching me. Somehow, instead of playing catch-and-throw with the dolls I bought for her, she likes to use them as her pillows. With his habit of hers, it gave me the inspiration to incorporate this action of her into the concept of my IoT device with her.
A more comprehensive research and background can be found here
This is a picture of Xing Xing lying on her dolls.
Inspirations
After researching, I started to look into all the IoT devices products where I could get some inspiration on how it could work on my and my dog. Below are some of my favorite products I have found that is similar in some way to what I am thinking of doing.
Pillow Talk by Little Riot in 2015 - This device lets you hear the real time heartbeat of your loved one over long distance. The device has two parts. One speaker goes under your pillow to hear the heartbeat of your loved one and a wristband you wear to measure your heartbeat to send over to your loved one. The psychical presence aspect of this IoT device is particularly strong and is one aspect that I want to try to incorporate into my own device with my dog.
Findster Duo (left) & TailTalk (right) - Both devices are tracking devices. Findster Duo reads your dog's fitness level to monitor his/her health and can track your dog's location. TailTalk reads the emotions of your dog by tracking their tail wags. IoT devices for dogs are mainly just tracking devices because of the difference with dog’s sensory system, they are unable to sense our presence unless we are physically there. I will have the same problem when designing for my dog.
03. Ideation
My initial concept was that I wanted to create a sleeping mat (for dog) and a pillow (for me) that are in sync and senses when she is lying on the mat which will in turn display the contour shape of her lying body on my pillow. When I see the shape of her body appears on my pillow, I can interact with it to turn on a soothing music that helps her sleep well. Since from my research, I saw that my dog was mostly lying down resting, I thought this was one of her most meaningful interaction that could connect to me. Below is the storyboard of how this device would work.
Storyboard
In Taipei, Taiwan
Pressure sensor
There is my dog, Xing Xing, walking around and beside her lies her bed. Her bed has a pressure sensor to detect when she lies on it.
In New York, USA
In Taipei, Taiwan
The pillow on the right of the diagram is connected to Xing Xing's sleeping mat.
When she sleeps on her bed or lies on it, the pressor sensor will detect her weight.
In New York, USA
The contour shape of her body will show up on my pillow in NY. I can interact with it to trigger a response over at her side.
In Taipei, Taiwan
In this case, the response is to play her favorite soothing music.
Although I quite like my initial concept, I felt that the interaction is a bit forced. After my user testing and feedback, mentors said my interaction was too tool-like. It didn't feel personal or special enough of a memory we share together. It feel like just me trying to find something to connect my dog to me. So I decided to continue brainstorm of the activities and memories we do together. What are the objects that remind of her specifically. Then I thought about the doll I bought for her from Japan. And this led me to my final revised concept of my project, also one that I really love.
04. Prototyping
After much a bit of revision, my final concept is to create an IoT device that consists of two main parts. The first part is the doll that she lies on which will have a pressure sensor and a mini speaker inside. When she lies on the doll at home in Taiwan which she loves to do, it will activate the camera embedded in the model doll of her (the second part of the device) that sits in front of my desk in New York. This doll will "stare" and "look" as I move around using face tracking. When I feel uneasy, I can easily turn the "stare" off by patting the head of the model. I can also turn on my dog's favorite soothing music that helps calm her down when agitated.
This device serves to reconnect me in New York and my dog in Taiwan to be able to feel her presence and recreate a moment we share together.
Overall Framework
With my concept confirmed, I began to think of how to actually build this and make this work. After some research, my whole framework to make this IoT device work is displayed in the diagram above. I am using the IFTTT webhooks service to send to communicate with my MESH sketch. Ideally, I would incorporate this with the MESH sensors and by using the IFTTT webhooks service to send to communicate them. After spending much time to make the technical part work, my code was working fine in Arduino and Processing, though, at times unstable. I decided that with the time I had, I would not make the IoT connection actually happen but just focus on the interactions and the physical doll of my device which is indicated by the dashed yellow lines on the left of the diagram.
Technical Prototype
User testing with face tracking in Processing connected to Arduino
Trying the face tracking process and connecting to the pressor sensor in Arduino
Making & Testing
Final Prototype Video
05.Visual Design
Color system
Type
06. Next Steps
There are many challenges I face making this project that I hope to continue work on. There is many more testing to be done because there were so many problems encountered just with building the physical doll of my project. For example, the webcam did not work will behind my fabric with some distance issues. The pressure sensor did not work as well as I thought and the complexity of the code. In the future, I would like to continue to refine the design and look of the doll and also make it actually work over local distance and let it be wireless with bluetooth.