Helping couples to co-sleep through mixed reality

A system that helps cohabitating couples become more synchronized in their sleep routines. It includes a set of holographic candles and an app that connect to the home’s lighting system.

Role: 3D interaction design, video prototyping, model making, research. 

Team: Rossa Kim, Julia Petrich, Nehal Vora

Sponsored by Phillips Healthcare in Fall, 2016 

Brief: Design a mixed reality sleep solution in a multi-user context. 

Presented at TEI 2017 as a demo paper in Yokohama, Japan 
Co-author: Austin S. Lee
In collaboration with Julia Petrich, Nehal Vora, Rossa Kim for Phillips Healthcare in 2016
I was responsible for 3D & video prototyping, model making, and concept generation.
Presented as a work-in-progress demo paper (co-authored by Austin S. Lee) at TEI 2017 in Yokohoma, Japan.

After looking at various family contexts, we became interested in the sphere of newly cohabitating couples, two adults with potentially divergent lifestyles start living together. These transitions can get a little rough when partners have to adjust to a new routine. How these transitions affect the health of the relationship was our main interest.

How Does Corus Work?

An Alternative Way to Control Connected Lights

The components of Corus are the main candle in the bedroom and other “satellite” candles around the home. The app connects to the main candle through the cloud. The satellite candles are connected to the main candle through Bluetooth and can also control the smart lights such as Philips Hue.  Users can control the lights two ways: through the app and by using gestural inputs on the candles.

The components of Corus are the main candle in the bedroom and other “satellite” candles around the home. The app connects to the main candle through the cloud. The satellite candles are connected to the main candle through Bluetooth and can also control the smart lights such as Philips Hue.  Users can control the lights two ways: through the app and by using gestural inputs on the candles themselves.


Visualizing the Sleep Compromise

The candles offer feedback to the users in their color. Each user has a different accent color on their app. The candle’s color will be somewhere in the middle of these two hues. This helps the couples get a sense of the balance in their preferred schedules, as in, who is getting closer to their preferred schedule and who might be making more of a compromise on average at any time.


A Day with Corus: From Sleep to Sleep

At sunset, the candles light. When bedtime is approaching, satellite candles around the home begin to flicker to notify the couple. With a gesture or using the app, the bedtime can be delayed. As bedtime nears, the flames and the lights in the living area dim/fade. The lights and the candle in the bedroom, however, light up/remain bright. This nudges the couple to relocate to the bedroom.

When the couple is in the bedroom and in bed, the main candle recognizes the audio level from the conversation and is responsive to it. While the conversation has energy, the lights and the candle remain lit. As the conversation slows, the lights and the candle dim. Once the conversation stops, the candle burns out, and the lights turn off.


Creating Engagement Through Personalized Holograms

As the sun rises, the holograph reappears as a gently animated waxy abstract “sculpture” of something relevant to the couple such as celebrating their birthdays or informing them about today's weather through 3D visualizations.

Onboarding via the Smartphone App

The smartphone app is used foremost to set a goal and initial preferences, but it also functions as a secondary notification of approaching bedtime and a dashboard of analytics of evening conversations' energy through visualizing the audio level.


An Alarm for Sleeping, Not Waking Up

While designing the app, we have examined the current trends of sleep-related apps. While many of them were alarm apps with various features, the market was lacking a proper bedtime alarm/notifier.


Continuity of the Physical Experience in the App

We have started our wireframes with the onboarding process. We have proposed how the home screen can lead to light controls for the home’s various rooms, to preference adjustment. We have also discussed different notification types: a pop-up for when users are on their phone and a screensaver that lights up to notify you.

Interacting Hologram Candles with Gestures

Users interact with the candle through three gestures.
1. "Spark the flame" to light a candle.
2. "Fan the flame” to snooze the bedtime.
3. "Pinch out the flame" to turn off the lights.

Prototyping: Using Wizard of Oz Methods

We have mainly used Pepper's Ghost technique, which is an almost century old technique that is used in theaters for creating illusions by using prism reflections. We have prepared our composition for Pepper's Ghost in After Effects, which we aligned and arrayed one of our animation instances in 360 degrees by 90 degrees.


Defining the Primary User Group

To focus our primary user group and problem area, we looked into various family contexts surrounding sleep.  We were most interested in cohabitating couples: researching the transitions when two individuals with potentially distinct lifestyles start living together and the adjustments they make to develop a new sleep routine.


Storyboarding Utopias and Dystopias

To share with our interviewees, we have developed three provocative interaction scenarios for far future. We have framed our storyboards in two categories: Utopia Storyboards for Personal Space & Comfort, Dystopia Storyboards for Over-indulge Couples.

Journey Mapping the Pain Points

We did some deeper mapping of the experiences of each of our interviewees. As we did this, we started to see a pattern: couples who have dyadic sleep schedules have few, if any, issues around sleep, and most couples who do experience pain points around sleep could improve their situation with a more synchronized sleep schedule.

Research Summary

Our research explorations have ranged from reading academic literature and articles, creating a survey with open-ended questions, and conducting more in-depth interviews, which included showing some design provocations to our potential users.


Research Findings

In our territory map, we imagined the journey of a couple as they decide to move in together, to when the big move-in day arrives, through the process of “settling down” together. We clustered these into areas of interest: (1) incompatible sleep behaviors, (2) different sleep cycles, and (3) the balance of privacy and intimacy.

We then started brainstorming the design with the question of what we constitute as ideal sleep in the context of couples. What do we think ideal couple sleep should be? And how can we apply these two areas we see in a couple’s mutual life for maintaining a dyadic sleep? 

Matching Interaction Ideals with Technologies

While on one hand focusing on these interaction "ideals", we wanted to keep in mind all of the new technologies that are becoming more dominant in our world. Our matrix exercise helped us see which technologies had the most potential for a couple sleep solution.


Final Concept

To promote healthy value in developing a more synchronized schedule in cohabitating couples, we designed a holographic candle system, Corus. It aims to help couples get a sense of balance in their preferred schedules by calculating the optimal, mutual go-to-bed time for the couple and make them aware of the compromises to be made at any time through subtle interaction modalities.

Prototyping and Designing

Testing the Hologram Idea

While discussing possible prototyping methods of our hologram idea, we have explored the Pepper's Ghost and immediately started to prototyping our mockup. Our earlier mockups on Pepper's Ghost helped us to finalize the form of our candle avatar.

Designing the Brand and UI

At the same time, we mapped the components of the whole system. We have also worked on brand identity as well as the initial wireframes of our app, which included onboarding and general usage screens.

Prototyping the Candle Base

We also have experimented with several forms of Pepper's Ghost prisms. Since our focus was to create a proof-of-concept, we have decided to incorporate the prism shape. Using a laser cutter and a 3D printer, we have fabricated our physical Pepper's ghost mockup.

Creating the Video Prototype

To generate a realistic candle flame and smoke, we have used Autodesk 3D Studio Max with FumeFX extension. For smoke, we used After Effects Trapcode extension.We have also experimented with RealFlow to create waxy animations of morning notifications. 

Finally, we have used Keyshot and Adobe After Effects to compose our effects, 3D renderings, and the shot footage.

© 2017 Meriç Dağlı