Helping co-sleeping via mixed reality

A system that helps cohabitating couples become more synchronized in their sleep routines. It includes a set of holographic candles and an app that connects to the home’s lighting system.

Brief: Design a mixed reality sleep solution for multi-users. 
Team: Rossa Kim, Julia Petrich, Nehal Vora
My role: User research, interface design, and prototyping
Duration: 6 weeks (Fall 2016)

Sponsored by Phillips Healthcare

Presented at TEI 2017 as a demo paper in Yokohama, Japan 
Co-author: Austin S. Lee

The challenge

From our user research, our team found out that the majority of the existing solutions focus on the sleep quality of an individual. As a team, we began to explore all kinds of questions to define the context of sleep and health to design for an emerging technology: mixed reality.


Research Highlights

Why Co-Sleeping?

Among many multi-user contexts, we decided to narrow in on cohabitating couples. We found that if couples can develop a routine of going bed and sleeping together, this may positively impact their relationship health.

To address sleep consistency and quality, we looked closely at the physical and relationship health of the couple through literature review, survey and target interviews. Three opportunity areas of conflict emerged from our research: (1) incompatible sleep behaviors, (2) different sleep cycles and patterns(3) balancing privacy and intimacy.


Storyboarding Utopias and Dystopias

To elicit reactions and gauge our users level of comfort with technologies, we developed two provocative future scenarios, each based on the concepts of utopia and dystopia.

Dystopic: Allow individuals to continue living their own different patterns and behaviors.
Utopic: Force a couple to find a place of harmony and balance

Design Principles

Our research led to three key insights about cohabiting couples in relation to their sleep and the time surrounding sleep. To transform these insights into our design, we concretized them into actionable design principles.

  1. Earlier the healthy arrangements, easier to maintain a relationship.
  2. Fairness runs the relationships: The decisions are give-and-take.
  3. Co-sleeping has a positive impact and couples don't know about it.

How Does Corus Work?

Controlling Lights to Unite Couple in Bedroom

Corus consists of a main candle in the bedroom and other satellite candles around the home. The satellite candles are connected to the main candle through Bluetooth and can control the smart lights such as Philips Hue. Users can control the lights two ways: through the app and by using gestural inputs on the candles.

The components of Corus are the main candle in the bedroom and other “satellite” candles around the home. The app connects to the main candle through the cloud. The satellite candles are connected to the main candle through Bluetooth and can also control the smart lights such as Philips Hue.  Users can control the lights two ways: through the app and by using gestural inputs on the candles themselves.

Corus System Map

A Day with Corus: From Sleep to Sleep

At sunset...

The lights are on. When bedtime is approaching, satellite candles around the home begin to flicker to notify the couple. With a gesture or the app, the bedtime can be delayed.

At bedtime nears...

The lights in the living area dim. The lights and the candle in the bedroom, however, light up/remain bright. This nudges the couple to relocate to the bedroom.

At night...

The main candle responses to the audio level of the conversation in the bedroom. As the conversation slows, the lights and the candle dim. Once it stops, the candle burns out, and the lights turn off.

As the sun rises...

The holograph reappears as an animated waxy abstract sculpture of relevant things such as birthday celebrations or informing users about today's weather via 3D visualizations.

Interacting Hologram Candles with Gestures

Users interact with the candle through three gestures.

"Spark the flame" to light a candle

"Fan the flame” to snooze the bedtime.

"Pinch out the flame" to turn off the lights.

Onboarding via the Smartphone App

While our brief asked us to design a mixed reality solution, we found that existing screen-based experiences are more suitable to onboard users to such experiences. The Corus app enables users to set a goal and input their preferences and also works as a secondary notification channel.


Corus App Splash - Main Screen - Goal Setup

Continuity of the Physical Experience

Along with the onboarding process, our wireframes showed how the home screen can lead to light controls for different rooms. We also explored different notification types: a pop-up for when users are on their phone and a screensaver that lights up to notify you.

wireframe [Recovered]
wireframe [Recovered]

Wireframing the User Flow

Visualizing the Sleep Compromise

The holographic candles offer feedback to the users in their color. Each user has a different accent color on their app. The candle’s color will be somewhere in the middle of these two hues. This helps the couples get a sense of the balance in their preferred schedules, in terms of who might be making more of a compromise.


A lock-screen notification can visualize which partner is comprimising more.

A Bed-Time Alarm for Couple

We found out that, the market was lacking a proper bedtime alarm. After we published Corus, Apple also released the bedtime feature.



Relearning the Design

Corus was my first interaction design project with an international team. Starting from the research, I relearnt a lot of things that I know about the design process such as user research and Wizard of Oz prototyping methods for mixed reality.

Issues around Bedroom Privacy

We evaluated Corus with participants from our exploratory research phase. While all participants liked how we approached the issue through subtle interactions, some of them shared their concerns about having a recording device in their bedrooms due to losing their privacy.

Going Forward

Detailed User Flows and Microinteractions

I believe we could have detailed our app user flows of more in terms of onboarding as well as micro-interactions.

More Prototyping on Smart Light Controlling

While our video prototypes sketched how the system can control the lights of the house, we should also do more research and prototyping to notice the nuances and trade-offs of such scenario. 

Design & Prototyping Process

Rapid Ideation Through Idea Matrix

To generate many concepts quickly, we created a matrix by which we crossed sleep goals with various technologies. This helped us be creative and forward thinking about our appreciation of tech for our project. We ultimately decide the final direction: focusing on dyadic sleep, with a secondary focus on the nighttime conversation.


Storyboarding the Final Concept: Corus

Our final concept, Corus, aims to help couples get a sense of balance in their preferred schedules by calculating the optimal mutual go-to-bed time for the couple. It also makes them aware of their sleep compromises through subtle interaction modalities.

Testing the Hologram Idea

While discussing possible prototyping methods of our hologram idea, we have explored the Pepper's Ghost and immediately started to prototyping our mockup. Our earlier mockups on Pepper's Ghost helped us to finalize the form of our candle avatar.


Designing the Brand and UI

At the same time, we mapped the components of the whole system. We have also worked on the initial wireframes of our app, which included onboarding and general usage screens.


Prototyping the Candle Base

We also experimented with several forms of Pepper's Ghost prisms. Since our focus was to create a proof-of-concept, we have decided to incorporate the prism shape. Using a laser cutter and a 3D printer, we have fabricated our physical Pepper's ghost mockup.


Creating the Video Prototype

To generate a realistic candle flame and smoke, we have used Autodesk 3D Studio Max with FumeFX extension. For smoke, we used After Effects Trapcode extension.We have also experimented with RealFlow to create waxy animations of morning notifications. 

Finally, we have used Keyshot and Adobe After Effects to compose our effects, 3D renderings, and the shot footage.