Helping co-sleeping via mixed reality

A system that helps cohabitating couples become more synchronized in their sleep routines. It includes a set of holographic candles and an app that connects to the home’s lighting system.

Team: Rossa Kim, Julia Petrich, Nehal Vora

Role: Leaded and did all types of prototyping including video, physical, and digital. Designed gestural interfaces and 3D interface elements. Assisted UX with wireframing and information architecture. Equally contributed to the research. Guided client meetings with designers and researchers from Phillips Healthcare.

Brief: Design a mixed reality sleep solution for multi-users. 

Duration: 6 weeks (Fall 2016)

Sponsored by Phillips Healthcare

Presented at TEI 2017 as a demo paper in Yokohama, Japan 
Co-author: Austin S. Lee
In collaboration with Julia Petrich, Nehal Vora, Rossa Kim for Phillips Healthcare in 2016
I was responsible for 3D & video prototyping, model making, and concept generation.
Presented as a work-in-progress demo paper (co-authored by Austin S. Lee) at TEI 2017 in Yokohoma, Japan.

The Challenge

Our team was asked to design a mixed reality sleep solution for multi-users by Phillips Healthcare. We saw that majority of the existing solutions focus on the sleep quality of an individual person. We began to explore all kinds of questions to define the context of family, sleep, and health while encouraging us to design for an emerging technology: mixed reality.

The Solution

Corus is a system that helps cohabitating couples become more synchronized in their sleep routines. It includes a set of holographic candles and an app that connects to the home’s lighting system. Corus aims to help couples to adjust their new life routines.


How Does Corus Work?

An Alternative Way to Control Connected Lights

Corus consists of a main candle in the bedroom and other satellite candles around the home. The satellite candles are connected to the main candle through Bluetooth and can control the smart lights such as Philips Hue. Users can control the lights two ways: through the app and by using gestural inputs on the candles.

The components of Corus are the main candle in the bedroom and other “satellite” candles around the home. The app connects to the main candle through the cloud. The satellite candles are connected to the main candle through Bluetooth and can also control the smart lights such as Philips Hue.  Users can control the lights two ways: through the app and by using gestural inputs on the candles themselves.

Corus System Map

Visualizing the Sleep Compromise

Each user has a different accent color on their app. The candle’s color will be somewhere in the middle of these two hues. This helps the couples get a sense of the balance in their preferred schedules, as in, who is getting closer to their preferred schedule and who might be making more of a compromise on average at any time.

A lock-screen notification can visualize which partner is comprimising more.

A Day with Corus: From Sleep to Sleep

At sunset, the candles light. When bedtime is approaching, satellite candles around the home begin to flicker to notify the couple. With a gesture or using the app, the bedtime can be delayed.

As bedtime nears, the flames and the lights in the living area dim. The lights and the candle in the bedroom, however, light up/remain bright. This nudges the couple to relocate to the bedroom.

The main candle responses to the audio level of the conversation in the bedroom. As the conversation slows, the lights and the candle dim. Once it stops, the candle burns out, and the lights turn off.

Corus Interaction Map

Engagement Through Personalized Holograms

As the sun rises, the holograph reappears as an animated waxy abstract sculpture of relevant things such as birthday celebrations or informing couple about today's weather via 3D visualizations.

Video Sketches of Suggested 3D Notifications

Onboarding via the Smartphone App

The app is used foremost to set a goal and initial preferences but also works as a secondary notification of approaching bedtime.

Corus App Splash - Main Screen - Goal Setup

An Alarm for Sleeping, Not Waking Up

We found out that, the market was lacking a proper bedtime alarm. After we published Corus, Apple also released the bedtime feature.


Continuity of the Physical Experience in the App

Along with the onboarding process, our wireframes showed how the home screen can lead to light controls for different rooms. We also explored different notification types: a pop-up for when users are on their phone and a screensaver that lights up to notify you.

Wireframing the User Flow

Interacting Hologram Candles with Gestures

Users interact with the candle through three gestures.
1. "Spark the flame" to light a candle.
2. "Fan the flame” to snooze the bedtime.
3. "Pinch out the flame" to turn off the lights.

The Wizard of Oz Prototyping

We mainly used Pepper's Ghost (PG) technique, which is a century-old technique that is used in theaters for creating illusions by using prism reflections. We used After Effects to prototype a digital PG.

Going Forward

Provider Digest No. 2
Provider Digest No. 2


Corus was my first interaction design project with an international team. Starting from the research, I relearnt a lot of things that I know about the design process such as user research and Wizard of Oz prototyping methods for mixed reality.

Detailed User Flows and Microinteractions

I believe we could have detailed our app user flows of more in terms of onboarding as well as micro-interactions.

More Prototyping on Smart Light Controlling

While our video prototypes sketched how the system can control the lights of the house, we should also do more research and prototyping to notice the nuances and trade-offs of such scenario. 

Real Candle as a Natural User Interface

In Corus, we used the metaphor of candle holographically, I also see an opportunity to use a real candle as a natural user interface for such applications while it has certain safety trade-offs. 


Defining the Primary User Group

We started with researching various family contexts surrounding sleep.  We were most interested in cohabitating couples: researching the transitions when two individuals with potentially distinct lifestyles start living together and the adjustments they make to develop a new sleep routine.

Our Preliminary User Group Hypothesis/Questions

Storyboarding Utopias and Dystopias

To share with our interviewees, we have developed provocative interaction scenarios for far future: Utopia for Personal Space & Comfort, Dystopia for Over-indulge Couples.

Journey Mapping the Pain Points

We did some deeper mapping of the experiences of each of our interviewees. As we did this, we started to see a pattern: couples who have dyadic sleep schedules have few issues with sleep. Most couples, who do experience such issues,  overcome them with a more synchronized sleep schedule.

Research Summary

Our research explorations have ranged from reading academic literature and articles, creating a survey with open-ended questions, and conducting more in-depth interviews, which included showing some design provocations to our potential users.

Some stills from Our Preliminary and Secondary Research 

Research Findings

In our territory map, we imagined the journey of a couple as they decide to move in together, to when the big move-in day arrives, through the process of “settling down” together. We clustered these into areas of interest: (1) incompatible sleep behaviors, (2) different sleep cycles, and (3) the balance of privacy and intimacy.

Carbon Territory Map and Design Directions
Our definition of ideal sleep as a group and focus areas for our final design

Design Directions

We started brainstorming our solution with the guidance of the design directions that we come up as a result of our research.


Matching Interaction Ideals with Technologies

While focusing on these interaction "ideals", we wanted to keep in mind all of the new technologies that are becoming more dominant in our world. Our matrix exercise helped us see which technologies had the most potential for a couple sleep solution.

Idea Generation Matrix

Final Concept

To promote healthy value in developing a more synchronized schedule in cohabitating couples, we designed Corus. It aims to help couples get a sense of balance in their preferred schedules by calculating the optimal, mutual go-to-bed time for the couple and make them aware of the compromises to be made at any time through subtle interaction modalities.

Idea Generation Matrix

Prototyping and Designing

Testing the Hologram Idea

While discussing possible prototyping methods of our hologram idea, we have explored the Pepper's Ghost and immediately started to prototyping our mockup. Our earlier mockups on Pepper's Ghost helped us to finalize the form of our candle avatar.

Designing the Brand and UI

At the same time, we mapped the components of the whole system. We have also worked on the initial wireframes of our app, which included onboarding and general usage screens.

Prototyping the Candle Base

We also experimented with several forms of Pepper's Ghost prisms. Since our focus was to create a proof-of-concept, we have decided to incorporate the prism shape. Using a laser cutter and a 3D printer, we have fabricated our physical Pepper's ghost mockup.

Creating the Video Prototype

To generate a realistic candle flame and smoke, we have used Autodesk 3D Studio Max with FumeFX extension. For smoke, we used After Effects Trapcode extension.We have also experimented with RealFlow to create waxy animations of morning notifications. 

Finally, we have used Keyshot and Adobe After Effects to compose our effects, 3D renderings, and the shot footage.

© 2018 Meriç Dağlı