Smartphone-based interaction sensing to innovate our face-to-face social lives

Advancing our smartphones as conversational interaction sensing platforms to promote social health in everyday life

Article  |  Spring 2014

A team of researchers in the Dept. of Computer Science at KAIST are leading new mobile initiatives to build social context platforms and create life-immersive social services. The key aspirations are to advance our commodity mobile devices such as smartphones towards a face-to-face social gateway, to acquire the real-time social contexts, and to create proactive services naturally overlaid on our everyday face-to-face social lives. Why social? Because it is an integral basis that constitutes many of our daily activities, such as family life, team-oriented work, and hanging out with friends. The research team believes that branching out to diverse social contexts around the user will help to realize future mobile systems that achieve a holistic understanding of the user’s needs and true life-immersive services.

The quest begins with a question: “What would be the basic context in our face-to-face social interaction?” As an initial and natural step, the research team focused on conversation, which would be the most prevalent tool to communicate with people in front of us. A number of new technical challenges arise in recognizing delicate conversational contexts with commodity mobile devices.

In 2013, the team reached the first milestone, SocioPhone, a mobile face-to-face interaction platform that focuses on meta-linguistic contexts in conversations. SocioPhone is described in a paper published in ACM MobiSys 2013[1] (International Conference on Mobile Systems, Applications, and Services) by a team consisting of KAIST graduate Youngki Lee, who is now an assistant professor of Information Systems, Singapore Management University (SMU), and eight other members. SocioPhone summarizes on-going conversations as a sequence of turns and pauses. and it provides applications with a set of intuitive APIs to monitor rich meta-linguistic context on the fly, without requiring computation-intensive semantic inference on conversation contents. In its core, the SocioPhone runtime monitors conversational turn-centric contexts in a highly-efficient and precise manner based on a new socially-leveraged collaborative sensing scheme. In addition to the paper publication, a live demonstration of SocioPhone won the Best Demonstration Award in ACM HotMobile 2013[2] (International Workshop on Mobile Computing Systems and Applications), which was conducted by a KAIST graduate student Chulhong Min and 14 others.

Continuing these efforts, the research team is now creating compelling applications are indispensable for a new platform to find its unique values. Notably, they proposed TalkBetter, a prominent initiative for everyday clinical care applications. The paper describing TalkBetter won the Best Paper Award in ACM CSCW 2014[3] (International Conference on Computer Supported Cooperative Work and Social Computing), which was authored by an interdisciplinary team consisting of KAIST graduate Inseok Hwang who is now a research assistant professor of Center for Mobile SW Platform in KAIST, Prof. Dongsun Yim in the Dept. of Communication Disorders, Ewha Womans University, and five others. They introduced a mobile in-situ intervention service to expedite everyday family-driven care for children with language delays. Through close collaboration with speech-language pathologists and patients regularly receiving speech-language care, TalkBetter has been designed and developed as an integrated therapeutic service that helps the parents to acquire clinically desirable conversation habits throughout natural conversations, not only depending on simple a-priori prescription as has been done thus far.

Today, many people observe growing concerns about the anti-social effects of smartphones, attempting to suppress the smartphone use in face-to-face social situations. However, the research described above provides an antithesis of such technophobia. Such research pursues the pro-social potential of pervasive mobile devices around us, and realizing the first-of-its-kind mobile systems to improve our everyday social lives.

These studies were supported by National Research Foundation funded by Ministry of Science, ICT, and Future Planning (MSIP) and IT R&D Program of MSIP/KEIT of the Korean Government.

1. Lee, Y., Min, C., Hwang, C., Lee, J., Hwang, I., Ju, Y., Yoo, C., Moon, M., Lee, U., Song, J. SocioPhone: Everyday Face-To-Face Interaction Monitoring Platform Using Multi-Phone Sensor Fusion. ACM MobiSys 2013, Taipei, Taiwan, June 2013.
2. Min, C., Hwang, I., Lee, J., Hwang, C., Yoo, C., Moon, M., Park, T., Lee, C., Lee, H., Kim, Y., Ju, Y., Lee, Y., Lee, U., Song, J. Demo: Bringing In-situ Awareness to Mobile Systems: Everyday Interaction Monitoring and Its Applications. ACM HotMobile 2013 (Demo), Jekyll Island, GA, USA, February 2013.
3. Hwang, I., Yoo, C., Hwang, C., Yim, D., Lee, Y., Min, C., Kim, J., Song, J. TalkBetter: Family-driven Mobile Intervention Care for Children with Language Delay. ACM CSCW 2014, Baltimore, MD, USA, February 2014.