RetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches

Abstract

The small screen size of a smartwatch limits user experience when watching or interacting with media. We propose a supplementary tactile feedback system to enhance the user experience with a method unique to the smartwatch form factor. Our system has a deformable surface on the back of the watch face, allowing the visual scene on screen to extend into 2.5D physical space. This allows the user to watch and feel virtual objects, such as experiencing a ball bouncing against the wrist. We devised two controlled experiments to analyze the influence of tactile display resolution on the illusion of virtual object presence. Our first study revealed that on average, a taxel can render virtual objects between 70% and 138% of its own size without shattering the illusion. From the second study, we found visual and haptic feedback can be separated by 4.5mm to 16.2mm for the tested taxels. Based on the results, we developed a prototype (called RetroShape) with 4×4 10mm taxels using micro servo motors, and demonstrated its unique capability through a set of tactile-enhanced games and videos. A preliminary user evaluation showed that participants welcome RetroShape as a useful addition to existing smartwatch output.

Publication
In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’17)
Date

Motivation

Smartwatches provide quick access to short-time entertainment applications, especially when users are on-themove, e.g. in a bus or train. However, user experience in such applications is limited due to the small screen area and limited input and output options. While smartwatch visual and auditory technologies have improved substantially, the potential of smartwatch-enabled haptics in video and game applications remains to be exploited. We leverage the user’s skin under the watch face for sensing haptic output with collocated visual content. Our approach enhances the viewing experience on a smartwatch using a shape-changing tactile display on the rear surface of the smartwatch

Prototype

RetroShape is composed of a 4 × 4 linearly actuated pin array. The pins were 3D printed, and have a 10 × 10 mm footprint with no space between them. Each pin is connected to miniature servo motors, which can extend to 7 mm from their resting position. Each servo motor weights 1.7 g, and has a dimension of 14 × 6.2 × 18 mm (l × w × h). Under the working voltage of 3.7v, each servo motor can exert a twisting force of 75g and rotate in a maximum speed of 1200° per second. The motors are controlled using an Adafruit 16-Channel 12-bit PWM/Servo Shield, which was connected to an Arduino Mega board, communicating to a MacBook Pro laptop. Software was written in Processing.

Prototype 3D model

Selected Press Coverage