Physical therapy (PT) is a crucial and highly beneficial activity that supports recovery from acute injury and generally promotes mobility and functioning throughout the lifespan. However, adherence to at-home PT is a major challenge given boredom, motivation, and difficulties tracking progress. Aiming to enhance people's PT experiences and outcomes, this project leverages the engaging, therapeutic power of music to provide intuitive, real-time feedback and adaptive guidance during exercises. Specifically, informed by the music therapy literature and a highly iterative user-centered design process, our system tracks and sonifies limb movements using state-of-the-art computer vision and music generation algorithms to capture motion data during rehabilitative exercises and convert that information into music and sounds that serve as acoustic guidance. Our lab-based evaluation shows this musical approach improves exercise performance and user engagement compared to traditional practices. Moving forward, we are working on deploying the system in-the-wild to evaluate its effects over time in real-world contexts.

Research assistants: Emma Kallman (22W-22S), Lindsey Kim (22S-22F), Ruiqi Ma (22S), Andrada Pantelimon (22W-22S), Ziyi Zhou (23S-present)

Graduate student collaborators: Yong Hoon Chung (Dartmouth Psychological and Brain Sciences PhD student), Yijing Feng (Dartmouth Computer Science PhD student), Alastair Huntley (Medical Assistant, Dartmouth College Health Service)

Academic and clinical collaborators: Prof. Michael Casey (Dartmouth Computer Science and Music), Eric Ellingson (MSPT), Prof. Viola Stormer (Dartmouth Psychological and Brain Sciences)