Physical therapy (PT) is a crucial and highly beneficial activity that supports recovery from acute injury and generally promotes mobility and functioning throughout the lifespan. However, adherence to at-home PT is a major challenge given boredom, motivation, and difficulties tracking progress. Aiming to enhance people's PT experiences and outcomes, this project leverages the engaging, therapeutic power of music to provide intuitive, real-time feedback and adaptive guidance during exercises. Specifically, informed by the music therapy literature and a highly iterative user-centered design process, our system tracks and sonifies limb movements using state-of-the-art computer vision and music generation algorithms to capture motion data during rehabilitative exercises and convert that information into music and sounds that serve as acoustic guidance. Our lab-based evaluation shows this musical approach improves exercise performance and user engagement compared to traditional practices. Moving forward, we are working on deploying the system in-the-wild to evaluate its effects over time in real-world contexts.
2022 - Present
Murnane Research Group at Dartmouth College | Contact: emurnane@dartmouth.edu | Last updated: October 2024 | Login