Visual feedback in therapy systems

Integrating image-based pose recognition into therapy systems to support patient motivation.

ongoing Visual feedback in therapy systems

Movement is key in many therapies – but motivation cannot be prescribed. The MotionComposer turns movement into music, making the joy of movement and self-efficacy tangible. Together with MotionComposer GmbH and FusionSystems, we are investigating how AI-assisted pose recognition can extend the system with visual feedback.

In our current project we are researching how visual feedback can support the activation and motivation of patients in movement therapy.

In cooperation with the MotionComposer – an innovative therapy system that turns movement into pleasant sounds – we are extending the concept with a visual component.

Using AI-assisted, image-based pose recognition, the users’ posture is captured and various kinds of visual feedback are generated.

Research question

Which kinds of visual feedback really support patients – and how must an AI capture body posture so that the feedback is reliable, understandable and free of distraction?

Approach

A camera captures the movement, and an AI model for pose recognition estimates the position of head, torso and limbs. These body points are used to generate visual feedback that complements the MotionComposer’s acoustic response. We test different forms of presentation and observe how people react to them.

People at the centre

Here, human-AI interaction becomes directly tangible: the system responds to the person in real time, and the person responds to the system. Good feedback must therefore be fast, unambiguous and attuned to people – not technically impressive, but helpful. The camera serves only to capture movement; careful handling of the data is part of the concept.

Research topic: human-AI interaction →

Status and outlook

Pose recognition has been connected to the MotionComposer in a prototype. Next steps are trials with users and selecting the forms of feedback that best support motivation.

Any questions?

Interested in working with us or would you like to learn more about our research? We look forward to hearing from you.

Get in touch →