Our original design for the interface of the tool began with a set of Powerpoint slides. These can be viewed in a video which we made available to our workshop attendees.

These drafts were put to our stakeholder reference group, made up of patients, carers, psychotherapists, psychotherapy trainers, and psychotherapy service managers.

We used the Trustscapes suite of tools from the UnBias project to gather information about concerns and worst case scenarios for this kind of tool, and discussed how concerns might be addressed.

As part of our Responsible Research and Innovation approach, we adapted a tool from acceptance and commitment therapy, the ACT matrix to gather views on how the workshops should work. The matrix we used is shown below.

Due to the pandemic, all of our workshop activities were conducted online using Miro an online whiteboard tool.

Through reflecting on the feedback from our stakeholder workshops, we decided on the following ways of progressing with the work started in this project:

• Future coproduction work is required to develop materials to help patients understand how recordings of their sessions would be used with our tool, to empower them to confidently agree/disagree to its use.
• Future studies will explore large scale patient acceptability of the tool, once fully developed.
• The tool won’t be used for therapists to review their own sessions, as they don’t have time to do this in their working day.
• We have begun exploring non-verbal utterances in psychotherapy recordings, to understand more about how emotion can be picked up in psychotherapy sessions.
• We are rethinking some of the explainability features from the draft design, which were found to be more confusing than helpful.
• The machine learning models developed are being tested in different datasets with different therapy types to ensure they are applicable across different therapeutic approaches.

You can hear more about the tool and what the stakeholders had to say about it in this video, which we presented at the Mindtech Symposium 2020.