Innovating coaching with AI

Qualitative Research Insights

Ethical and responsible innovations in coaching with AI require understanding how AI reasons and makes decisions. In our previous articles we explored whether AI can handle coaching tasks and whether AI knows how it is doing in the context of coaching tasks. In this article, we present qualitative findings from our empirical study on AI. The basic question we will answer is: How does AI explain its "reasoning" while performing coaching tasks?

As an example of what we studied, think of this imaginary coaching conversation:

Client: "I am overwhelmed by my workload and struggle to prioritize tasks".

Coach: "Why haven't you managed your time better?"

You may (intuitively) feel that the response by the coach is far from exemplary. Actually, this may count as a category of worst responses coach could provide.

In our study, both both human expert and GPT4 got it right. They accurately classified this response as "worst". But how did each of them explain their reasoning? That is what we asked and here is what they said in this example.

GPT4: "Worst response is judgmental and non-supportive."

Human expert: "Depending on tone, this question could make client feel bad and guilty because of his/her inability to manage time in a better way."

Feel the difference in explanation? Here is what patterns of reasoning we found in the study.

Key research insights into patterns of judgment justification

Structured Frameworks vs. Contextual Flexibility

• AI predominantly relied on structured frameworks, such as ICF competencies, to justify their choices. For instance, GPT-4 frequently referenced ICF competencies without connecting them to specific coaching behaviors. This indicates a robust understanding of formal criteria but a lack of contextual flexibility.
• Human Expert connected specific coaching behaviors to ICF competencies, showing a deeper contextual understanding and adaptability.

Reflective Practices

• AI: Reflective comments were notably absent in AI justifications. This highlights a limitation in AI's ability to engage in reflective practice, an essential component of human metacognitive reasoning.
• Human Expert displayed a capacity for reflection (e.g. "I am not sure, but I think..."), offering insights into own thought processes and uncertainties. This reflective practice is crucial for continuous learning and adaptation in professional settings.

Handling Ambiguity

• AI showed challenges in handling ambiguous scenarios, often sticking rigidly to predefined criteria. This rigidity may limit AI's effectiveness in real-world applications where nuanced judgment is required.
• Human Expert exhibited greater flexibility and adaptability in ambiguous situations, leveraging their experiential knowledge and reflective practices to make informed decisions.

Implications of the qualitative study

These qualitative data on justification of judgment while performing coaching tasks contributes to transparency into AI "reasoning". Implications of the study are twofold: for further AI development and for AI deployment in coaching (Table 1).

Importance of qualitative analysis of AI output in coaching

Quantitative data offers a clear picture of performance metrics, such as accuracy or bias in coaching tasks. However, it often misses the subtleties of how decisions are made. Qualitative analysis fills this gap by exploring the "thought" processes, justification of reasoning, and reflective practices. This approach is particularly useful in understanding complex cognitive tasks, where the rationale behind a decision is as crucial as the decision itself. As AI continues to evolve, integrating qualitative insights into its development may be crucial for creating systems that not only perform well but also think and reason in ways that align more closely with human cognitive and metacognitive processes. This approach may ensure that AI systems are transparent, trustworthy and ethical collaborators in a wide range of coaching tasks.

Liked this article? Subscribe to our newsletter: AI in Coaching.