Let me not tell you what I think

I’m working on a risk assessment with a social and environmental impact fund operating in some challenging markets. Last week we discussed different ways to run the project, and the team chose the most remote and arms-length option. I think that was a great idea, and this post explains why.

Sheep

In a June 2016 Harvard study called “Getting an Honest Answer: Clickers in the Classroom”, Dan Levy, Joshua Yardley, and Richard Zeckhauser found that we herd. When asked to raise our hands (as a polling tool), we vote with the majority answer. A potentially significant problem if you - like me - have used this tactic to:

✅ Confirm understanding.

✅ Assess levels of agreement/disagreement.

✅ Determine preferences.

✅ Make decisions.

In the context of our risk assessment, we discussed the relative merits of interviews, workshops (with quiz and poll features), and using our surveying platform and tools to gather multiple individual inputs. Luckily, the impact fund chose the latter option (to start). In the Harvard study mentioned above, when they moved from a show of hands to clickers, the voting disaggregated from the majority answer, revealing a much-changed picture.

By gathering individual perspectives without the contamination of the pack, we get a more accurate reflection of what people think, but we may also get better and more actionable data. Various studies - including a curious NPR one involving guessing the weight of a cow - indicate that when we make individual estimations, the median is relatively accurate.

Shepherds

We influence each other, especially if one party is supposed to know what they’re doing. Have you ever gone against your gut instinct because someone with apparent expertise voiced a contrary opinion? Or do you know certain things because of the opinions of experts?

If you’re thinking, “That's not me; I’m no sheep," write down all the reasons you can to explain how gravity works, why the sky is blue, or why the earth is round. Was it a compelling and impressive list, sure to convince a cynical adult (or worse, a curious toddler)? We are more easily influenced by assumed or received knowledge than we’d like to recognise.

Or have you listened to the opinion of someone respected - just not in the field they’re talking about. In the movie, Team America, I remember The Film Actors Guild railing against “Corporations being corporation-y”. It was funny but also concerning, as we are swayed by the views of successful people even when they’re talking on topics where their knowledge may be suspect, at best.

We must avoid contaminating the process with questions that lead or assume knowledge. I have done this in the past for what I felt were good reasons. I used to lead with questions about challenges I knew were common in the given market or sector to provide examples of the information we needed. I did this because you can get rubbish answers if you ask broad and open questions about what risks people might face in their work. It’s an abstract question. I was wrong. Here are a few better ideas to get you out of these pitfalls:

1. Ask about drivers of risk (pressure, leadership, targets, actions of third-parties, etc.).
2. Use plausible scenarios and ask people to explore any potential issues.
3. Ask about what people do and know, not lead them with what they should do or not do.

Slaughterers

Shepherds will have (generally) good intentions - a desire to help facilitate or further the discussion. However, shepherding can kill conversation when ill-intentioned or overly dogmatic. These slaughterers may not even realise what they’re doing. I’m sure you’ve sat in a meeting with a David Brent-type-leader (if you’re unsure who that is, watch this and tell me you haven't been in this room):

In the risk assessment context, it’s not uncommon to get answers that don’t stand up to scrutiny. People amid regulatory crises sometimes argue that their compliance controls or culture are excellent - nobody calls baby ugly!

The job is not to dismantle their argument showcasing your mastery of the topic. It’s much more challenging than that - it’s to allow space for dissenting views from your stakeholders without spurring needless confrontation. You can then progress to present the median (or more balanced) perspectives. It’s challenging to face a leader in denial in a room with cowed or supine subordinates. A few hacks:

1. Gather views of more junior staff first, and ideally in private.
2. Present these views to leaders in private initially, and explain how we might change the less favourable opinions.
3. If you’re identifying who said what, try and keep it to the team level (minimum of five people) to provide some shielding.
4. Use scales, ranking, and other similar tools that depersonalise any disagreement.
5. Move scenarios into the future (depersonalising and removing any stigma of past failure).
6. Let people be the baddies - “How might you get around [Process Y] if you were a fraudster?”
7. Add some less-charged topics into the question mix to (subtly) highlight the dangers of myopic thinking.

On that last point, I like to poll people on whether they are ethical (seldom to always), and then the next poll is whether others are ethical. The inevitable contrast explains that we judge ourselves by intention and others by action. It also shows us that right and wrong can be a matter of perspective, so let’s think freely and broadly (not in binary terms).

Breaking away from the pack

To gather accurate data - in training, assessments, information gathering, or just general feedback - we have to walk several fine lines:

💡 Anonymity vs knowing who thinks what (so we can help).

💡 Explaining the concept well without leading.

💡 Challenging without harming.

We must:

👉 Allow people to voice opinions personally, preferably anonymously, to start.

👉 Be careful not to contaminate views by introducing (supposed or actual) expert opinions.

👉 Let all voices be heard, quietest first, and beware dominant or overbearing types.

This is a short article, and more is left out than put in. I look forward to hearing how you gather information, avoiding the sheep, shepherd and slaughterer shenanigans!