How Good is Your Judgement?

In a 2016 article Paul Schoemaker and Philip Tetlock remind us that “companies and individuals are notoriously inept at judging the likelihood of uncertain events, as studies show all too well. Getting judgements wrong, of course, can have serious consequences. Steve Ballmer’s prognostication in 2007 that there’s no chance the iPhone is going to get any significant market share, left Microsoft with no room to consider alternative scenarios,” (p.74).

 

There’s a direct link between judgement and accountability. For example in our youth our judgement often goes awry simply because we don’t feel accountable for our actions and hence don’t think things through properly. Even organizations have issues with ‘judgement’ because individuals aren’t held accountable; and hence don’t ‘learn’ to think things through quickly. In fact one of the downsides of command and control style leadership is that these leaders use their power to avoid accountability when it comes to ‘bad’ judgements being made.

 

Schoemaker and Tetlock mention “the experience of a UK bank that lost a great deal of money in the early 1990’s by lending to U.S. cable companies that were hot but then tanked. The chief lending officer conducted an audit of these presumed lending errors, analysing the types of loans made, the characteristics of clients and loan officers involved, the incentives at play, and other factors. She scored the bad loans on each factor and then ran an analysis to see which ones best explained the variance in the amounts lost. In cases where the losses were substantial, she found problems in the underwriting process that resulted in loans to clients with poor financial health or no prior relationship with the bank – issues for which expertise and judgement were important. The bank was able to make targeted improvements that boosted performance and minimized losses,” (p.74).

 

There’s the other side of the coin too – organizations that are so ‘risk averse’ that employees make judgements based on ‘fear of failure’ which rarely leads to optimum solutions for the organization in the short or long term. Most of us learn through the mistakes that we make and through the risks that we have taken in our lives. Being able to take risk is based on values like accountability, but also integrity and excellence, and safety where appropriate. Our core values, if aligned correctly, should be enough to allow us to make those judgements that have risk attached but the rewards are worth it; where even failure has a reward, as we learn ‘what doesn’t work.’

 

Schoemaker and Tetlock highlight how “most predictions made in companies, whether they concern project budgets, sales forecasts, or the performance of potential hires or acquisitions, are not the result of cold calculus. They are coloured by the forecaster’s understanding of basic statistical arguments; susceptibility to cognitive biases, desire to influence others’ thinking, and concerns about reputation. Indeed, predictions are often intentionally vague to maximize wiggle room should they prove wrong. The good news is that training in reasoning and debiasing can reliably strengthen a firm’s forecasting experience,” (p.75).

 

A key cognitive bias is the perceived culture of the company and the perceived impact this culture has on the judgements you make. This perceived culture will be different for employees in different departments; or at different levels or at different stages in their career; as well as the personal values of each of the employees in the organization.

 

Schoemaker and Tetlock remind us that “cognitive biases are widely known to skew judgement, and some have particularly pernicious effects on forecasting. They lead people to follow the crowd, to look for information that confirms their views, and to strive to prove just how right they are. Training can help people understand the psychological factors that lead to biased probability estimates, such as the tendency to rely on flawed intuition in lieu of careful analysis. Another technique for making people aware of the psychological biases underlying skewed estimates is to give them confidence quizzes,” (p.75).

 

We live in a world where judgements that are made can impact more than just the individual making the judgement, but can impact the whole world. Take global warming, a scientific phenomenon that if correct and not ‘checked’ will be the end of the world as we know it. Yet recently the President of the United States broke away from the Paris Accord on Climate Change, claiming ‘global warming’ was a politically driven ‘con’ and that because his grandfather was a Professor at MIT, he had an instinct for science – hmmm, making judgement on a phenomenon that could be the end of life on earth requires patience, accountability, integrity and much more serious debate.

 

Schoemaker and Tetlock remind us of the importance of building the right teams. “Whether a team is making a forecast about a single event or making recurring predictions, a successful team needs to manage three phases well: a diverging phase, in which the issue, assumptions, and approaches to finding an answer are explored from multiple angles; an evaluating phase, which includes time for productive disagreement; and a converging phase, when the team settles on a prediction. In each of the three phases, learning and progress are fastest when questions are focused and feedback is frequent,” (p.77).

 

The key to judgement in the 21^st century, besides the values already discussed is the concept of trust. We need to recruit the right people with the right experience and values; then trust them to make the right judgement calls and trust them to be flexible in their approach so that they will be honest when things start to go wrong or don’t work.

 

The judgements organizations make today shouldn’t just be about short term shareholder value, but about long term sustainable growth. The UK has seen the ‘high street’ change so much in the last 12 months. Retail stores that have been around for hundreds of years closing their doors for good or closing down a large proportion of their stores laying off thousands of employees. These closures are due to poor judgement and now everyone pays, even the shareholders.

 

As we become more technologically advanced; and as we see the divide between rich and poor become even greater, we need leaders, both from business and politics to make better judgement calls for the world in general – otherwise the future could be very bleak.

 

Finally Schoemaker and Tetlock suggest that “companies should systematically collect real time accounts of how their top teams make judgements, keeping records of assumptions made, data used, experts consulted, external events, and so on. Where well-run audits can reveal post facto, whether forecasters coalesced around a bad anchor, framed the problem poorly, overlooked an important insight, or failed to engage team members with dissenting views. Likewise they can highlight the process steps that led to good forecasts and thereby provide other teams with best practices for improving predictions,” (p.78).

 

References:

 

Schoemaker, P.J.H. and Tetlock, P.E. (2016). Superforecasting: How To Upgrade Your Company’s Judgement. Harvard Business Review, May, p.72-78.