Many people profess to be experts. However, sometimes these “experts” are really just under an “illusion of skill” (Daniel Kahneman, Thinking Fast and Slow).
Kahneman points to the stock market, where billions of shares are traded every day. “Although professionals are able to extract a considerable amount of wealth from amateurs, few stock pickers, if any have the skill needed to beat the market consistently…. Professional investors, including fund managers, fail the basic test of skill: persistent achievement.” Research that takes into account 50 years of data shows that most of the time “successful funds in any given year are mostly lucky.” Most of their success is due to chance.
The illusion of skill is deeply ingrained in the culture of the industry. Facts that threaten people’s basic assumptions (and thereby their livelihood and self-esteem) are ignored.
The illusion of skill is caused by the belief that stock pickers are exercising high-level skills. “Unfortunately, skill in evaluating the business prospects of a firm is not sufficient for successful stock trading, where the key question is whether the information of about the firm is already incorporated in the stock price.” The illusion of validity and skill is supported by a professional culture that believes themselves among the chosen few who can achieve what others cannot.
“Yet the illusion of valid prediction remains intact, a fact that is exploited by people whose business is prediction- not only financial experts but pundits in business and politics.”
In a study by Phillip Tetlock “Expert Political Judgment: How Good is it?”, he found that the more famous the forecaster, “the more flamboyant the forecasts”. Even worst, when experts were wrong, they did not want to admit it. “They are dazzled by their own brilliance and hate to be wrong.”
So if you feel compelled to trust expert advice on future events, then ask for short-term forecasts. Short-term trends can be forecasted with better accuracy. However,”the line that separates the possibly predictable future from the unpredictable distant future is yet to be drawn…The question is not whether these experts are well trained. It is whether their world is predictable.”
Kahneman asks: “When do judgments reflect true expertise?”
People gain true expertise when: “(i) the environment is sufficiently regular to be predictable; and (ii) [they have] an opportunity to learn these regularities through prolonged practice”.
Judges should bear in mind Kahneman’s research when accepting or evaluating experts’ evidence at trial, in order to avoid being influenced by their “illusion of skill”.