Assessing the Expert
In many domains, we have to trust the expertise of others to guide our decisions. Yet not all experts hold rational beliefs, and many people who are framed as experts in media are not actually experts.
Below are some questions to ask when assessing the credibility of people who are framed as experts. Remember, COVID-19 is a developing pandemic, look for people with related expertise.
Check what their expertise is. A psychiatrist might be an expert in mental health, but they might not be an expert at public health. Find the experts on the specific subject. Additionally, are multiple experts in agreement on the idea? This is particularly relevant when someone disagrees with expert consensus. Sometimes people get a platform just because they have a novel or interesting idea, even if that idea is unconvincing to a knowledgeable person. It is of course possible that the non-expert is right and the experts are wrong, but it’s unlikely. As COVID-19 is further researched even experts recognize they were wrong at the beginning and with new information have changed their opinion.
People who have one big idea to explain everything are very bad at accurately modeling the world, predicting outcomes, and recommending effective actions. These people are often selected for media attention because they are clear and confident about their beliefs. The world is a complex place, and people who are able to model that complexity in their minds have better information than those who aren’t. Look for people who tend to use multi-factor explanations.
There are usually exceptions and nuances. Is this person able to build them into their mental model? As mentioned above, COVID-19 is a developing issue. Experts need to be able to recognize when they made an error. What was said in January 2020 about the virus may have been found to be inaccurate or only a small part of the problem. Inability to change one’s mind is a hallmark of an irrational belief system. Can you find examples of this person changing their mind in the past when presented with new evidence or a better interpretation of existing evidence?
Large-scale conspiracies are improbable and blaming the government as a scapegoat for all the world's ills (without specific examples) should raise suspicion. This has been a common tactic where the pandemic has become a political issue.
This is common among experts, less common among the best experts, and very common among non-experts. It’s also often easy to spot with a little effort. Do a search for a topic. Are there multiple sources with similar evidence and conclusion or are they cherry-picking individual studies that support their position?
Corruption, irrationality, and dishonesty do exist among experts, but if someone makes these claims without presenting clear evidence for them, it’s a bad sign. Accusing opposing experts of “lies”, “scams”, and other similar language is a red flag that the person making the claim is not objective. Conflicts of interest are common among experts, and good to keep in mind, but it’s useful to consider possible conflicts of interest of the person pointing them out as well. This is common to see on news outlets with the "talking heads" where multiple experts are presented and hurling accusations at one another.
The COVID-19 pandemic has become very political in some groups and public health guidelines are seen as an infringement on their freedoms. In this context, beliefs are often driven by group affiliation rather than rational consideration of evidence and the common good. The likelihood that someone is providing high-quality information in these domains is lower than in less controversial areas like physics or neuroscience. As you dive deeper into your specific topics and away from news outlets, you will find these arguments because less prevalent.
Adapted with permission of the author, from http://www.stephanguyenet.com/quickly-assessing-the-credibility-of-public-experts/
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.