Baloney Detection: How to draw boundaries between science and pseudoscience by Michael Shermer
1. How reliable is the source of the claim? : Do the data and interpretations show signs of intentional distortion? When an independent committee established to investigate potential fraud scrutinized a set of research notes in Nobel laureate David Baltimore’s laboratory, it revealed a surprising number of mistakes. Baltimore was exonerated because his lab’s mistakes were random and nondirectional.
2. Does this source often make similar claims? Pseudoscientists have a habit of going well beyond the facts. Watch out for a pattern of fringe thinking that consistently ignores or distorts data.
3. Have the claims been verified by another source? Typically pseudoscientists make statements that are unverified or verified only by a source within their own belief circle. We must ask, Who is checking the claims, and even who is checking the checkers.
4. How does the claim fit with what we know about how the world works? When people claim that the Egyptian pyramids and the Sphinx were built more than 10,ooo years ago by an unknown, advanced race, they are not presenting any context for that earlier civilisation. Where are the rest of the artifacts of those people? Where are their works of art, their weapons, their clothing, their tools, their trash?
5. Has anyone gone out of the way to disprove the claim, or has only supportive evidence been sought? This is the confirmation bias, or the tendency to seek confirmatory evidence and to reject or ignore disconfirmatory evidence.
6. Does the preponderance of evidence point to the claimant’s conclusion or to a different one? The theory of evolution, for example, is ‘proved’ through a convergence of evidence from a number of independent lines of inquiry. Creationists conveniently ignore this confluence, focusing instead on trivial anomalies or currently unexplained phenomena in the history of life.
7. Is the claimant employing the accepted rules of reason and tools or research, or have these been abandoned in favour of others that lead to the desired conclusion?
8. Is the claimant providing an explanation for the observed phenomena or merely denying the existing explanation? This is a classic debate strategy—criticise your opponent and never affirm what you believe to avoid criticism. It is next to impossible to get creationists to offer an explanation for life (other than ‘God did it’.)
9. If the claimant proffers a new explanation, does it account for as many phenomena as the old explanation did? Many HIV/AIDs skeptics argue that lifestyle causes AIDs. Yet their alternative theory doesn’t explain nearly as much of the data as the HIV theory does.
10. Do the claimant’s personal beliefs and biases drive the conclusions or vice versa? All scientists hold social, political and ideological beliefs that could potentially slant their interpretations of the data, but how do those biases and beliefs affect their research in practise? Usually during the peer-review system, such biases ad beliefs are rooted out, or the paper or book is rejected.
How to Detect Fallacious Arguments (From Why People Believe Weird Things by Michael Shermer)
Reductio ad Absurdum: A reduction to absurdity; the disproof of a proposition by showing its consequences to be impossible or absurd when carried to a logical conclusion. Similar to Slippery Slope.
Theory Influences Observations: Heisenberg ‘What we observe is not nature itself but nature exposed to our method of questioning. The theory in part constructs the reality. Reality exists independent of the observer, of course, but our perceptions of reality are influenced by the theories framing our examination of it. Thus, philosophers call science ‘theory-laden’.
The Observer Changes the Observed: The act of studying an event changes it. (Social sciences and tribes.) Science tries to minimize and acknowledge the effects of the observation on the behaviour of the observed; pseudoscience does not.
Equipment Constructs Results: The equipment used in an experiment often determines the results. The size of our telescopes, for example, has shaped and reshaped our theories about the size of the universe. What my telescope can’t see isn’t there, and what my test can’t measure isn’t intelligence.
Anecdotes Do Not Make A Science: Anecdotes are told by fallible human storytellers…we need physical evidence of an alien spacecraft…
Scientific Language Does Not Make a Science: Dressing up a belief system in the trappings of science by using scientific language and jargon, as in ‘creation-science,’ means nothing without evidence experimental testing and corroboration.
Bold Statements Do Not Make Claims True: Something is probably pseudoscientific if enormous claims are made for its power and veracity but supportive evidence is scarce as hen’s teeth… the more extraordinarily well-tested the evidence must be.
Heresy Does Not Equal Correctness: They laugh at Copernicus. They laughed at the Wright brothers. Yes, well, they laughed at the Marx brothers. Schopenhauer, ‘All truth passes through three stages: First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as self-evident.’ But, ‘all truth’ does not pass through these stages. Lots of time ideas are accepted without ridicule or opposition violent or otherwise. Einstein’s theory of relativity was largely ignored until 1919, when experimental evidence proved him right.
Burden of Proof: The person making the extraordinary claim has the burden of proving to the experts and to the community at large that his or her belief has more validity than the one almost everyone else accepts.
Rumours Do Not Equal Reality: ‘I read somewhere that…’ or ‘I heard from someone that…’ passes into, ‘I know that.’
The Unexplained is Not Inexplicable: Many people are overconfident enough to think that if they cannot explain something, it must be inexplicable and therefore a true mystery of the paranormal… Even those who are more reasonable at least think that if the experts cannot explain something, it must be inexplicable… it is okay to say, ‘We do not yet know but someday perhaps we will.’ The problem is that most of us find it more comforting to have certainty, even if it is premature, than to live with unsolved or unexplained mysteries.
Failures Are Rationalized: In science, the value of negative findings—failures–cannot be overemphasized…failures are how we get closer to the truth. [Pseudoscienists] ignore or rationalize failures.
Post hoc, ergo procter hoc: After this, therefore because of this. Correlation does not mean causation.
Coincidence: When the connection is made in a manner that seems impossible according to our intuition of the laws of probability, we have a tendency to think something mysterious is at work. As the behavioural psychologist B.F. Skinner proved in his laboratory, the human mind seeks relationships between events and often finds them even when they are not present.
Representativeness: Aristotle: The sum of the coincidences equals certainty.’ We forget most of the insignificant coincidences and remember the meaningful ones. … [We] remember hits and ignore misses.
Emotive Words and False Analogies: Emotive words are used to provoke emotion and sometimes to obscure rationality. Likewise, metaphors and analogies can cloud thinking with emotion or steer us onto a side path.
Ad Ignorantiam: This is an appeal to ignorance or lack of knowledge and is related to the burden of proof and unexplained is not inexplicable fallacies, where someone argues that if you cannot disprove a claim it must be true.
Ad Hominem and Tu Quoque: …these fallacies redirect the focus from thinking about the idea to thinking about the person holding the idea. Calling someone an atheist, a communist, a child abuser, or a Neo-Nazi does not in any way disprove that person’s statement. …Similarly, for tu quoque. If someone accuses you of cheating on your taxes, the answer, ‘Well, so do you,’ is no [???] way or the other.
Hasty Generalization: In logic, the hasty generalization is a form of improper induction. In [???] it is called prejudice… Conclusions are drawn before the facts warrant it. …this fallacy is one of the most common of all. A handful of members of a group are used to judge the entire group.
Overreliance on Authorities: We tend to rely heavily on authorities in our culture, especially if the authority is considered to be highly intelligent. The IQ score has acquire nearly mystical proportions in the last half century, but I have noticed that belief in the paranormal is not uncommon among MENSA members; some even argue that their ‘Psi-Q’ is also superior.
Either-Or: Fallacy of negation or the false dilemma, this is the tendency to dichotomize the world so that if you discredit one position, the observer is forced to accept the other. A new theory needs evidence in favour of it, not just against the opposition.
Circular Reasoning: Fallacy or redundancy, begging the question or tautology, this occures when the conclusion or claim is merely a restatement of one of the premises. ‘Is there a God? Yes. How do you know? Because the Bible says so. How do you know the Bible is correct? Because it was inspired by God.’ Science also has its share of redundancies: What is gravity? The tendency for objects to be attracted to one another, [???] are objects attracted to one another? Gravity.’
Effort Inadequacies and the Need for Certainty, Control and Simplicity: Most of us most of the time, want certainty, and want nice, near, simple explanations., Alfred Mander in Logic for the Millions: ‘People with untrained minds should no more expect to think clearly and logically than people who have never learned and never practised can expect to find themselves good carpenters, golfers, bridge players or pianists.’ We [???] always work to suppress our need to be absolutely certain and in total control and our tendency to seek the simple and effortless solution to a problem.
Problem-Solving Inadequacies: Psychologist Barry Singer has demonstrated that when people are given the task of selecting the right answer to a problem after being told whether particular guesses are right or wrong, they: a) Immediately form a hypothesis and look only for examples to confirm it. B) Do not seek evidence to dispose the hypothesis. C) Are very slow to change the hypothesis even when it is obviously wrong. D) If the information is too complex, adopt an overly-simple hypothesis or strategies for solutions. E) If there is no solution, if the problem is a trick and ‘right’ and ‘wrong’ is given at random, form hypothesis about [???] relationships they observed. Causality is always found.
Ideological Immunity, or the Planck Problem: In day-to-day life, as in science, we all resist fundamental paradigm change… the more knowledge individuals have accumulated , and the more well- [???] the theories have become (and remember, we all tend to look for and remember confirmatory evidence, not [???] evidence), the greater the confidence in their idealogies. The consequences of this, however, is that we build up ‘immunity’ against new ideas that do not corroborate previous ones. Historians of science call this the Planck Problem, after physicist Max Planck, who made this observation on what must happen for innovation to [???] science: ‘An unrepentant scientific innovation rarely makes its way by gradually winning over and converting its opponents: What does happen is that its opponents gradually die out and that the growing generation is familiarized with the idea from the beginning.’ Psychologist David [Perkies] conducted an interesting correlational study in which he found a strong positive correlation between intelligence (measured by a standard IQ test) and the ability to give reasona for taking a point of view and defending that position he also found a strong negative correlation between intelligence and the ability to consider other alternatives.
Spinoza’s Dictum: Skeptics have the very human tendency to relish debunking what we already believe to be nonsense. It is for this reason that it is so important for us to understand the history of both science and pseudoscience. If we see the larger picture of how these movements evolve and [???] and how their thinking went wrong, we won’t make the same mistakes. Spinoza: I have made a ceaseless effort not to ridicule, not to bewail, not to scorn human actions but to understand them.
William of Occam: No more things should be presumed to exist than are absolutely necessary.
Adam Smith: Science is the great antidote to the poison of enthusiasm and superstition.
…Still, without Brouwer’s suggestion I would never have gotten myself to engage intensively with your book because it is written in a language inaccessible to me. The word God is for me nothing more than the expression and product of human weakness, the Bible a collection of honorable, but still purely primitive, legends which are nevertheless pretty childish. No interpretation no matter how subtle can (for me) change this. … For me the Jewish religion like all other religions is an incarnation of the most childish superstition. And the Jewish people to whom I gladly belong … have no different quality for me than all other people. As far as my experience goes, they are also no better than other human groups, although they are protected from the worst cancers by a lack of power. Otherwise I cannot see anything “chosen” about them….”
In January of 1954, just a year before his death, Albert Einstein wrote the following letter to philosopher Erik Gutkind after reading his book, ‘ Choose Life: The Biblical Call to Revolt ‘. Apparently Einstein had only read the book due to repeated recommendation by their mutual friend Luitzen Egbertus Jan Brouwer .
Einstein penned the letter on January 3 1954 to the philosopher Eric Gutkind who had sent him a copy of his book Choose Life: The Biblical Call to Revolt. The letter went on public sale a year later and has remained in private hands ever since.
In the letter, he states: “The word god is for me nothing more than the expression and product of human weaknesses, the Bible a collection of honourable, but still primitive legends which are nevertheless pretty childish. No interpretation no matter how subtle can (for me) change this.”
Einstein, who was Jewish and who declined an offer to be the state of Israel’s second president, also rejected the idea that the Jews are God’s favoured people.
“For me the Jewish religion like all others is an incarnation of the most childish superstitions. And the Jewish people to whom I gladly belong and with whose mentality I have a deep affinity have no different quality for me than all other people. As far as my experience goes, they are no better than other human groups, although they are protected from the worst cancers by a lack of power. Otherwise I cannot see anything ‘chosen’ about them.”