Critical analysis

Figure 1. Dave, the veteran, the legend, in the pub.

Whilst 100% truth is a thing, like saying water is wet, the truth can sometimes be more ambiguous. That said, information that tends to be true is usually based on scientific fact and information that tends to be false usually is anecdotal, like the stories from your "ex-special forces" friend Dave at the pub (See Figure 1 - (we love Dave!)). Everyone trusts their sources differently (Dave might actually be a rocket scientist!) but it's VERY difficult to tell if people are just making shit up. To make this process easier, some use cheat sheets based on science and reputation which anyone can use to point them in the right direction. However, again these sheets are potentially made by a bias source. Instead, you can do it yourself using a series of critical analysis techniques...

Critical Analysis Techniques

Figure 2. Hierarchy of Sciences

‘All science is either physics or stamp collecting’

~ Ernest Rutherford, a man who is considered the father of nuclear physics.

Figure 3: Scientific weighting hierarchy.

As everything is made of atoms, it is arguable that every science is essentially a child of physics as physics is based on the most elementary particles, using the most elementary form of analysis, mathematics. This can be depicted in a hierarchical sense (Figure 2) the further down the hierarchy you move the more complex and multifaceted the science becomes. For instance, a mathematical proof of the Higgs Boson is provable by mathematics, whereas in social sciences because there are so many interlaced factors it is more difficult to get to the absolute root cause.

Primarily, the information we consume should be unbiased and as factual as possible (Figure 3). The closest and most abundant source of this are scientific journal articles. Articles go through an approval process, the editor receives an article and then sends it out to other scientists, who provide feedback and recommend whether the paper should be published. This practice only became a requirement by most journals in the 1970s and ’80s and reduces poor science by:

• 

  Figure 4. All scientific fact should be testable as true by a third party.
  reducing information overload by eliminating irrelevant or weak studies.
• identifying the most relevant papers.
• distinguishing evidence from opinion, assumptions, misreporting, and belief.
• assessing the validity of the study.
• assessing the usefulness and clinical applicability of the study.

Replication Crisis

The replication crisis refers to the fact that the majority of scientific findings published in journals cannot be reproduced by another scientist. This is contrary to the core of science which insists on reproducibility and puts a big dent in the credibility of many studies, i.e. it is expected that if one scientist says something is fact by the use of a methodology this methodology should be able to be used by a third party and the results reproduced (See Figure 4). Given this, studies which sit at the top of the scientific weighting hierarchy (Figure 3) plus have been shown to be reproducible by an unbiased third party should be considered the best knowledge available.

Best Practice Methodology

Taking all of this into account, Burnzero uses a referencing system to try and get to as close to scientific truth as possible. In fact, on every page, at the bottom is a section named "Reference", by clicking on the link next to the individual referenced article ([1], [2], [3] etc) you will be taken to an abstract or full article which is only featured if it fits within the higher levels of this hierarchy - which are then graded based on the quality of their evidence as A, B, C or D (See figure 5). Therefore, no news sources are used, only verifiable scientific papers, the intention of this is to bring us closer to fact than fiction.

Figure 5. A-E quality measurement system used on Burnzero.

If you would like to learn more on this subject, you can join one of The Transparent Company's Critical Analysis Course via Prepare.Online.

References