Critical analysis: Difference between revisions

From BurnZero
mNo edit summary
mNo edit summary
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[File:Dave in the pub.jpg|alt=Dave in the pub|thumb|'''Figure 1'''. Dave, the veteran, the legend, in the pub.]]
[[File:Dave in the pub.jpg|alt=Dave in the pub|thumb|'''Figure 1'''. Dave, the veteran, the legend, in the pub.]]
'''Information can be true or false. Information that tends to be true is information based in scientific fact. Information that tends to be false usually is [[Anecdotal Evidence Fallacy|anecdotal]], like the stories from your "ex-special forces" friend Dave at the pub''' (See '''Figure 1''' ''- we love Dave!'')'''.''' Everyone trusts their sources differently (Dave might actually be a rocket scientist!) but it's VERY difficult to tell if people are just making shit up. To make this process easier, some use a [[Referencing|cheat sheet]], based in science which everyone can use to point them in the right direction. Other than that, there are a series of Critical Analysis techniques which can help you figure it out for yourself.
'''Whilst 100% truth is a thing, like saying water is wet, the truth can sometimes be more ambiguous. That said, information that tends to be true is usually based on scientific fact and information that tends to be false usually is [[Anecdotal Evidence Fallacy|anecdotal]], like the stories from your "''ex-special forces''" friend Dave at the pub''' (See '''Figure 1''' ''- (we love Dave!)'')'''.''' Everyone trusts their sources differently (Dave might actually be a rocket scientist!) but it's VERY difficult to tell if people are just making shit up. To make this process easier, some use [[Referencing|cheat sheets]] based on science and reputation which anyone can use to point them in the right direction. However, again these sheets are potentially made by a [[Cognitive Biases|bias]] source. Instead, you can do it yourself using a series of critical analysis techniques...
==Critical Analysis Techniques==
==Critical Analysis Techniques==
[[File:Hierarchy of Sciences.png|alt=Hierarchy of Sciences|thumb|'''Figure 2'''. Hierarchy of Sciences]]
[[File:Hierarchy of Sciences.png|alt=Hierarchy of Sciences|thumb|'''Figure 2'''. Hierarchy of Sciences]]
Line 8: Line 8:
|-
|-
|<small>~ '''Ernest Rutherford''', a man who is considered the father of nuclear physics.</small>
|<small>~ '''Ernest Rutherford''', a man who is considered the father of nuclear physics.</small>
|}As everything is made of atoms, it is arguable that every science is essentially a child of physics as physics is based on the most elementary particles, using the most elementary form of analysis, mathematics. This can be depicted in a hierarchical sense ('''Figure 2''') the further down the hierarchy you move the more complex and multifaceted the science becomes. For instance, a mathematical proof of the Higgs Boson is provable by mathematics, whereas in social sciences because there are so many interlaced factors it is more difficult to get to the absolute root cause.
|}[[File:Weighting hierarchy.png|alt=Weighting hierarchy|thumb|'''Figure 3:''' Scientific weighting hierarchy.]]As everything is made of atoms, it is arguable that every science is essentially a child of physics as physics is based on the most elementary particles, using the most elementary form of analysis, mathematics. This can be depicted in a hierarchical sense ('''Figure 2''') the further down the hierarchy you move the more complex and multifaceted the science becomes. For instance, a mathematical proof of the Higgs Boson is provable by mathematics, whereas in social sciences because there are so many interlaced factors it is more difficult to get to the absolute root cause.


Primarily, the information we consume should be unbiased and as factual as possible ('''Figure 3'''). The closest and most abundant source of this are scientific journal articles. Articles go through an approval process, the editor receives an article then sends it out to other scientists, who provide feedback and recommend whether the paper should be published. This practice only became a requirement by most journals in the 1970s and ’80s and reduces poor science by:
Primarily, the information we consume should be unbiased and as factual as possible ('''Figure 3'''). The closest and most abundant source of this are scientific journal articles. Articles go through an approval process, the editor receives an article and then sends it out to other scientists, who provide feedback and recommend whether the paper should be published. This practice only became a requirement by most journals in the 1970s and ’80s and reduces poor science by:
*reducing information overload by eliminating irrelevant or weak studies.
*[[File:The reproducibility Crisis.png|alt=The reproducibility Crisis|thumb|'''Figure 4.''' All scientific fact should be testable as true by a third party.]]reducing information overload by eliminating irrelevant or weak studies.
*identifying the most relevant papers.[[File:Weighting hierarchy.png|alt=Weighting hierarchy|thumb|'''Figure 3:''' Scientific weighting hierarchy.]]
*identifying the most relevant papers.
*distinguishing evidence from opinion, assumptions, misreporting, and belief.
*distinguishing evidence from opinion, assumptions, misreporting, and belief.
*assessing the validity of the study.
*assessing the validity of the study.
*assessing the usefulness and clinical applicability of the study.
*assessing the usefulness and clinical applicability of the study.
This is illustrated in '''Figure 4''' and is the key filtering technique that Burnzero uses to write articles. In fact on every page, at the bottom is a section named '''Reference''', by clicking on the link next to the individual referenced article ([1], [2], [3] etc) you will be taken to an abstract or full article which is only featured if it fits within the higher levels of this hierarchy. Therefore, no news sources are used, only verifiable scientific papers, the intention of this is to bring us closer to fact than fiction.


=== Replication Crisis ===
The [[Replication Crisis|replication crisis]] refers to the fact that the majority of scientific findings published in journals cannot be reproduced by another scientist<ref>'''Why most published research findings are false'''. PLoS Med. 2005 Aug;2(8):e124. doi: [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ 10.1371/journal.pmed.0020124]. Epub 2005 Aug 30. Erratum in: PLoS Med. 2022 Aug 25;19(8):e1004085. PMID: 16060722; PMCID: PMC1182327.</ref>. This is contrary to the core of science which insists on reproducibility and puts a big dent in the credibility of many studies, i.e. it is expected that if one scientist says something is fact by the use of a methodology this methodology should be able to be used by a third party and the results reproduced (See '''Figure 4'''). Given this, studies which sit at the top of the scientific weighting hierarchy ('''Figure 3''') plus have been shown to be reproducible by an unbiased third party should be considered the best knowledge available.
=== Best Practice Methodology ===
Taking all of this into account, Burnzero uses a [[referencing]] system to try and get to as close to scientific truth as possible. In fact, on every page, at the bottom is a section named "'''''Reference''"''', by clicking on the link next to the individual referenced article ([1], [2], [3] etc) you will be taken to an abstract or full article which is only featured if it fits within the higher levels of this hierarchy - which are then graded based on the quality of their evidence as A, B, C or D (See figure 5). Therefore, no news sources are used, only verifiable scientific papers, the intention of this is to bring us closer to fact than fiction.
[[File:Reference Rating.png|alt=Reference Rating|thumb|'''Figure 5'''. A-E quality measurement system used on Burnzero.]]
'''If you would like to learn more on this subject, you can join one of The Transparent Company's [https://prepare.online/critical-analysis.html Critical Analysis Course] via [https://prepare.online/ Prepare.Online].'''
'''If you would like to learn more on this subject, you can join one of The Transparent Company's [https://prepare.online/critical-analysis.html Critical Analysis Course] via [https://prepare.online/ Prepare.Online].'''
'''References'''

Latest revision as of 23:31, 7 November 2023

Dave in the pub
Figure 1. Dave, the veteran, the legend, in the pub.

Whilst 100% truth is a thing, like saying water is wet, the truth can sometimes be more ambiguous. That said, information that tends to be true is usually based on scientific fact and information that tends to be false usually is anecdotal, like the stories from your "ex-special forces" friend Dave at the pub (See Figure 1 - (we love Dave!)). Everyone trusts their sources differently (Dave might actually be a rocket scientist!) but it's VERY difficult to tell if people are just making shit up. To make this process easier, some use cheat sheets based on science and reputation which anyone can use to point them in the right direction. However, again these sheets are potentially made by a bias source. Instead, you can do it yourself using a series of critical analysis techniques...

Critical Analysis Techniques

Hierarchy of Sciences
Figure 2. Hierarchy of Sciences
‘All science is either physics or stamp collecting’
~ Ernest Rutherford, a man who is considered the father of nuclear physics.
Weighting hierarchy
Figure 3: Scientific weighting hierarchy.

As everything is made of atoms, it is arguable that every science is essentially a child of physics as physics is based on the most elementary particles, using the most elementary form of analysis, mathematics. This can be depicted in a hierarchical sense (Figure 2) the further down the hierarchy you move the more complex and multifaceted the science becomes. For instance, a mathematical proof of the Higgs Boson is provable by mathematics, whereas in social sciences because there are so many interlaced factors it is more difficult to get to the absolute root cause.

Primarily, the information we consume should be unbiased and as factual as possible (Figure 3). The closest and most abundant source of this are scientific journal articles. Articles go through an approval process, the editor receives an article and then sends it out to other scientists, who provide feedback and recommend whether the paper should be published. This practice only became a requirement by most journals in the 1970s and ’80s and reduces poor science by:

  • The reproducibility Crisis
    Figure 4. All scientific fact should be testable as true by a third party.
    reducing information overload by eliminating irrelevant or weak studies.
  • identifying the most relevant papers.
  • distinguishing evidence from opinion, assumptions, misreporting, and belief.
  • assessing the validity of the study.
  • assessing the usefulness and clinical applicability of the study.

Replication Crisis

The replication crisis refers to the fact that the majority of scientific findings published in journals cannot be reproduced by another scientist[1]. This is contrary to the core of science which insists on reproducibility and puts a big dent in the credibility of many studies, i.e. it is expected that if one scientist says something is fact by the use of a methodology this methodology should be able to be used by a third party and the results reproduced (See Figure 4). Given this, studies which sit at the top of the scientific weighting hierarchy (Figure 3) plus have been shown to be reproducible by an unbiased third party should be considered the best knowledge available.

Best Practice Methodology

Taking all of this into account, Burnzero uses a referencing system to try and get to as close to scientific truth as possible. In fact, on every page, at the bottom is a section named "Reference", by clicking on the link next to the individual referenced article ([1], [2], [3] etc) you will be taken to an abstract or full article which is only featured if it fits within the higher levels of this hierarchy - which are then graded based on the quality of their evidence as A, B, C or D (See figure 5). Therefore, no news sources are used, only verifiable scientific papers, the intention of this is to bring us closer to fact than fiction.

Reference Rating
Figure 5. A-E quality measurement system used on Burnzero.

If you would like to learn more on this subject, you can join one of The Transparent Company's Critical Analysis Course via Prepare.Online.

References

  1. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. doi: 10.1371/journal.pmed.0020124. Epub 2005 Aug 30. Erratum in: PLoS Med. 2022 Aug 25;19(8):e1004085. PMID: 16060722; PMCID: PMC1182327.

Share your opinion