3 Chapter 3 (Evaluating and Using Sources)

Basic Review

Unfortunately, just because a source came up in a search and it has a few of its own sources, that doesn’t mean it’s right for your purposes. All sources need to be evaluated. This is especially important for sources you find on the internet, where it’s easier to publish with no oversight, but it’s important to evaluate the sources you find through the library, too. Evaluation isn’t just about whether a source is “good” or “bad;” it’s also about determining whether a source is the best choice for your particular information need. When you begin evaluating sources, what should you consider?

Transparency: Does the source tell you about itself? A good source will be clear on who wrote it, on why it was written, and on where the information it provides comes from; it might provide links to its own sources directly or it might cite its sources, but it will provide this corroboration. If it’s an original study, it will let you know which studies informed it. Good sources also let you know about who provided any funding for the study.

Internal Consistency: Does the source meet its own standards? This includes the source providing clear definitions of its terms and sticking to them (i.e. not moving the goal posts) and also holding all parties to the same standard of evidence, not to varying standards depending on what they are saying. At a minimum, the burden of proof is always placed on those making an affirmative claim, not those denying the claim.

Independent Confirmation: Does the source match what others find? This is tricky, because we want to be open to new information. However, any time one source deviates significantly from the majority of the other sources, it’s time to worry a little. The trick here is not to find another source that agrees but to find independent sources that agree. Imagine for the moment that I search for “Bees dying” and I find Harvard University’s website is reporting on a study performed by Chensheng Lu that bees are dying at an alarming rate. I then find that the Chicago Tribune says that bees are dying at an alarming rate, basing this on a write-up in the science journal Bulletin of Insectology. I go to the science journal Bulletin of Insectology and find the Harvard study conducted by Lu.

Guess what? Even though I found it in three different places, I only found one source. All three are really just different versions of the Lu study. Worse, it’s a study that’s pretty thoroughly discredited. Students curious about exactly how bad it is should read Jon Entine’s essay “Bee Experts Dismantle Touted ‘Harvard’ Neonics-Colony Collapse Disorder Study As ‘Activist Science.,” which was published in HuffPost. It’s a tour de force of refutation. However, even without that extra bit of reading, a little bit of hunting around reveals that the study is hotly contested.

TRAAP

Some students should be familiar with “TRAAP guidelines”, a series of prompts designed to help students remember easy-to-check aspects of source evaluation. The total list stands for Timeliness, Relevance, Authority, Accuracy, and Purpose.

Timeliness refers to the need to make certain that the source is actually current—or at least current enough for the argument’s needs. For example, an economics source that wanted to evaluate the impact of regulation on the banking industry that was written before 2000 would completely be missing analysis from the 2008 housing market crisis, the 2010 regulatory efforts, and the 2018 rollback of regulatory efforts. Knowing when something was written is also important in understanding the specific issues being discussed. A news article that talks about “the current presidential administration” could be talking about a different administration! Evaluating timeliness requires students to know whether or not the older information has become outdated, and that means knowing how readily information changes in a given field.

Relevance refers to extent to which the source is linked to the subtopic involved. A source might be on the general topic (for example, college tuition), but it might be on a subtopic that is not useful to the researcher’s specific needs. Imagine a student writing about college tuition who is writing about publicly funded state schools in Illinois, but whose source deals largely with private schools or with the California university system. Note that this is not an excuse to disregard information the student doesn’t like!

Authority as a prompt addresses whether or not the person (or group, or organization) writing the source is actually someone who might reasonably know about the subject. It’s a good idea to check out each author and look for reasons to disqualify the author as someone who has a conflict of interest (e.g. a person who runs a vitamin business who tells you that vitamins are good for you) or who is unlikely to know or understand the technicalities of the information at hand (e.g. a professional football player giving advise on cryptocurrency). Note that students need to be careful of arguments made from authority. In other words, someone having meaningful credentials or being a popular source on a subject does not mean that person should be listened to. No source is valid simply because of who wrote it. However, a source can be questionable because of who contributed to it.

Accuracy is a much more complex topic. Sources should be evaluated for whether or not they leave out information make claims without evidence (see Chapter 2).

Purpose addresses knowing why source was created. If the purpose of the website is to sell a product, like vitamins, then claims about the benefits of vitamins or the failings of typical diets are automatically suspect. A source with an obvious bias or prejudice should be suspect, and this will frequently involve the researcher doing a bit of checking on the political, economic, and personal interests of the source.  If the source is too informal or humorous, there is a very good chance that accuracy has been sacrificed for tone.

Remember that a student who includes low-quality sources in their thought processes is accepting their errors. A source that is adequate for informing a discussion between friends is probably not enough to support a college essay, and the source that fit a college essay just fine is completely inadequate for a report to a supervisor at work about a multi-million dollar project.

Putting Sources in Place

The Toulmin Model introduced earlier is one method of building arguments, and it is a model that can serve as a minimum framework for a complete paragraph. Imagine a “topic sentence” or a claim being made in a paragraph. Sample topic sentences or basic claims:

• Many people believe in Bigfoot
• Properly worn facemasks reduce the spread of respiratory diseases

Each of these claims might be accepted by some and rejected by others. Someone might think that the idea of Bigfoot is so ridiculous that only a few people believe in such a creature, while others might think that facemasks have little benefit. Each of these claims needs evidence to support it (or data). Mentioning the source of the evidence (either in a statement that explicitly lists the source or by referring to the source’s entry on a reference page) is a minimum step.

• According to CivicScience.com, 11% of Americans believe Bigfoot is a real creature.
• A study published in Nature magazine demonstrated that facemasks reduced the spread of viral agents in multiple respiratory illnesses, including influenza, SARS-CoV, and rhinovirus.

Finally, however, it is usually a good idea to explain how the evidence supports the claim, instead of allowing the reader to fill in the blanks.

• While the poll taken by CivicScience only surveyed a few thousand people, its methodology was sound.
• The results of the Nature study were independently confirmed by other researchers and match the results found when masks were used during the COVID-19 pandemic.

Thus, in order to have the basic components of an argument in place, even a weak paragraph should have all three elements together.

• Many people believe in Bigfoot. In fact, according to CivicScience.com, 11% of Americans believe Bigfoot is a real creature. This poll only surveyed a few thousand people, but its methodology was sound.
• Properly worn facemasks reduce the spread of respiratory diseases. A study published in Nature magazine demonstrated that facemasks reduced the spread of viral agents in multiple respiratory illnesses, including influenza, SARS-CoV, and rhinovirus. The results of this study were independently confirmed by other researchers and match the results found when masks were used during the COVID-19 pandemic.

Avoid Plagiarism

While plagiarism includes dramatic cases like students turning in fully copied papers or work that someone else wrote for them, plagiarism is not limited to deliberate and blatant cases of cheating. The most common form of plagiarism is when students include information that they learned from a source and then fail to give credit. Simply stealing someone else’s ideas (not just their words) constitutes plagiarism. For example, each of the following would both represent a form of plagiarism:

• 11% of Americans believe Bigfoot is a real creature.
• Facemasks reduced the spread of viral agents in multiple respiratory illnesses, including influenza, SARS-CoV, and rhinovirus.

Because these passages do not credit the CivicScience or Nature articles, they represent a form of plagiarism in a strict academic sense. Likewise, taking a passage from a source and changing out a few words (while preserving the original sentence structure and basic meaning) is still plagiarism. The best way to prevent plagiarism is to make certain that as you include specific information, you indicate where that information came from. Do not think you’ll remember where something came from and then try to go back later to fill in the blanks. Get the citations in place as you write.

As a cautionary note, students should remember that there are plenty of programs out there that claim to format sources for them. If they want to use them, that’s fine. Just remember that if they provide the wrong advice, it’s still the student’s grade that will suffer. Cross-checking and proof-reading applies to everything, including the student’s own work.

About Guidelines: So, the most commonly used citation systems for most college papers are MLA and APA. These initials stand for the Modern Language Association and the American Psychological Association. In both cases, these are professional organizations that put out their own style guides on how to cite sources, and then they charge people money to buy those style guides.

In other words, they make a bunch of rules that they say you have to follow, and the way you’re supposed to follow the rules is by paying the people who made the rules to teach them to you. This is pretty infuriating. Fortunately, there are a number of perfectly legal style guides out there that are pretty easy to find, and this packet will get you headed in the right direction.

The Basic Formula: Do not simply follow whatever guidelines a former teacher told you ‘was good enough.’ Instead, consult a handbook based on the teacher’s preferred citation system. However,  at a minimum presume that this basic formula is the least amount of information that needs to be included:

• Paraphrased information that is written by the author of the essay, but derived from another source (Patel 1).

This kind of information must still be cited in order to give proper credit. When in doubt, assume most claims need at least this minimum level of support.

• “Directly quoted information that is written by the source” (Smith 2).

Even if you are only quoting ‘a few words,’ use quotation marks for those words!

Giving Credit to a Source of Information: Here is the original passage as it appears in Carl Sagan’s The Demon-Haunted World on page 33:

At high speeds and strong gravities, Newtonian physics breaks down. This is one of the great findings of Albert Einstein’s Special and General Relativity, and is one of the reasons his memory is so greatly honored.

Here are two ways it could be paraphrased in an essay. Neither requires quotation marks, because they are not word-for-word copies. However, either does require a parenthetical source citation.

• Sagan claims that Einstein in revered, in part, because he found that Newton’s laws of physics were not absolute under certain conditions (33).
• Newtonian physics do not apply when speed and gravity reach certain extremes (Sagan 33).

The first example uses what is called a “signal statement,” and it signals the source within the body of the text. This is not strong enough to go across multiple sentences, but it does introduce a source clearly. The second example includes the full amount of information that is needed for a minimal citation, and it is strong enough to cover an entire passage from one source, so long as it is all in the same paragraph.

Do not think that specific information can go un-cited because ‘you already know that stuff.’ When in doubt, assume that this is the minimum level of support required for any claims made in an essay.

Giving Credit to the Author of a Direct Quotation: Here is another original passage as it appears in Carl Sagan’s The Demon-Haunted World, this time from on page 223:

Some claims are hard to test—for example, if an expedition fails to find the ghost or the brontosaurus, that doesn’t mean it doesn’t exist. Absence of evidence is not evidence of absence. Others are easier—for example, flatworm cannibalistic learning or the announcement that colonies of bacteria subjected to an antibiotic on an agar dish thrive when their prosperity is prayed for (compared to control bacteria unredeemed by prayer).

Here are different ways that a student might include direct quotations from this source:

• Sagan explains that “some claims are hard to test,” giving examples from parapsychology and cryptozoology (223).
• “If an expedition fails to find the ghost…that doesn’t mean it doesn’t exist” (Sagan 223).
• As even a leading skeptic admits, “absence of evidence is not evidence of absence” (Sagan 223).

If you don’t have an author, you can always cite a shortened version of the title. However, a better question is this—why are you trusting a source that won’t even tell you who wrote it?

 Indirect Quotation: Sometimes, we do not have the original source. We only have part of it that is quoted in the source we are using. Below is a passage from Plato that Sagan quotes on page 6 of The Demon-Haunted World:

All freemen, I conceive, should learn as much of these branches of knowledge as every child in Egypt is taught when he learns the alphabet. In that country arithmetical games have been invented for the use of mere children, which they learn as pleasure and amusement. I…have late in life heard with amazement of our ignorance in these matters; to me we appear to be more like pigs than men, and I am quite ashamed, not only of myself, but of all Greeks.

In an ideal world, I would have a chance to go check Plato and make sure that it says what Sagan claims. However, sometimes I am stuck using a source indirectly. To reflect the fact that I am now taking Carl Sagan’s word for it on what Plato says, I need to credit both authors.

• Plato was concerned with poor education in Athens, claiming that he was “ashamed, not only of myself, but of all Greeks” (qtd. in Sagan 6).
• Plato felt that every free man should learn math and science (Sagan 6).
• “In [Egypt], arithmetical games have been invented for the use of mere children” (Plato, qtd. in Sagan 6).

Bibliographic Information: When you finish the essay itself, you still need to put together the Works Cited page. The minimum information is usually as follows.

• Last name, first name. “Short Title.” Long Title. Publication data.

If you are missing a component, then you skip that step. Likewise, if you don’t have a name for the author, then the title replaces the name in the parentheses.

Remember to place all entries in alphabetical order. Do not arrange your bibliography in order of source use, or by using some arcane numbering system.

Be Careful

Most people are good at being critical of sources that deliver information they disagree with. Being able to be critical of all sources, even the ones that deliver welcome ideas, is an essential skill for a a college-level researcher.