Evaluating sources: Find out more

Peer review is the evaluation of the quality of a work by fellow experts before publication. Peer review methods are used to maintain quality standards, improve performance and provide credibility.

Library NCSU. This video is published under a Creative Commons 3.0 BY-NC-SA US license

If you also want to use information from websites other than regular scholarly articles and books you should be extra careful and think about the role you give that information in your argument or analysis.

Ask yourself the following questions when reading webpages and be extra careful if the answer is negative most of the times.

1. Is the name of the author/maker available (and do you know more about the author or organisation)?
2. Can you find an e-mail address of the author/maker?
3. Is the webpage free of (lots of) advertisements?
4. Is the use of language careful, not childish, correct?
5. Is it clear how the information on the page came about?
6. Are sources mentioned (so no phrases like "research shows that") without any quotation of sources?
7. Are claims well-founded (so no phrases like "as is common knowledge")?
8. Is it a balanced piece or does all information point in the same direction?
9. Is the author open about matters still unknown or uncertain?
10. Can you find when the page was written or updated?
11. Is the page unbiased or in any case without strongly political or commercial aims?

The criteria for selection and inclusion applied by a particular search engine may tell you something about the basic quality of the sources included.

What are the criteria used in search systems?

• Citations: the more often a particular source is cited, the better this source is represented in the search system. In such cases a citation is viewed as a kind of voice for the quality of the source. For instance in Web of Science citations of a source are an important criterion for selection and inclusion
• Advice of a 'content selection & advisory board': Some search engines include sources based on the advice of a group of experts. For instance Scopus and Web of Science.
• Peer reviews:  search systems may use peer review as a condition for including sources. This is the case for most scientific search engines. Peer review (blind or half blind) means that articles are presented to experts before publication. In this way a minimum of scientific quality is guaranteed. However, this does not mean that articles from peer reviewed sources are always better than the ones without peer review.
• Relevance:  library catalogues and subject-specific search engines include sources because of their relevance to education and research for the institution or discipline in question.
• Technical criteria: several search engines apply technical conditions for including sources. For instance Google Scholar also uses an automated system that recognizes texts as being scientific based on several elements.

The various types of databases and search engines differ in their scientific reliability.

General bibliographic databases: scientifically reliable

• There are general bibliographic (and citation) databases (e.g. Scopus and Web of Science) that only contain scientific material and peer reviewed articles in all disciplines.

Subject-specific bibliographic databases: scientifically reliable

• Subject-specific bibliographic databases (for instance PsycINFO, PubMed, Historical Abstracts) attempt to make scientific material in a particular discipline as broadly searchable as possible.

Large hybrid general search engines: scientific level not guaranteed

• Umbrella search systems such as Google Scholar are less trustworthy as to scientific quality, because the selection criteria are fairly crude and partly technical by nature. Many sources and databases are included so there is no certainty whatsoever about the scientific level

University library catalogues: scientifically relevant

• Material in university library catalogues is included because of its relevance for education and research of the university in question. This means that the catalogues may contain non-specialist and non-scientific sources which are nonetheless relevant.

The CRAPMAP consists of a list of questions that help you to assess whether the (online) information you have found is reliable and useful. Depending on your situation, items from the list are more or less important. NB! Always stay alert! This checklist is just a tool and not always sufficient.

Currency
The timeliness of the information

 When was the information published or posted?
 Has the information been revised or updated? Or perhaps even retracted?
 Does your topic require current information, or will older sources work as well?

Relevance
The importance of the information for your needs

 Does the information relate to your topic or answer your questions?
 Who is the intended audience?
 Is the information at an appropriate level (not too simple or too advanced)
 Have you looked at a variety of sources before determining this is one you will use?
 Would you be comfortable citing this source in your research paper?

Accuracy
The reliability, truthfulness and correctness of the content

 Where does the information come from?
 Is the information supported by evidence?
 Has the information been reviewed or refereed?
 Can you verify any of the information in another source or from personal knowledge?
 Does the language or tone seem unbiased and free of emotion?
 Are there spelling, grammar or typographical errors? [Second task or achievement]

Publication
The source of the information

 Who is the author/publisher/source/journal/sponsor?
 What are the author's credentials or organizational affiliations?
 Is the author qualified to write on the topic?
 Is the publisher/journal scientific? Are they reputable?
 Is there contact information, such as a publisher or e-mail address?

Metrics*
The impact a source has on the scientific field

 Are you able to find metrics for the source?
 Is the source cited by current research?

Altmetrics*
The impact a source has in general

 Are you able to find altmetrics for the source?
 Is your research impacted by how well the source is received?
 What does the public reception of a source indicate?

 * For information about metrics and altmetrics see the LibGuide Research Impact & visibility: traditional and altmetrics

Purpose
The reason the information exists.

 What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade?
 Do the authors/sponsors make their intentions or purpose clear?
 Is the information fact, opinion or propaganda?
 Does the point of view appear objective and impartial?
 Is there political, ideological, cultural, religious, institutional, financial or personal bias?

This checklist is based on the CRA(A)P test, originally developed by the library of the California State University, Chico

Fake news, fake messages, satire and hoaxes have been around for a long time, but their spread has been rapid since the rise of social media. Everyone can post content on the internet nowadays. As a result, the boundaries between news, fake news and advertisements seem to blur. Beware of using fake publications and satire in science. 

Not all journals pretending to be scientific act in good faith.

When you are searching for scientific articles in general search engines, you might end up on the website of a so-called predatory journal. These are journals that perform no or very bad quality checks in the form of peer reviews. Their aim is to attract authors who want to pay for quick publication of their articles. The websites of these journals sometimes have an amateur look, but often they are difficult to recognise as mala fide.

When in doubt, it is best to check for especially journals in the English language if the journal is included in scientific databases, such as Web of Science, Scopus or the Directory of Open Access journals. You can also check if the library has a subscription (digital or print) to the journal. If the journal does not tick these boxes, it does not have to be a predatory journal, but you must be more on your guard. Apart from the journal itself, you must always evaluate the contents of the journal article with the help of the tips in this guide.

 Please ask the library when you remain uncertain of the journal's reputation.