LibGuides: Master ISS literature search: Selecting and evaluating sources

Index

Selecting sources - Peer review - How to recognize a peer reviewed article - Determine relevancy - Determine scholarly nature of sources - Crapmap (checklist)

selecting sources

How do you determine whether an item is suitable for you to use?

Start with the abstract and possibly the keywords present in the article. From this you can often deduce whether the article is relevant to you or not. Try to answer the following questions: what are the important concepts and/or theories of the article and do they relate to your research question/topic? Does the article only tangentially touch on your topic or is it really the core? And does it have a similar context (e.g. target group, country)?
If you are unable to find an abstract, or if it is not available, open the full text. Is the article in a language that is unreadable for you, or is the full text not available? Then this source is immediately eliminated. If you can read the article, start with the introduction and look at the headings and first sentences of the paragraphs, look for tables or graphics and see what they are about and read the conclusions. Based on this you can quickly get a good impression of the content of an article.
Do you recognize the following elements in the article: a problem or question, (scientific) method used, findings or results, discussion, most important conclusions/arguments and references to scientific literature (in the bibliography or notes)?
Also look outside the article. Has it appeared in a scientific peer-reviewed journal? Or is it from a scientific book/ edited volume? Take a look at where the authors work. Is the author a journalist or a scientist and affiliated with a scientific institution? What else has the author published?
By scanning/skimming an article and it's context in this way you can quickly get an impression of the usefulness of the article.

Would you like to know more about relevance and scientific reliability? Then take a look at the boxes below and our LibGuide on evaluating sources.

Note: not all your sources always have to be academic, sometimes you can also use reports or documents from government institutions.

Peer review

Peer review is the evaluation of work by one or more people with similar competences as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility. In academia, scholarly peer review is often used to determine an academic paper's suitability for publication.

source: Wikipedia

How to determine the relevance of sources?

In determining if a source is relevant, you may try to answer the following questions:

Does the source help you to answer your main questions and sub-questions?
Does the source answer your whole question/sub-question or only one aspect?
To what extent does the main question of the source you found match with your own questions?
How strong are the similarities between the research object or the analysis unit in the source you found and those in your own paper/thesis? The research object may be a period, or a person, a group, an area, a substance, a disease, a process etc.
Is the context of the research object the same as in your case?
When was the source published and when was the research that was written about executed?

Think that you will rarely find a source that provides a complete answer to your main questions and sub-questions and that gives a report of the exact same research or problem you are working on.

Need to see the full text of an article? The next page is about how to get to the full text.

How do you recognize a peer reviewed article

You often cannot see in the article itself whether an article has been peer-reviewed or not. But how do you find out?
You can use the tips below:

If you are lucky, the article will say: 'published in a peer reviewed section of the journal' or something similar. In that case you will know soon enough.
You can check whether the journal has a peer review procedure. This is often stated on the homepage of the journal itself, sometimes under the heading 'instructions for authors' or 'instructions for reviewers'. For a quick check, you can also search Google for the title of the journal in combination with words such as 'peer reviewed' or 'refereed', but if you don't find anything this way, it does not mean that the journal does not use peer review.
An article can be published in a peer-reviewed journal, but not by itself be peer-reviewed. Consider, for example, book reviews, case descriptions or editorials. If you are unsure whether an article is suitable, also check whether the section/section of the journal in which the article was published is a peer-reviewed section (see the 'about section' of the journal).
Some databases, such as Scopus, Web of Science and many subject-specific databases (for example PsycInfo or PubMed), apply a strict selection and only include peer-reviewed journals. Some databases also offer the option to filter on peer-reviewed articles. This also allows you to filter out (so not include) books or chapters from books, among other things.

How to determine the scholarly nature of sources?

The crux of science lies in the extent to which an author/researcher performs his work objectively and makes it verifiable. In determining the quality and scientific nature of sources you may start from three kinds of checks:

Check by others, preceding publication
- editors: editors of scientific journals are stricter than editors of non-scientific journals
- publishers: some publishers only publish scientific books
- peer review: some journals but also some book publishers ask experts for a (blind) judgement before publication
- search engine/online bibliography: some search engines only include articles from high-quality, peer reviewed journals (for instance Scopus and Web of Science
- financiers: some journals demand to know who funded the research
Check by others, after publication
- reviews (in the case of books): are the reviews positive?
- citations (mainly in the case of articles): is the piece cited often (taking into account how long it has been available) and more importantly:what is being said?
Check by yourself
- are author and date of the text mentioned (particularly in the case of webpages)?
- affiliation of the author: the author's job may tell you more about the scientific level, for instance if the author is employed by a good university
- is the target group mentioned: (in particular in the case of websites and reports)
- presence of explicit research questions and conclusions
- presence of an explanation of the method used: how was the research conducted, where are the data coming from?
- presence of enough and high-quality literature references or notes: what insights are used?
- language level and well organised text

You can find more information in the Libguide Evaluating Sources

Checklist for sources: the CRAPMAP

The CRAPMAP consists of a list of questions that help you to assess whether the (online) information you have found is reliable and useful. Depending on your situation, items from the list are more or less important. NB! Always stay alert! This checklist is just a tool and not always sufficient.

Currency
The timeliness of the information

 When was the information published or posted?
 Has the information been revised or updated? Or perhaps even retracted?
 Does your topic require current information, or will older sources work as well?

Relevance
The importance of the information for your needs

 Does the information relate to your topic or answer your questions?
 Who is the intended audience?
 Is the information at an appropriate level (not too simple or too advanced)
 Have you looked at a variety of sources before determining this is one you will use?
 Would you be comfortable citing this source in your research paper?

Accuracy
The reliability, truthfulness and correctness of the content

 Where does the information come from?
 Is the information supported by evidence?
 Has the information been reviewed or refereed?
 Can you verify any of the information in another source or from personal knowledge?
 Does the language or tone seem unbiased and free of emotion?
 Are there spelling, grammar or typographical errors? [Second task or achievement]

Publication
The source of the information

 Who is the author/publisher/source/journal/sponsor?
 What are the author's credentials or organizational affiliations?
 Is the author qualified to write on the topic?
 Is the publisher/journal scientific? Are they reputable?
 Is there contact information, such as a publisher or e-mail address?

Metrics*
The impact a source has on the scientific field

 Are you able to find metrics for the source?
 Is the source cited by current research?

Altmetrics*
The impact a source has in general

 Are you able to find altmetrics for the source?
 Is your research impacted by how well the source is received?
 What does the public reception of a source indicate?

* For information about metrics and altmetrics see the LibGuide Research Impact & visibility: traditional and altmetrics

Purpose
The reason the information exists.

 What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade?
 Do the authors/sponsors make their intentions or purpose clear?
 Is the information fact, opinion or propaganda?
 Does the point of view appear objective and impartial?
 Is there political, ideological, cultural, religious, institutional, financial or personal bias?

This checklist is based on the CRA(A)P test, originally developed by the library of the California State University, Chico