Skip to main content
Universiteitsbibliotheek – LibGuides

Training information skills for Sustainable Development Master in GEO4-2328: 5. Evaluate

Analysing governance practices

EVALUATING YOUR SOURCES

  1. Have a look at the info on the right.
  2. Find out the affiliation (where people work) of the authors of the one of the books you found. Also check what else they have written (books or journal articles).
  3. Find out whether the first article you found in Google Scholar is published in a journal that is peer reviewed or not. You can often find this on the journal website, but also in the list of journals on the library site.
  4. Using the full text of the last article you found evaluate:
    • the quality of the references cited in this article;
    • whether the article explicitly mentions methods used or the way data or sources used in the article/research were collected.

Evaluating the relevancy of your search results

You can determine the relevancy of search results by asking these questions:

  1. Does the source answer your central research question or your minor related questions?
  2. Does the source answer the full question or just one aspect of it?
  3. How strong are the similarities between the questions raised in the source found and those in your paper/thesis?
  4. How strong are the similarities between the units of research and analysis in the source found and those of your own research. The unit of research may be a (historical) period, a person, a group, an area or country, a natural phenomenon, a process etc..
  5. Is the context of the research unit in the source the same as in your own research/paper?
  6. When has the source been published and when has the research on which the piece is based been carried out?

You have to realise that only seldom you will find a source exactly matching all your own questions. You will have to combine information from many sources to answer your questions.

The special LibGuide evaluating sources has more on how to evaluate information.

Evaluating search results: how scientific are they?

The  reliability  (scientific nature) of sources can be verified by three kinds of checks:

  1. Check by others, before publication
    • editors: editors of scientific journals are stricter than editors of non-scientific journals
    • publisher: some publishers only publish scientific books
    • peer review: some journals but also some book publishers ask experts for a (blind) judgment before publication
    • search engine/online bibliography: some search engines only include articles from high-quality peer reviewed journals (for instance Scopus and Web of Science) 
    • financier: some journals demand that the names are published of those who have funded the research
  2. Check by others, after publication
    • reviews (in the case of books): is the book review positive?
    • citations (particularly in the case of articles:): how many times is the article cited and especially; what is said about the article?
  3.  Your own check:
    • who is the author and when was the article published (especially with web pages)
    • affiliation of the author: the job may tell you more, for instance if the author is employed by a (good) university
    • what is the intended audience of the publication (for websites and reports) 
    • how explicit is the phrasing of the question? Does the article contain conclusions?
    • is the used method explained: how was the research organised, where do the data come from?
    • are there enough references? Are they of high quality?: on which insights is the theory based?
    • language use: level and grammaticality

In the special LibGuide Evaluating sources   you will learn how to deal with these matters

Peer review: the crux of scientific communication?

some publishers (e.g. Copernicus) experiment with new forms of peer reviewAccording to many, peer review, the system by which fellow experts judge the quality of a piece before publication is one of the pillars of science. Editors can never have enough expertise to judge submitted articles or books. Experts can. Without them knowing the identity of the author, they assess whether the piece meets the scientific norms: stating where the data originate from, a logical analysis, references to relevant sources. Besides they say what  the article has to contribute to science. Peer reviewing an article is more about the question of a piece being written according to a good scientific method than about reviewers agreeing with the content. The advice to the editors will be either publishable, publishable with certain adjustments/ not publishable.

The peer rewiew system is subject to much criticism, although many people say that is the 'best worst system' we  have. Criticism focusses on:

  • Delays: the system can delay publication for months, sometimes for over a year
  • Interests: sometimes the impartiality of reviewers is doubted: in small disciplines or specialisms it is often easy to find out who the author is. As a result, publicatons by others may be prevented or the reviewer, acting out of self-interest, may demand that the author refers to the articles of the reviewer
  • Some reviewers who are in great demand spend a lot of (unpaid) time on peer reviewing articles
  • Strongly innovative, experimantal research and the accompanying publications have less chance to be published because they deviate from the norm and will be deemed less publishable by reviewers
  • The English has to meet high standards and may cause problems for  non-native speakers. As a result, these demands may prevent international publication of research that may well be excellent.
  • Stating the interest for science is arbitrary

There are sites where authors share their experiences with the peer review of journals. The Dutch initiative SciRev is one of them.

Some journals, such as PLOS One (Public Library of Science One) are experimenting with new forms of peer review. Peer reviewers only indicate if the articles are written in the right method. Assessing the scientific interest is done at a later stage, by the readers. The use of this method leads to a speedier publication of many articles.