Training Public International Law: 5. Evaluate results

It is obvious that you should evaluate the relevance of your sources. You evaluate their scientific nature to increase the reliability of your own piece. Indirectly reliability is also determined by the kind of sources you use. If the scholarly nature of your sources is beyond doubt, your paper itself will less likely be doubted.

To determine if a publication or source is relevant, you can try to answer the following questions:

1. Does the source help you to answer your main questions and sub-questions?
2. Does the source answer your whole question/sub-question or only one aspect?
3. To what extent does the main question of the source you found match with your own questions?
4. How strong are the similarities between the research object or the analysis unit in the publication you found and those in your own paper/thesis? The research object may be a period, or a person, a group, an area, a substance, a disease, a process etc.
5. Is the context of the research object the same as in your case?
6. When was the source published and when was the research written about executed?

Think that you will rarely find a source that provides a complete answer to your main questions and sub-questions and that gives a report of the exact same research or problem you are working on. 

If you also want to use information from websites other than regular scholarly articles and books you should be extra careful and think about the role you give that information in your argument or analysis.

Ask yourself the following questions when reading webpages and be extra careful if the answer is negative most of the times.

1. Is the name of the author/maker available (and do you know more about the author or organisation)?
2. Can you find an e-mail address of the author/maker?
3. Is the webpage free of (lots of) advertisements?
4. Is the use of language careful, not childish, correct?
5. Is it clear how the information on the page came about?
6. Are sources mentioned (so no phrases like "research shows that") without any quotation of sources?
7. Are claims well-founded (so no phrases like "as is common knowledge")?
8. Is it a balanced piece or does all information point in the same direction?
9. Is the author open about matters still unknown or uncertain?
10. Can you find when the page was written or updated?
11. Is the page unbiased or in any case without strongly political or commercial aims?

The crux of science lies in the extent to which an author/researcher performs his work objectively and makes it verifiable. In determining the quality and scientific nature of sources you may start from three kinds of checks:

1. Check by others, preceding publication
   • editors: editors of scientific journals are stricter than editors of non-scientific journals
   • publishers: some publishers only publish scientific books
   • peer review: most scholarly journals but also some book publishers ask experts for a (single or double blind) judgement before publication
   • search engine/online bibliography: some search engines only include articles from high-quality, peer reviewed journals (for instance Scopus and Web of Science
   • financers: some journals demand to know who funded the research
2. Check by others, after publication
   • reviews (in the case of books): are the reviews positive?
   • citations (mainly in the case of articles): is the piece cited often (taking into account how long it has been available) and more importantly:what is being said?
   • open annotation and post publication peer review in systems like PubPeer and (for preprints) Peer Community In and comments at the preprint platforms themselves.
3. Check by yourself
   • are author and date of the text mentioned (particularly in the case of webpages)?
   • affiliation of the author: the author's job may tell you more about the scientific level, for instance if the author is employed by a good university
   • is the target group mentioned: (in particular in the case of websites and reports)
   • presence of explicit research questions and conclusions
   • presence of an explanation of the method used: how was the research conducted, where are the data coming from?
   • presence of enough and high-quality literature references or notes: what insights are used?
   • language level and well organised text