Author: Amor Pérez-Rodríguez – Translation: Erika-Lucia Gonzalez-Carrion

Evaluating a scientific text is a complex task that requires, on the one hand, knowledge of the subject matter of the text to be evaluated, but also professional ethics. Those who in some way are the gateway or the referees of knowledge in scientific publications must have a criterion […]

Evaluating a scientific text is a complex task that requires, on the one hand, knowledge of the subject matter of the text to be evaluated, but also professional ethics. Those who in some way are the gateway or referees of knowledge in scientific publications must have the most objective and solid criteria possible, based on experience, balance and rigor in their appraisals.

When a manuscript is submitted to a journal, the process for its approval or rejection involves a series of steps. First, formal aspects are evaluated, which are usually very visible and have to do, for example, with:

• Required files and templates.
  • Metadata: authorship, affiliation, orcid, abstract, keywords, section, funding.
  • Thematic and/or focus of the journal, call guidelines.
  • Referencing systems, anonymization.

This phase, on many occasions, means a withering rejection, especially when it is verified that the journal’s regulations or the previous check section have not been taken into account.  For the editorial board it is an indication that the journal is not known, that it has not been read, and that not much care has been taken in preparing the text. Articles may be rejected because they do not comply with the formal aspects (metadata, structure and length, number and accuracy of keywords, reference standards, etc.).

In relation to the object of study of the manuscript, many publications make a first analysis of the content, in a general way, to see if it fits the subject matter of the journal. Often manuscripts are submitted that are not related to the focus and, therefore, what proceeds is a rejection because the object of study does not fit the theme/focus. In this sense, journals clearly delimit their field, the type of studies they receive, or the models of research structure. If a text is sent that does not follow these indications, rejection is direct. Also cause for rejection in this first phase is that the work presents a small number of references, that these are outdated, that the research does not provide a solid methodology, that the sample is small and/or local, or that the conclusions are not original and significant.

If the submitted text passes this first phase, it goes to peer review. Many journals have very rigorous processes with several rounds in which experts evaluate the consistency of the work in order to meet the requirements of a scientific text, the research methodology and its structure.

The review reports are made with rigor, courtesy and constructive criticism. It is therefore important that their indications are accepted as an option for improvement and learning. Often the attention, justification and accurate answers to the recommendations and indications help the text to be definitively approved.

A review is based on a good reading of the text to be evaluated from an objective perspective. This implies that, from the experience and knowledge of the reviewer, the soundness of the theoretical framework and state of the art in terms of how it is presented, and the quality and quantity of information in accordance with the novelty of the contribution and the theoretical construct to be proposed, should be verified. The evaluations are made according to:

– How the title and abstract clearly expose the object of the work with an adequate structure.

– The relevance, timeliness and originality of the work in terms of the subject matter.

– The scope of the literature review.

In this sense, it is important to pay attention to the bibliographic references, varied, updated and coherent with the line of argumentation. It is not advisable to abuse self-citation or to cite unnecessarily. A good reviewer captures these circles and trends, and a bad reviewer biases his assessment by personal theoretical options.

• Varied and appropriate citations.
  • References used.

Another important aspect of the review focuses on the methodology used, as well as the results and discussions. Depending on the study and its scope, methodological rigor can be seen in how the research questions, objectives, hypotheses and, consequently, the type of research and/or model followed by the study are stated. All this must be related to what is stated in the state of the art. In addition, the definition of the instrument or instruments used, if applicable, their validation and piloting are another indication of a good methodological approach. Finally, the review should estimate the degree of progress in terms of the results and the discussion of them. Presenting the findings without commentary and discussion impoverishes the quality of a scientific text and limits the significance of what is contributed.

Regarding the conclusions, a review looks at the consistency of the results obtained, in accordance with the objectives set. It is usually penalized if the conclusions do not convincingly expose the achievements, and if they repeat ideas that are not deduced from what has been worked on, or are obvious.

Lastly, a review also indicates appreciations related to the structure and organization of the article, as well as questions of writing, argumentation, scientific writing and, sometimes, the use of English.

Reviewers may offer different opinions due to their conceptual affiliations, their epistemological approaches, methodological knowledge, ideologies, etc. The editors adjust these divergences and send the reports, to which a response must be given, answering clearly and explaining the changes made according to the recommendations.