We all understand the need to evaluate. Evaluation is a process that allows us to improve, critique our work, establish continuous improvement plans, detect problems and encourage progress. To achieve this, the evaluation process must be thorough and continuous; and you can use different instruments that allow you to obtain metrics and ratings (something is "good" or "bad") of the product or service to be evaluated.
Unfortunately, academic evaluation systems rarely use validated assessment instruments, which meet the characteristics of the product to be evaluated, its contribution to the area of research, the progress and development of science, practical utility, the potentially benefited population and the promotion of dialogue, discussion, debate and analysis.
Before adopting a policy of hiring and / or promotion of academic staff based on commercial metrics. Funders, universities, research centers and government agencies should analyze their local and social environment, reflect on their mission and vision, and choose and / or create instruments for academic evaluation based on their needs. That allows its staff to develop high quality academic products with ethical elements and the highest creative processes.
It is impossible to make a research product without a creative process. Which is born of inspiration, and as a result you can get a product as diverse as the inspiration itself.
Some of the academic products that can be used to evaluate the activity of academics and / or researchers are: monographs, book chapters, books, seminars, database design and development, specialized and / or dissemination web pages, videos, posters , Journals Club, academic and / or social programs, curriculum design, exam design, conferences, software design, prototype design and development, educational program criticism, peer review, repository development, brochures, newspapers, notebook, newsletter informative, technical reports, thesis, articles, among many others.
In addition, when evaluating an academic and / or researcher, other activities, such as participation in courses, human resources training, mentoring programs and management of research groups should be considered.
We must face "publish or perish" or "publish for publication", if we adopt this erratic policy of academic evaluation, far from having significant advances in science, we can have a large number of publications, with very poor contributions.
Universities and research centers must eliminate the fear of betting on large projects, those who have years of research to be published, in which numerous research groups participate, who are responsible for several graduate students, in which each student it will be difficult to obtain an independent publication, because, if so, the article would be divided. In these huge research projects, evaluation instruments must be articulated for each member and their contribution to the work must be analyzed.
To develop research work with great impact (measured and qualified in many ways), time must be, what it really is; "relative". The formula of the research hypothesis: the researcher knows, understands and questions.
Both researchers and universities and / or research centers, we must reflect on the evaluation process and the criteria for hiring and / or promotion. Generate appropriate instruments for our area of specialization, which impact our mission and vision, that foster the professional and personal development of our research groups, which generate collaboration among other groups and / or research areas.
The development of appropriate academic evaluation instruments is our responsibility, we must decide the publication schemes that we will use, as well as the guidelines that must be covered to ensure the quality of the publication. The comfort that commercial metrics give us only makes us dependent on them. It is important to note that we adopted them for the academic evaluation process, and that their nature had a different purpose.
If we are really determined to work for a comprehensive evaluation system, we should not seek to make it "easy" or "difficult" to elaborate, it must be necessary and sufficient for the purpose we seek and the result we pursue.
One possible way to start developing the evaluation instruments may be to first know the underlying problem in our organization, that is, look for the relationship between what we want to achieve and what we are doing to achieve it, for that we can use a research method qualitative, such as focus groups, that allow us to gather information and give us a complete point of view of the problem. Subsequently develop instruments to evaluate the quality and relevance of the different research products, for which we can use: rubrics (analytical or holistic), assessment scales and checklists. Form collegiate groups that are responsible for the evaluation of research products, which can be open or closed in pairs. Seek that the most important thing for employers is the quality of the research product, its social, technological and / or scientific application.
The paradox of the institutions that reward the number of publications without evaluating the quality of individual works is that they want to be leaders in an area without even knowing the contents of the works or the findings.
It is extremely complex to have complete and comprehensive academic evaluation processes, but if we limit ourselves to just based on metrics and forget the qualitative component of the evaluation, the one that gives us the qualification (in terms of quality, relevance, reproducibility ...) and what form part of the evaluation process, then we do not have an academic evaluation process, we have a metric system of research and / or researchers.
Now, the evaluation systems based on measurements (number of appointments, downloads, visits….) Are not all bad, what is wrong is to believe that they are synonymous with quality of work. The big mistake is that, as scientists, we do not question it and blindly reap it.
The work to articulate an integral process of academic evaluation is in our hands, it can be as difficult or easy as we want to see it and as fast or slow as we want it to happen, but it is something we must do to have "appropriation of science", " added value to research "," teaching-learning communities "and, most importantly, the training of new and better researchers with innovative ideas that allow them to generate wonderful hypotheses to promote science and better social environments.