A critical review of science in Slovenia

Zora Janžekovič, an internationally renowned burn surgeon who was named one of the “25 most influential doctors of the 20th century” and one of the “top 50 influential doctors in history”, died in a small Slovenian town in 2015. Although she made groundbreaking medical discoveries while working at a hospital in Slovenia, she is still virtually unknown to the local public. Her innovative method of treating severe burns has saved many lives by dramatically reducing infections and hospitalization times. Through hard work, courage and perseverance, she has shown that it is possible to achieve cutting-edge scientific achievements even in ill-equipped facilities, with almost no resources or support. Her method has become the standard for treating deep third-degree burns. But despite her great accomplishments, which in Slovenia are hard to find parallels in any other scientific discipline, she died at the age of 96 almost unknown to her fellow citizens.    

Between invisible and fake scientific achievements

The attitude of the Slovenian scientific and academic community towards Zora Janžekovič shows that excellent, influential and internationally recognized scientific achievements do not necessarily lead to appropriate recognition at the national level. Although Zora Janžekovič conducted all her research in Slovenia, which was part of Yugoslavia at the time, and is internationally regarded as a key innovator in her field of expertise, she was never a member of the Slovenian Academy of Arts and Sciences, nor was her work considered particularly outstanding in the Slovenian scientific community.

If Zora Janžekovič is an example of a scientist with exceptional achievements who still does not have a proper place in the local academic hierarchy, we are witnessing in Slovenia, on the other hand, the rise of individuals and groups who are successfully climbing the academic ladder, while at the same time their work has almost no substance or real scientific value. There are a growing number of scientists who can prove to be extremely productive, according to the bibliometric algorithms that now automatically measure their performance. But over-reliance on bibliometrics sometimes gives the wrong impression of a researcher’s true scientific merit. Superficial research achievements have no real value for science or society, but they can allow some individuals to rise to positions of influence and obtain research funding.

The rise of fake science on the one hand, and cutting-edge achievements that go unnoticed on the other, are just two extremes in a very diverse range of problems plaguing science in Slovenia. In this paper, we will take a closer look at some of the key factors that have had a significant impact on the development of the Slovenian scientific community. In particular, we will highlight the process of the transfer of academic authority from scientific elites to algorithms and describe the consequences of this change which has led, among other things, to a significant impoverishment of the opportunities to pursue academic careers. We will also describe the gradual transformation of public research funding, which has created a new layer of “scientific aristocracy”. Finally we will outline the rise of the ideology of “excellent science”, which has replaced a commitment to research integrity and quality.

The beginnings of the scientific and academic community in Slovenia

Ensuring the authority of those who speak in the name of knowledge is one of the key functions that the scientific community provides to society. Science is not only a very efficient system for evaluating ideas, it is also a system for evaluating experts. While scientific theories are usually evaluated in the international scientific community by critically confronting ideas in scientific journals, experts are evaluated primarily on the basis of their achievements and reputation in the professional community, which is usually reflected in academic and professional titles.

Traditionally, the competences of new experts have been assessed academically by already recognized scientists. During the period when today’s Slovenia was part of Austria and later the Austro-Hungarian Empire, many Slovenian-born scientists established themselves in the German-speaking academic community, which was one of the most fertile environments for scientific research in the 19th century.

After the First World War, some of them returned and started to build a local Slovene scientific and academic community, which also began to introduce Slovene language more actively into the academic environment. A key role in this was played by the establishment of the University of Ljubljana, and subsequently by a number of scientific institutes, which initially operated within the Slovenian Academy of Sciences and Arts, but later became independent research institutions. 

When the University of Ljubljana was founded more than a hundred years ago, established scientists were sought throughout Europe to take over specific fields of study. They were also helped by professors who had fled Russia after the Revolution and were looking for work in Europe. Together with Slovenians who returned from professorships at European universities, Russian emigrants became an important foundation of university studies in Slovenia.

Although the first person to receive a doctorate at the newly founded University of Ljubljana was a woman, it was not until after the Second World War that the situation for women scientists began to improve. However, equality for women in science is still not achieved at the highest academic ranks and in the membership of the Slovenian Academy of Arts and Sciences, where women make up only a few percent of the members.

During the Yugoslav period, Slovenian science had a relatively good research record. Among other things, Slovenian scientists have been highly visible in various international initiatives to improve the quality of life of people around the world, which have taken place within the framework of the United Nations. They also had relatively free access to literature and Western technology, which enabled the rapid development of specific fields of expertise. 

Research in Slovenia received a major boost in the 1980s, when some officials, who had previously been active scientists, managed to significantly increase the amount of public funding for research, enabling the purchase of important research equipment. The introduction of a highly successful system of public funding for the most successful postgraduate students, the so-called “young researchers”, was also very important.

Shifting academic authority from scientific elites to algorithms  

With the change of the political system and the establishment of the Republic of Slovenia in the early 1990s, doubts began to be publicly expressed about the real professionalism of the domestic academic and scientific elite. Since there was no longer strong enough confidence that the Slovenian scientific and academic hierarchy was based on genuine professional competence, the search began for ways to put the authority of experts on a new, more reliable footing.

The digitization that was then taking place in many areas of society led to an initiative to create a more objective way of assessing academic competences on the basis of bibliographic data. At the time, this was a relatively new approach, originally developed to compare the productivity of researchers, especially in the natural sciences and engineering. However, such a system could only be introduced once most of the data on scientific articles and books has been digitised and entered into large databases.

In the 1990s, the COBISS and SICRIS catalogues were launched, which still collect data on publishing and other achievements of researchers. At the time, most members of the scientific community considered such databases to be an impartial tool that would help to create a more reliable ranking among scientists. Instead of relying on established academic names that could no longer be trusted in some fields, the new idea was to establish a hierarchy of expert authorities using algorithms based on bibliographic data collected in large databases. Challenging academic elites has thus led to a shift in authority from recognized experts to automatic evaluation through analysis of bibliographic data. With the formation of the Slovenian Research Agency (ARRS), the automated evaluation approach has finally established itself as the only true reference.

Initially, many researchers supported the new method, as doubts about the expertise of academic elites in many fields were justified. However, relatively soon the new rating system also proved problematic, especially when the amount of money received was added as one of the parameters in the algorithm. Although it became increasingly apparent that the automatic evaluation system was not as good as it first seemed, the community did not seriously question it because the other evaluation options were considered to be even worse.

Thus, some of the more influential researchers have started to adapt the evaluation algorithm to achieve the best possible outcome for themselves and their research teams, while others have found that it only makes sense for them to create the bibliographic records that will give them the best possible score. The automatic evaluation system has also been shown to reward the accumulation of achievements by the individual researcher, who is usually also the team leader. Since scientists who have risen to the top of the hierarchy in the new system are not interested in changing the evaluation mechanisms, and others may no longer have enough influence to make a difference, serious reform has not yet taken place.

Recently, the scientific administration has argued that algorithms no longer have as much influence as they used to, since the actual content of research is supposed to be assessed by a system of independent peer reviewers. Unfortunately, nothing substantial has changed in practice so far. The system is still based on algorithmic evaluation, since even the experts who are invited to give peer reviews more or less just look at the evaluation given by the algorithms and then summarize it in words. Another major problem is that most evaluations cannot be challenged, even if there are obvious misunderstandings and errors.

Bibliographic metrics still dominate the evaluation of research proposals and promotion in the Slovenian academic community. But if academic elites once had to take at least personal responsibility for their decisions, now they no longer have to. The results of algorithmic evaluation are seen as objective indicators of the value of scientific achievements and researchers’ abilities.

In recent decades, the functioning of Slovenian science has undergone a fundamental change, which could be described as a transformation of who constitutes academic authority. Academic authority has been transferred from scientific elites to bibliometric algorithms. If internationally renowned burn surgeon Zora Janžekovič was excluded from the local Slovenian academic community fifty years ago by provincial academic elites, today she would most likely be excluded by the seemingly objective algorithms behind which contemporary “digital” elites hide.

Rigid pathways to pursue an academic career  

A serious negative consequence of the introduction of algorithmic assessment of academic performance is that the chances of pursuing an academic career have been significantly reduced. Because there are precise unwritten rules that prescribe in advance the sequence of steps that everyone has to take to become a professor, a young scientist has little freedom in how he or she enters science. The emphasis on formalism means that the real content and impact of research is playing a diminishing role. All that matters is that the researcher meets the predefined criteria or requirements built into the algorithms.

If someone diligently pursues a career along a predetermined academic path, he or she is almost automatically promoted and considered a successful scientist. Of course, it is not worth asking what anyone has really achieved in science, since it is assumed that the system of evaluation by algorithms is the best we have. All other known methods of evaluation are supposed to be worse.

Of course, the human mind is not at its most creative and productive when it is heavily controlled. Unfortunately, Slovenian scientists have to prove themselves over and over again in the light of what fits the algorithms, not with actual achievements in their field, which means that they are constantly under scrutiny. Many capable scientists have already left academia to pursue their careers in business or in other countries. They simply cannot identify with the values and rules that locally determine the only possible path for a scientific or academic career. 

It has also become clear that some influential scientists simply do not understand that there are other legitimate career paths than the ones they have taken or that are recognised in their discipline. It is by no means necessary that all scientists should follow the same path as, for example, a successful experimental physicist. To encourage only “excellent science”, local academic authorities have used algorithms to exclude anyone who has not conformed quickly enough to established academic stereotypes. In a small environment like Slovenia, however, singling out those who do not conform to the apparent ideal of a true scientist is very damaging. 

There are also unusual cases where scientists go to a neighbouring country for a few months to obtain the necessary conditions to work in a foreign academic environment, as defined by the algorithms. For these visits to foreign universities, it is only the duration of the stay that matters, not the purpose and content of the trip. If the purpose of the trip is to learn about new techniques, to have personal conversations or to browse archives, it is of course useful, but it has no value in itself.

Unfortunately, it is currently only possible to be successful in Slovenian science if a young scientist decides in advance to follow an established academic career path, from which he or she will not deviate in the slightest, and will keep quiet about any potential problems. If a young scientist breaks the unwritten rules, he or she will be swiftly punished.

Funding reform and the creation of a new scientific aristocracy    

In addition to the transfer of evaluation responsibilities to algorithms and the limitation of possible career paths for scientists, another important change in the scientific system took place after Slovenia’s independence. A new way of distributing public money to support scientific research was introduced.

In the past, public money for science has gone to research organizations, which have had considerable autonomy over what they spend it on. Of course, much of the work done in state-aided institutes has been to support industry and improve people’s quality of life. Some public money has also been distributed for research projects of several years’ duration, decided more or less independently by the administrators of the individual scientific disciplines. The system was, of course, not transparent and allowed for decisions that many did not consider fair.

This led to a reorganization of science funding, with the creation of the new Slovenian Research Agency (ARRS). The former direct state funding of research at universities and institutes was transformed into funding for programe groups, and the funding of concrete research proposals was given a place in the annual call for research projects.

As a direct consequence of the new scheme of stable funding for science through the programe groups, the real decision-making power has been transferred from the management of the institutes to the heads of the programe groups. As a result, the management of the Institutes has lost much of the power to decide on their own strategic development and staffing decisions. At the same time, the power of individual researchers who, by a series of accidents, became programe team leaders has increased dramatically. These groups received money directly from the Research Agency (ARRS) and had to report directly to the Agency on their research achievements.

It is also important to note that Programe Team Leader is not an elected position, but a feudal title that can only be transferred to another researcher if the current leader so chooses. If a programe group is accidentally abolished, which is rarely the case, the money is distributed among other programe groups in the same institution. In the current system, programe team leaders have a great deal of formal and de facto power that is almost impossible to control. Although many researchers do not take advantage of their position, this is by no means the case for all programe group leaders.

The system of research funding in Slovenia makes it difficult for institutes and universities to propose and follow their own development strategy and does not promote collaboration. In 2021, a new Law on Scientific Research and Innovation (ZZrID) was adopted, which restores autonomy to research organisations regarding human resources and research policy, but it will be necessary to wait for its implementation to see what real changes the new regulation will bring. 

The ideology of “excellent science” as a substitute for research integrity and quality

In recent years, we have often heard everyone pushing for as much “excellent science” as possible. But when you try to find out more about what this slogan means, it is usually explained that science is excellent when it produces highly cited publications in journals with a high impact factor.

Of course, this all sounds nice, but the scientific community needs to ask itself whether a high number of publications and citations is really the only value to be celebrated. Perhaps we should also ask how a particular researcher came to have a large number of publications and citations?

In addition to the method of publication, might the content of the research, its relevance to the development of science and the advancement of society, also be important? Does it also matter what role the integrity of researchers plays in the conduct of research? Can someone be a good scientist if he keeps quiet when he knows that his colleague is harassing female students, doing bad science or misleading the public about his research achievements?

These and similar questions, which are by no means simple, must be constantly asked by any healthy system that wants to do its job well. In a well-functioning community, especially one that is supposed to value ideas effectively, there must be an open and lively debate in which no one part of the community dominates and all voices are represented.

In Slovenia, we are currently at a stage where we are only beginning to become aware of the systemic problems in science, but we do not yet have functioning mechanisms that would systematically detect and address the problems. Sometimes it seems that we are still naively convinced that the basic rules of science will effectively prevent anomalies, but unfortunately this is not the case. 

In recent years, we have had a number of high-profile cases of publication based on false data, cases of serious citation and editorial anomalies, cases of doctoral theses defended on the basis of pseudoscience, and many other similar incidents that often do not even make it into the media. But more than these cases per se, the problem is that there is no proper systemic response to them. Individuals who are caught breaking the rules are generally not even effectively detected by the academic system, let alone sanctioned. In some cases, such behavior is even rewarded and the whistleblower is retaliated against.

While there are not a few problems in the scientific community, it is also true that a number of scholars are making sincere efforts to improve the situation, but unfortunately, at least so far, they are not succeeding. One of the reasons for the failure of the reforms probably lies in the realization that in Slovenia it is only possible to operate in a fairly peaceful and successful professional manner in fields where people who happen to be good scientists and have healthy collegial relations have managed to get together.

However, for such bubbles of good science to be successful, their members must remain silent on larger systemic problems. Any publicity can only be in support of the system and should not in any way draw attention to the problems. It is also very useful for an individual scientist to join one of the large academic interest groups that lobby and advocate together for their interests.

Unfortunately, scientists in Slovenia have in recent decades not been motivated to seriously reflect on the functioning of the local scientific community. Science in Slovenia urgently needs a great deal of self-reflection and questioning. It must adapt to new circumstances while remaining as true as possible to its core values and mission.