The Value of Transfer of Knowledge in Bridging the Gender Gap in STEM

Abstract

Spanish researchers’ recent evaluations regarding the transfer of knowledge have shown that activities that could help close the gender gap in STEM areas are penalized upon evaluation. The results were very disappointing and, upon inspection, could lead readers to reach certain conclusions: first, Spanish researchers transfer research to society in a mediocre way; second, female researchers are even less capable of transferring knowledge in a profitable, beneficial way; and third, activities without a high economic impact for either society or research institutions are not promoted and do not boost the prestige of researchers in this research area. With the help of an ad hoc questionnaire responded to by 513 researchers (both male and female), we examined the causes of the low scores received by female researchers and whether transfer of knowledge activities focused on promoting STEM among female students end up penalizing the female researchers dedicated to those activities. This issue is compounded by the problem of the low visibility of female role models for young female students. The conscientious analysis of the results of the questionnaire may help to improve the fairness of future application rounds, avoiding subsequent disastrous results and encouraging researchers and administrators to work toward fostering an equal society, even if no economic value is directly derived from female-led research.

1. Introduction

Gender discrimination in academia has been the subject of a wide variety of articles in many different contexts, countries and circumstances ([1,2,3,4] and references therein). However, to the best of the authors’ knowledge, little attention has been paid to transfer of knowledge (TOK) and the possible—indeed, nearly certain—existence of a gender gap there as well. The main problem arises when looking for a common definition of TOK, even when there is general agreement that TOK and innovation are indispensable tools for economic growth and wealth creation [5].

The scarce data on female involvement in TOK show that the current organizational practices and attitudes of decision-makers continue to reinforce traditional gender roles and “typically male” approaches and behaviors [6,7]. Best et al. [8] made a first attempt at quantifying the gender dimension in knowledge and technology transfer in Germany, where the capacity for innovation is estimated to be very high compared to the rest of the world. These authors reached the conclusion that the quantitative and qualitative components of the gender dimension are barely integrated among TOK stakeholders and decision makers across all levels. They observed that the laws and the regulations, funding policies, and cultural stereotypes were found to be highly inflexible and male-oriented, with an overrepresentation of men in all organizational structures. Palmén et al. [9] applied the protocol of a European study framework to three cases in Europe: Austria, Spain, and Catalonia. The authors concluded that, although there was a clear understanding that the gender dimension should be included in all steps of a research project in all of their three case studies, building researcher competences in the gender dimension is still a long-term process, and working on this aspect may eventually lead to a more inclusive way of producing and transferring science, which would correct the current loss of human capital in both the public and private sectors. Calvo et al. [10] analyzed the attitudes and decisions of research groups led by men and groups led by women in terms of their collaboration with firms in joint research and development projects. They worked with a sample of 420 research groups from eight regions of Spain, France, and Portugal and showed that women were more poorly positioned in the social networks of industrial collaboration and commercialization than men. Research groups led by men were shown to have around a 10% higher probability of showing interest in R&D cooperation with firms.

According to a report by the Ministry of Science and Innovation [11], women attach more importance to TOK than men, although their participation in formal knowledge exchange and transfer activities is lower. However, the same study also indicated that women participate more than men in informal interactions such as media or channels promoting TOK and less in other types of activities that may involve the creation of companies and spin-offs via patent trading.

Attention to the gender dimension has increased in recent years. However, it seems as if decision makers from scientific organizations have not implemented the right measures to attract women to the TOK area, or to correctly evaluate the importance of TOK to society, which seems to be a more female-predominant field, as opposed to the direct economic dimension, which has traditionally been dominated by male researchers.

One step forward in the analysis of TOK in academia was supposed to be taken in Spain when a new government evaluation program was launched in 2018: the six-year period TOK award program (see description below). The objective of this program was to gauge how TOK is conducted by researchers (male and female) at universities and research institutions in Spain. Adopted in 2011, the Spanish Science, Technology, and Innovation Act contains the legal framework and strategies related to gender issues, as mandated by the Europe 2020 Strategy, the Innovation Union Flagship Initiative, and the Horizon 2020 Framework Programme, which provide funding instruments to achieve strategic TOK objectives. As mentioned above [11], substantially fewer women participate in TOK than men in many—if not all—scientific areas, and this is an initial filter when evaluation tools are used to measure this participation. Consequently, evaluations recognize the researchers that are more prominent in this aspect of their work. It is therefore quite easy to predict the outcome of any evaluation process. However, in the Spanish chart of research areas from the Ministry of Science, Innovation and Universities (as it was called when the evaluation program was launched; it is now the Ministry of Universities), TOK is exclusively related to engineering and technology [12], which are traditionally male-oriented areas [1,13]. However, this manual clearly states that research and development should be transferred to society, regardless of the economic benefits. It seems that the Spanish call for applications did not take this dimension of TOK into account, and evaluators also dismissed is, giving very lower scores to TOK activities with a high social impact than to those with high economic impacts. The evaluation results were published more than one year after the call was made, and the results could not have been more dramatic in their implications for all parties, but especially for women, as we discuss in this paper.

The confusion surrounding the definition of TOK and the very complicated call for applications (see below) became the first filter for the proposals to be submitted for evaluation. In total, 16,151 proposals were evaluated [14]. Broken down by area, 8% were from the arts and humanities, 29% were from the social sciences and law, 32% were from engineering and architecture, 10% of the proposals corresponded to the health sciences, and researchers from the sciences in general (chemistry, physics, biology and Earth sciences) submitted 21% of the proposals. Only 35% of the proposals were from female researchers (about 1 in 3 proposals). The final results published by the Spanish Agency for Quality Assessment and Accreditation (ANECA) also showed that the application round had a success rate of 42%, but when disaggregated by sex, the differences were staggering, since only 34% of the requests made by female researchers received a positive evaluation compared to 47% of the requests made by male researchers. That is a difference of 13 points, translating to a 30% gender gap in total positive evaluations [14].

The results by field of knowledge can be seen in Figure 1. The plot shows that there were fewer applications from women than from men in all fields. Additionally, the fact that in all fields it was men who received the highest percentage of positive evaluations is also striking. Thus, for example, in Field 3 (engineering and architecture) there was a 16-point difference between the positive evaluations received by men and women, representing a gap of 38%; in Field 5 (arts and humanities) with a close number of applications submitted by both genders, the gap was 24%; and even in an area as female-dominated as health sciences (Field 2) the difference was 12 points, representing a gap of 26% in favor of men.

We thought these data deserved a detailed analysis to understand exactly what happened to yield these unsatisfactory results. In fact, the results were so dramatic that even national newspapers wanted to know the position of feminist associations such as the Women in Research and Technology Association (AMIT), regarding the results. Consequently, the authors of this work were invited write a column in one of the most important Spanish national newspapers, El País: https://elpais.com/ciencia/2020-07-10/transferencia-del-conocimiento-estamos-haciendo-bien-las-cosas-las-investigadoras-espanolas.html (accessed on 5 May 2021), which triggered a detailed analysis of the results. A first approach was published in a Spanish academic journal [15], and a presentation was given in a STEM-related session of the TEEM conference, resulting in this paper.

Created in 2002, AMIT was started by a group of researchers who saw the shortcomings of the Spanish scientific system for female researchers at that time. AMIT defends the interests, equal rights, and opportunities of Spanish researchers and technologists in contexts where it has been claimed that there are no quality female researchers to be added to committees, panels, and government agencies [16]. At present, AMIT has more than 1000 members. AMIT promotes compliance with the recommendations of the European Commission to achieve gender equity, the Spanish regulations contained in the Organic Law for Effective Equality between Women and Men [17], and the development of all issues related to gender set out in the Spanish Law of Science (see above). Among the tools used to achieve these objectives is the analysis and monitoring of various activities, such as calls for awards, contests, competitive examinations or promotions, publications, and the composition of panels in scientific events. Even though there are increasing numbers of women in the field, when higher education and doctoral studies are compared (58% doctoral graduates are women [18]), the number of female researchers in the highest decision-making positions is well below that of male researchers. When imbalances like these are observed, AMIT raises public awareness via the media or specific publications. Thus, the authors of this paper, along with other AMIT members, have developed an ad hoc survey that was sent to all members and any researchers interested in participating in the study, with the intention of investigating whether there was indeed a gender disparity not only in the results, but also in the whole process. Five hundred and thirteen people (72 men and 441 women) responded to our survey, which represents more than half of AMIT members. It is worth noting that the results not only show the low rate of success among female researchers, but among researchers overall: 48%.

Sustainable development requires knowledge production that strikes a balance between scientific and other types of knowledge [19]. The results of this paper are contextualized not only within the Spanish research evaluation system but also in consideration of the role of female researchers in meeting sustainable development goals, including the increase of women in STEM studies. The shortage of female students enrolled in STEM disciplines was noticed already in the 1990s [20]. To this day, a lack of promotion among students in general, and particularly among female students, exists in the scientific and technological fields that society and the labor market are demanding [21]. Promoting STEM subjects among students should be considered as a proper TOK to bridge this gap, and it is normally women who take the lead in this task. Programs like Girls4STEM [22] are a useful tool to overcome the lack of female students in STEM, and this line of TOK will help in reaching gender diversity in these areas in the context of a sustainable future.

This paper highlights the imbalance in the evaluation of TOK in researchers who devote their time to the transfer social values, including the role of female researchers in STEM subjects. This type of TOK is mainly done by female researchers, which is reflected in the outcome and conclusions of our research.

The Spanish Research Evaluation System

As mentioned above, the TOK evaluation process is part of what is referred to in Spain as six-year period (“sexenios” in Spanish) evaluation awards, which are monetary productivity bonuses for the research and output of academics and researchers working at public institutions. Until now, there were only six-year period research awards (“sexenios de investigación”). To receive a six-year period research award, researchers must have tenure at the institution and at least five papers published in high-impact journals within a period of six years. Now, there are also six-year period TOK awards (“sexenios de transferencia”), which evaluate the transmission of research to the industrial sector, the financial sector, and society at large. The expectation is that this TOK should translate to an impact on the country’s economy and, allegedly, to social wellbeing. Evaluations for both types of six-year period awards are performed by ANECA, with a maximum of six given in total. Although the selected researchers receive money in addition to their salaries, the most significant outcome of this process is professional prestige. Talks are currently underway to implement a six-year period teaching bonus, likely related to teaching excellence.

The call for applications examined in this study was based on the guidelines set out in the Association of Rectors of Spanish Universities (CRUE) report written by a group of experts. This report provides recommendations to encourage TOK in two ways. On the one hand, it provides an institutional setting in which a new system of quality indicators is defined to promote the prestige of Spanish universities. On the other, it sets up a scenario of merit and individual rewards through a TOK award, which is intended to recognize the efforts of researchers promoting TOK.

The proposed indicators to assess the transfer of research knowledge are divided into four sections or fields:

-

C1: TOK through the training of young researchers. This section includes activities such as training young researchers and the creation of start-ups and/or spin-offs within a competitive entrepreneurial culture.

-

C2: TOK through interactions with other institutions. In this section, research outputs and active memberships on highly relevant committees are considered.

-

C3: TOK creating economic value at different levels. Royalties generated by research, patents, and any other activities that create financial gain for institutions are considered.

-

C4: TOK generating social value. Activities to benefit society as a whole, as well as different actors and stakeholders. The call mentions that promoting the quality of the institution involved in the research would be considered in the assessment, including:

• Involvement in agreements and/or contracts with non-profits or government agencies for activities with special social impact.
• Outreach activities (e.g., publications, long-term engagement on social media, outreach to the industrial sector).

To be evaluated positively, high-quality contributions had to fall under at least two of the four sections.

This paper analyzes the results of this TOK assessment with an ad hoc survey created to understand why the results of this first call were so disappointing not only in terms of TOK, but also gender. If only one-third of the successful proposals were from female researchers, the questions are: are Spanish female researchers doing the right thing transferring the knowledge derived from their research and its outcomes? Should they consider investing their time in activities more profitable than promoting STEM, which in theory should be evaluated under the fourth section of contributions?

2. Materials and Methods

To analyze and discuss the results of the TOK evaluation, we used the data published by ANECA and an ad hoc survey developed by the authors in consultation with other AMIT members. The survey was created using Google Forms, and the link was made available to all AMIT members through a call on the AMIT website; we made clear that any other researchers interested in participating would be welcome regardless of whether they applied for the six-year period TOK award. No exclusion criteria were applied to participant recruitment. The survey was anonymous, but we assume that most participants were AMIT members.

2.1. Survey Structure

The survey consisted of three blocks. The first block was the participant’s academic profile, the second block was the participant’s response to the call (if the respondent had applied) and, the third block asked for suggestions to improve future calls. The structure is described in detail below, and Figure 2 contains the corresponding flowchart:

2.2. Block One: Survey Participant Profile

Data on sex, age, university or organization where the participants (applicants to the call or not) were enrolled or worked were requested to understand participant profiles, professional categories, and the number of research awards of this type they had received. It should be noted that the call established that only researchers and civil servants with permanent contracts could apply for the six-year period TOK award. They also had to have received at least one six-year period research award.

The last question in this block was: Have you applied for the transfer term?

2.3. Block Two: Participant Results

In this block, two possible itineraries opened depending on the answer—“yes” or “no”—to the last question in block one:

In case the answer was “no”, the survey requested information on why. Participants could choose from:

• Did not meet the conditions of the call (e.g., having at least one six-year research period);
• The criteria were too imprecise; the rules were not entirely clear;
• It required additional effort on top of our already cumbersome workload with no guarantee of a fair evaluation, due to unclear criteria;
• It seemed to me that my merits were not of the caliber requested;
• I have participated in TOK actions but I have not been the lead researcher or project lead;
• I preferred to focus my efforts on the six-year period research award because the requirements are more clearly defined;
• Other.

Afterwards, the survey asked about plans for the next call: “Do you plan to apply for the next call? Briefly justify your answer.”

In case the answer was “yes”:

There were once again two possible paths from the question “Was the evaluation result positive or negative?” In both cases, questions were asked about the merits discussed in the application and the score received, though the latter details were known only by respondents who received a negative evaluation. No feedback was offered for successful submissions.

In case of a positive result, the questionnaire asked participants to rate from 0 (not at all) to 5 (very much) the aspects that they believed made the difference between a successful and unsuccessful application, with dimensions including not only the merits presented, but also the way they were presented and how their importance was underscored.

Finally, the questionnaire asked for tips for people who were unsuccessful and planning to apply again in the next call.

2.4. Block Three: Tips to Improve the Call

With this final block, we wanted to learn how participants felt future calls could be improved and gave them the opportunity to point out any aspects or considerations of the evaluation they would like to share that were not sufficiently covered in the survey.

The specific questions were:

“What aspects do you think should be improved in future calls? (Mark all that apply)”

• Publication of the scoring scale for each item.
• More information on the types of contributions that correspond to each item assessed.
• Add an FAQ section including examples of successful submissions from this first call.

“What aspects do you think should be modified in future calls to increase the participation of female researchers? (Mark all that apply)”

• Balance the evaluation and coordination teams in terms of gender/sex.
• Add coordinators in the areas with fewer positive results for women.
• Offer examples of contributions with high scores.
• Use inclusive language.
• Make the scoring system public.

“The results show that the percentage of women who applied to the call, as well as of those who received a positive evaluation, is lower than that of men. In your opinion, what are the causes of the gender gap in these results? (Mark all that apply)”

• Lack of gender/sex analysis in research training for evaluation teams.
• Lack of gender/sex parity on evaluation and coordination teams.
• Lack of clarity in the scoring criteria.
• Insufficient information on these six-year period awards and their possibilities.

3. Results of the Survey

A total of 513 researchers, 72 men (14%) and 441 women (86%) responded to our survey, which represents more than half of AMIT members. We consider this sample to be representative of the AMIT community, although a larger sampling should be made in order for the results to be representative of the larger community of applicants. The age distribution peaks around 53 for women and 55 for men, as can be seen in Figure 3.

Distribution of the pool in the different research areas (Figure 4) was as follows: 12% corresponded to arts and humanities; 18% were from engineering and architecture; health sciences submitted 4% of the proposals, 34% were from social sciences and law; and sciences in general (chemistry, physics, biology and earth sciences) submitted 32%. It seems that STEM areas were the best represented among the survey respondents. The respondents’ research qualifications were endorsed by the number of six-year period research awards received (70% claim to have received three or more). As can be seen in

Table 1. Summary of the AMIT survey results.

Gender/Sex	Did You Apply for This Call?
	YES		NO
	Number and % of responses	Positive outcome	Number and % of responses	I plan to apply in the future	I do not plan to apply in the future
Female	249 (56%)	116 (47%)	192 (44%)	86 (45%)	106 (55%)
Male	47 (65%)	26 (55%)	25 (35%)	15 (60%)	10 (40%)
Total	296 (58%)	142 (48%)	217 (42%)	101 (47%)	116 (53%)

, despite these excellent qualifications, only 296, i.e., 58%, claimed to have applied for the six-year-period TOK award. Disaggregated by sex, the data show that 56% of women respondents applied for it, compared to 65% of men. On the whole, the outcomes of the six-year period award were well in line with the outcomes reported by ANECA: 48% of applications were positively evaluated (42% reported by ANECA), 52% of applications were negatively evaluated (58% reported by ANECA).

It should be noted that the 53% of the respondents who did not apply for the six-year period award stated that they have no intention of applying for it in the future; at this point the difference between men and women was significant, given that only 45% of women, compared to 60% of men, asserted that they would apply again or in the future if they did not do so in the first call. The reasons why they did not apply are shown in

Table 2. Reasons why the people surveyed did not request the six-year TOK award.

It required additional effort on top of our already cumbersome workload with no guarantee of receiving it	25%
It seemed to me that my merits were not of the caliber requested	19%
The criteria were too imprecise; the rules were not entirely clear	17%
I preferred to focus my efforts on the six-year period research award, because the requirements are more clearly defined	9%
I did not meet the criteria of the call (e.g., one six-year period research award)	9%
I have participated in TOK actions but I have not been the lead researcher or project lead	4%
Other	18%

(note that they could indicate more than one answer) and the most common reason was the lack of clarity in the call for applications.

With respect to the kinds of TOK contributions yielding positive or negative outcomes, the results were studied in detail, considering the four fields according to the classification provided by ANECA (see above). Figure 5 sets out the contributions according to the ANECA classification, both for positively- and negatively-evaluated applications. Notably, there was a difference of 9 points between negative and positive evaluations in Field 3 (TOK creating economic value, skewed toward positive evaluations), and 14 points in Field 4 (TOK generating social value, skewed toward negative evaluations). There was also a significant difference in Field 1 (TOK through researcher training), where, according to the results of the survey, the contributions were concentrated in the subfield “Number of people hired responsible for R&D projects and contracts.” This field also skewed toward positive evaluations and, conversely, there was a tendency (6 points) toward negative evaluations in Field 2 (TOK through interactions with other institutions).

TOK contribution data disaggregated by sex are presented in Figure 6, which highlights that women presented fewer projects with financial value (Type C3 and Type C1). However, female contributions to Type C2 and, especially, Type C4, which are related to transfer of one’s own knowledge to society or other activities that benefit society, were higher than those of men. In Type C3: Transfer generating economic value, men presented 53% vs. 35% presented by women; in Type C1: Transfer through the training of researchers, which includes the creation of start-ups and/or spin-offs, there was a difference of 2 points in favor of men; conversely, in Type C4: Transfer generating social value, women contributed with 41% vs. 27% of men’s contributions, and in Type C2: Transfer of own knowledge, women’s’ contributions were 7 points higher than men’s.

To assess how contributions by men and women have been valued, Figure 7 depicts the types of contributions, disaggregated by sex in case of: (a) Positively-evaluated applications, and (b) Negatively-evaluated applications. The most significant difference between a positive and a negative result was that positively-evaluated submissions tended to be contributions with economic value (Type C3 and Type C1) while negatively-evaluated submissions contained more contributions with social value (Type C4 and Type C2). It must be considered that type C4 contributions included many actions to raise awareness of the lack of female students in STEM careers, publications on the miniscule number of citations of female researchers in STEM journals, and the efforts made by researchers to participate in activities to highlight this reality, even if these activities were organized by local, regional, and national organizations (e.g., Girl’s Day, Talent Girl). Most of these activities are organized and carried out by female researchers.

Figure 8 shows the scores received for the different merits in the applications with a negative evaluation. It should be noted that scores ranged from zero to 10 for each criterion, and the minimum total score required to have the chance of a positive evaluation was six. The average value of the score received was 3.9 with a standard deviation of 2.4, and for each type of contribution the average results were 2.0 in Type C1, 3.2 in Type C2, 4.6 in Type C3, and 3.8 in Type C4. As can be appreciated, it seems that the evaluation committees concluded that contributions of Type C4 submissions did not have as much TOK value as those of Type C3.

4. Discussion and Conclusions

The Spanish research system has two ways of evaluating the production of researchers. One is publications (six-year period research evaluation) and the other (implemented as a 2018 pilot test) is the TOK six-year period evaluation. The objective of the latter was to gauge how TOK was done by researchers (male and female) at universities and research institutions in Spain.

The results of this pilot experience suggest a gender bias reflected in different aspects; starting with the low number of applications submitted by women (35% of the total applications) in all research areas, including female-dominated areas, and continuing with the low success rate of female researchers in this process. Our study has a research limit in the conclusions, as the main target of our survey consisted of about the 10% of the total applications, and we assume that most replies came from AMIT members. Still, we believe it to be representative for our purpose.

Although the overall success rate for this round of applications was 42%, there were important differences linked to the sex of the applicants. In all fields of knowledge, male researchers were more successful than female researchers. Consequently, only 34% of the applications submitted by women received a positive evaluation while men received 47%, resulting in a 30% gender gap in the total of positive results. These differences cannot be ignored. They are significant, and indicate the presence of gender biases in this pilot test that will have to be corrected in the design of the call itself, its implementation, and its evaluation: the gender perspective must be considered throughout the process. Moreover, the results of the survey designed by the authors to deepen understanding of the causes of such discouraging results highlights the gender gap not only in the outcomes (47% positive for female researchers), but in interest or self-concern in applying for the six-year period TOK evaluation. It has to be considered that the results of this program will be translated into a merit and economic promotion of the researchers, and these results create a gender imbalance in women’s prestige. Our data show that despite their excellent research qualifications, nearly half of the respondents surveyed said they did not apply for the six-year period TOK award (44% of women and 35% of men). In addition, the disaffection with this call (or perhaps with the transfer activities) goes further, since the majority of respondents claimed they have no intention of applying for it in future calls. The reasons provided were mainly the poorly defined criteria of the call (Table 2). This aspect could be linked to the greater self-criticism of women when assessing their own merits and capabilities [23], which clearly compounds when criteria are not well defined, as occurred in this call. Add to this the double burden that women still bear to a greater extent than men [24] and one can find numerous causes for low participation. In other words, the results suggest that women, to a greater extent than men, analyzed the time and effort required to apply (it should be noted that many of the respondents agreed on the cumbersome nature of the application) and the probability of receiving the award given the poorly defined criteria, and made their decision on whether or not to apply for the award accordingly.

When van den Besselaar and Sandström [3] studied gender differences in scholarly productivity and their impact on the promotion of researchers, they offered several possible explanations for the productivity gap. Their study determined that there was a clear relationship between researcher position and productivity. Because women still hold the lowest positions in academia, while men still hold most decision-making positions in institutions, the effect of gender bias is a constant throughout an academic career. van den Besselaar and Sandström also studied gender differences in research performance to analyze their impact on the development of men’s and women’s research and academic careers [2] and found that the existing performance gap was more pronounced at a later stage in academic life, while it was much closer at early stages. Their study was based on a large dataset, with an average age of 43. The average age of our pool is around 53. From our observations, it seems that this increasing gap is maintained into the senior stages of academics’ careers.

Regarding the TOK contribution types in the Spanish six-year period program, the data collected indicated the predominance of economic aspects over social ones. It is precisely these two aspects, the economic and the social, which made the difference between a positive evaluation and a negative evaluation. In this sense, the most significant differences were observed among contributions to Field C3: TOK generating economic value, which skewed 9 points toward positive evaluations, and contributions to Field C4: TOK generating social value, which skewed 14 points toward negative evaluations. There was also a difference in Field 1 (TOK through researcher training) where, according to the results of the survey, the contributions were concentrated in the subfield “Number of people hired responsible for R&D projects and contracts.” This also skewed towards positive evaluations and, conversely, there was a bias (6 points) toward negative evaluations in Field C2 (TOK through interactions with other institutions). These results point to a marked economic bias when evaluating TOK activities in this pilot test and, consequently, this bias may have played a significant role in the gender gap observed, in the sense that women contributed fewer merits from Fields C3 and C1, both related to activities with financial value, and, conversely, more merits from Field C4 and Field C2, related to activities with a higher social impact. The scores from each field reinforce these arguments, given that merits from Field C4 received lower scores than merits from Field C3, which, in addition, received the highest scores on average.

Since the general perception is that these contributions that did not have a large economic impact, they were not considered valid TOK. In this sense, the work done by female researchers to get girls interested in science and STEM matters in general does not generate direct economic value to compete with the high-priced projects presented mostly by the male scientific community. A number of studies demonstrate that one of the concluding points about girls’ reduced interest in STEM subjects is not only the lack of role models [9,25,26,27] but also the lack of knowledge of the social utility of these disciplines; i.e., TOK to society. If the work of female researchers promoting STEM areas (e.g., conferences, workshops, specific publications and other activities dedicated to girls and STEM) is not valued, and TOK-generating social value is trivialized, women’s interest in these subjects will decline and girls’ perception on the role of women in STEM areas will further deteriorate.

To conclude, it appears that the entire evaluation process of this award was not mature enough to be implemented. The call was very intricate, criteria were not clearly specified, and the evaluation committees were likely not trained on how to evaluate all the dimensions of TOK, including gender. European programs such as the Marie Sklodowska-Curie program could serve as an example of good practice, as it includes very clear criteria for assessing proposals. Gender aspects should be considered in the proposals when needed, and TOK should be evaluated as a research impact. The evaluation of the measures to leverage and disseminate the research results should focus not only on how the new knowledge discovered in the proposed research will be realistically disseminated and leveraged, but also on the target audiences: scientific, industry, finance, professional organizations, policymakers, and the wider community. Only where applicable should the evaluation take into account the potential commercialization of results. This evaluation protocol seems appropriate and straightforward and would likely encourage more female researchers to participate in the call for applications.

If the evaluation process is not reviewed and the shortcomings tackled, researchers in general and female researchers in particular will dedicate their efforts toward more rewarding activities like publishing in highly-recognized journals—this will always be considered in the parallel evaluation process of the six-year period research award, which has a better-defined, more transparent evaluation process—and all the efforts to bridge the gender gap in STEM subjects will fall by the wayside.

In the end, a number of researchers who received a negative result applied for an administrative remedy with ANECA. However, over one year later, most have not received a response. Under these circumstances, and if the agency does not address the different points highlighted in this paper, the next six-year period TOK award will not succeed in its objectives and female researchers who once promoted STEM to bring female students into our classrooms will dedicate their time to other, more productive activities. Based on the research questions proposed in the Introduction, we conclude that Spanish female researchers are doing the right thing by transferring the knowledge gained from their research in all fields, including those with a social impact. In cases when researchers have not received a positive evaluation, only by balancing their personal and professional interests should they consider investing their time in more academically and economically rewarding matters. Further considerations should be made once the results and outcomes of the administrative solutions are published.