Trends and Gaps in Empirical Research on Open Educational Resources (OER): A Systematic Mapping of the Literature from 2015 to 2019

For almost 20 years, Open Educational Resources (OER) are an integral part of the debate about the digitisation of education. However, the empirical landscape of OER research is vivid and largely obscure. This article reviews the state of international empirical research on OER to reveal trends and gaps and, in this manner, identify possible desiderata for further research. We use a systematic mapping approach to examine the empirical English-speaking research landscape from 2015 to 2019. The results reveal that research primarily concentrates on the higher education sector while only a few studies are available for the school and other educational sectors. In terms of research methodologies, quantitative approaches are prevalent, with most of them being survey-based. The main research interests of the empirical studies lie in the perception of OER and the barriers to their use in educational practices. Open textbooks as a form of OER and their comparative cost advantages or qualitative comparability with traditional educational material constitute an emerging research field that is almost exclusively located in the U.S. Research gaps exist regarding the usability and user-friendliness of OER repositories. Addressing these gaps could support the numerous initiatives in different countries to establish and equip repositories. A dditional gaps for empirical research were identified regarding the effects of the use of OER on pedagogical approaches and established educational practices.


INTRODUCTION
The idea of Open Educational Resources (OER) has emerged as an influential concept in the current discourse on the digitisation of education (Bozkurt et al., 2019;Zawacki-Richter et al., 2020). However, OER can meanwhile look back on a history of almost 20 years. In 2002, the term OER was initially coined by the UNESCO Forum on the Impact of Open Courseware for Higher Education in Developing Countries (UNESCO, 2 / 18 2002). In the course of numerous international follow-up conferences, OER has been recognised as a nucleus of the demand for open education . A preliminary peak of this demand was the recommendation for OER by the UNESCO General Conference in 2019 (UNESCO, 2019).
The core objective behind OER is to deliver digital teaching and learning resources openly available and accessible for any potential user without restrictions, usually through using Creative Commons (CC) licenses (Wiley & Hilton, 2018). Although no canonical definition exists, the latest definition provided by UNESCO describes OER as "Learning, teaching and research materials in any format and medium that reside in the public domain or are under copyright that have been released under an open license, that permit no-cost access, reuse, re-purpose, adaptation and redistribution by others" (UNESCO, 2019, p. 3 f.).
Despite this long history and the current broad recognition of OER, the expressed demand for OER is contradicted by its low adoption in educational practices worldwide (Mishra, 2017;Otto, 2019). The eminent problem is that the discussion about OER often remains at a conceptual level. Only a few initiatives exist in different countries to implement OER in educational practices that primarily depend on third-party funding (UNESCO IITE, 2019). In addition, especially recent events such as the Covid-19 pandemic have revealed the general need for OER across all educational sectors. This need can be illustrated by the case of Germany, where the Federal Ministry for Education and Research (BMBF) made closed repositories accessible for schools to enable the nationwide use of digital teaching and learning material freely and thus also for remote teaching (BMBF, 2020).
Apart from this recent OER prominence due to Covid-19, a glance at the research literature suggests that the state of research on OER is still complex and partly fragmented. As aforementioned, a significant part of the existing literature focuses on conceptual discussions about OER. These conceptual discussions are visible in the debate about OER's core definition and follow-up concepts, mainly Open Educational Practices (OEP) and Open Pedagogy and their interference with OER (Green, 2017;Mishra, 2017;Wiley & Hilton, 2018). Besides, a rather philosophical strand of the discussion revolves around the virtues of OER as part of the broader debate about openness and the normative aspect of educational justice (Hodgkinson-Williams & Trotter, 2018;Lambert, 2018;Peters, 2014;Peters & Peter Roberts, 2011). What, however, is missing or obscured are systematic overviews about evidence-based research perspectives on OER. While many OER assumptions occur on a conceptual level, empirical studies that corroborate these assumptions are often absent or unidentified. This ambiguity implies a closer look at the fundus of the available empirical studies on OER to determine the current state of the empirical research literature. Although this status determination is a prerequisite to identifying trends and gaps for further research on OER, such a literature review does not exist from an empirical perspective. Although OER can look back on almost 20 years, only a few systematic reviews (Clinton, 2019;Zancanaro et al., 2015) and meta-analysis (Otto, 2019) of empirical research exists. However, one explanation for this lack might be because OER is a concept with no evident roots, and research thus occurs in various scientific disciplines and educational sectors (Bozkurt et al., 2019;Otto, 2021).
Based on these observations, we seek to fill this gap by mapping the current empirical research on OER. We are confident that this can serve as an essential basis for further steps in empirical research on OER and disclose desiderata that can lead to establishing a future research agenda. The latter is also necessary to provide educational policymakers and the public with evidence-based research findings that could strengthen the position of OER in the broader educational debate.

CONCEPTUAL APPROACH AND RESEARCH QUESTIONS
For the generic exploration of an unclassifiable research field, it is essential to select an adequate conceptual approach. For the case of OER, an aggravating factor for this exploration is that the concept itself has no disciplinary roots or an initial subject area. Moreover, OER cannot be assigned to a specific educational sector (Otto, 2019(Otto, , 2021. Consequently, research stimuli are to be expected from any scientific discipline or educational sector. In order to address these challenges, we use a systematic mapping approach for our 3 / 18 review. The initial idea of systematic mappings stems from medical research, but the first practical implementations can be found in software engineering (Fernandez et al., 2011;Petersen et al., 2008). However, systematic mappings have also gained prominence in educational technology (Dicheva et al., 2015;Rasheed et al., 2019).
A systematic mapping examines the existing research literature on a subject area to obtain a first generic overview of the existing research contributions, especially the different types of research carried out so far. It thus offers the possibility to map a research area by identifying the quantity and quality of available studies, the types of research, the methodologies used, and the main focuses within the research area (Petersen et al., 2008). Systematic mappings also facilitate revealing publication trends by mapping the frequency of publications over time. Besides, the objective of a systematic mapping can be to identify prevalent scientific disciplines and educational sectors in which research results are published. Concisely, a systematic mapping provides a generic overview of a research area.
In contrast to systematic mapping approaches, the more frequently used systematic review approach examines relevant studies in an already established research area to rigorously scrutinise, evaluate and interpret findings available or a specific research question (Newman & Gough, 2020). On the other hand, a systematic mapping process aims to map an emerging research field that has not yet been explored entirely. With its core aim to provide an overview of the entire spectrum of research on a topic, a systematic mapping is particularly suitable for the case of OER. Although the literature implies the emergence of a research field (Bozkurt et al., 2019;Zawacki-Richter et al., 2020), there is a lack of an inventory and systematisation of the available empirical studies. Reviews for OER such as narrative reviews (Clinton, 2019), bibliographic analyses (King et al., 2018) or meta-analyses (Otto, 2019) are hitherto hardly available. Therefore, the benefit of systematic mappings lies mainly in its capability of identifying desiderata for a future research agenda.
In order to achieve the objectives mentioned above, we investigate two main focuses based on the mapping study. In the first step, we characterise the OER studies at a descriptive level. For this purpose, the following research questions were derived: -What are the focal areas in OER research so far? Can prevalence be identified concerning subject disciplines, educational sectors and countries?
-What are the research methods used in empirical OER studies?
In the second step, we examine the main content areas of the OER studies at the empirical level, which is reflected in the third research question: -What are the prevailing thematic empirical focuses on OER research?

METHODOLOGY
The methodology of our article follows the systematic mapping approach of Petersen et al. (2008). Accordingly, the systematic mapping process steps are the definition of research questions, the search for relevant literature, the screening of the literature, the classification of abstracts, and the data extraction and mapping of the results (see Figure 1). Figure 1. Systematic mapping approach of Petersen (2008) To map the status of empirical studies on OER as comprehensively as possible, we aimed to identify all empirical studies on OER based on the research questions mentioned above. However, although this systematic mapping study aims to provide a comprehensive representation of the international empirical research literature on OER, there are limitations due to the criteria used in the research process that have to be considered.
We searched the Web of Science, Scopus and ERIC as these are accepted to be the most relevant databases for peer-reviewed resources in the field of educational technology Ramirez-Montoya, 2020). Whereas Web of Science Core Collection and Scopus ensure interdisciplinary and broad geographical coverage (Mongeon & Paul-Hus, 2016), ERIC focuses on educational science and, therefore, especially on OER.
For our search, we defined inclusion and exclusion criteria ( Table 1). Publications were included from 2015 up to the end of 2019. We decided to set 2015 as a starting year for our mapping as by then, the debate about the core definition of OER had ended mainly with Wiley (2014) introducing the 5Rs of OER combined with Wiley et al. (2014) requesting to increase research on OER. To reflect the international discourse as best as possible only publications in the English language were considered. Only journal articles were included in the systematic mapping, as this publication type is the main focus in the selected databases (Mongeon & Paul-Hus, 2016). Other document types, such as book chapters and conference papers, are not comprehensively represented in the databases and were thus excluded. Using the fixed sequence "Open Educational Resources", we searched Web of Science and Scopus via the search fields Title, Abstract and Keywords. In ERIC, we searched across all fields, as abstract does not exist as a single search field. The number of results was narrowed down according to the listed inclusion and exclusion criteria of publication year, publication type and language. Based on the principle that key concepts in scientific publications are initially written in the abstract, the acronym "OER" was omitted from the search. In this way, a lower result rate (recall) of related publications was accepted in favour of the accuracy (precision) of relevant hits (Ting, 2010).
After reducing the number of duplicates from the three different databases, the search carried out on 07.01.2020 resulted in a list of 550 titles. These were subject to a screening process based on the abstractsand, if necessary, the full texts -to identify relevant publications. As a result, 278 titles were excluded that 5 / 18 did not represent either empirical studies (139 titles) or whose primary thematic focus was not on OER (139 titles). Thus, 272 publications were finally included in an in-depth analysis.

Figure 2. Research process and selection of publication
For our analysis of the publications, we used a category scheme developed and validated based on a deductive-inductive procedure (Mayring, 2000). All studies were analysed and categorised according to formal and content-related criteria. As a first step, we deductively defined first-level categories that we derived from our initial research questions established at the beginning. Then, subcategories subsumed under these first-level categories were derived inductively based on the publications' content. Therefore, we used a random sample of 10% of the publications to classify the subcategories. Based on this sample, the subcategories were finally defined, evaluated and agreed upon in joint discussions. In addition, the sample was used to test the intercoder reliability and disclose discrepancies in the coding that required adjustment. The titles that were included in the random sample were coded by three researchers, with a median Cohen's Kappa of 0.68, which represents a substantial agreement (Landis & Koch, 1977). With this category scheme, including the approved definition of subcategories, we conducted the final analysis and categorised all 272 studies.
The category scheme ( Table 2) contained eight categories, whereas scientific disciplines and countries or continents in which the studies were conducted are not presented in Table 2 for clarity reasons. Moreover, educational areas such as school, higher education, further education, vocational education and cross educational were included as a formal category.
The studies' research methods were divided into quantitative methods (data analysis, survey, experiment, meta-analysis, reviews), qualitative methods (interviews, observation, case studies and interpretative approaches) and mixed methods.
On the content level, the category main investigation focus disclosed whether the studies examined the practical applications of OER in teaching or learning contexts (learner focus) or the creation and use of OER by teachers (instructor focus). Additional focuses were an institutional focus that concentrates on the strategic implementations of OER, for instance, to establish a culture of sharing in institutions or the potentials and limitations of OER at the macro level. Finally, a technical focus was coded when, for instance, studies concentrated on repositories, infrastructure or the findability of OER.

/ 18
Moreover, our objective was to reveal the research topics of the studies. Therefore, in order to obtain a comprehensive picture, an article could be assigned to more than one category. This approach resulted in the distinction of the following categories: technical infrastructure; usage and adoption of OER comprising the creation, adaptation and sharing of materials; perception and attitude towards OER; learning outcomes of using OER in learning settings; effects of OER on costs, time or collaboration; barriers for OER; quality of materials and strategic aspects of OER.
For the next category, we identified related concepts when these were subject to the OER studies. These concepts encompassed, among others, Open Pedagogy, Open Textbooks, Open Science, Massive Open Online Courses (MOOCs), and Open Educational Practices (OEP). Related concepts are of particular interest because different opinions occur in the literature on whether these are interrelated and can, thus, all be subsumed under the broader umbrella of openness or if they are not interwoven but instead loosely coupled (Otto & Kerres, In Press). In general, the relationship of OER to the other concepts of openness requires further research to examine it in more detail. The last category of our analysis was the identification of the primary contribution of the OER studies. The first subcategory was the sole evaluation of measures, e.g., in the form of lessons learned. The second subcategory was the development of recommendations when the study intended to derive concrete guidelines or suggestions. Other than lessons learned that look backwards at the case investigated, recommendations intend to inform or give guidelines for further studies on OER or their use. The category teaching strategies was coded when studies aimed to develop concrete approaches on how to integrate or use OER in teaching contexts. The fourth category was a contribution to theory building; for instance, applying or developing a theory in the study conducted.

RESULTS
The For a better overview of the origin of the studies, i.e., the countries where they were conducted, the studies were cumulated to the respective continent (Figure 3).

Figure 3. Distribution of titles by continent, in percent
The distribution of the studies across continents shows that with 37%, more than one-third of the studies were conducted in North America. Moreover, most studies originate in the U.S., followed by studies from Europe and Asia, with about 19% and 17%. Studies from Africa and Latin America seem rare, with roughly 7% and 4% -although a need for OER is frequently uttered precisely for these regions (Lambert, 2018). For 14 publications, no precise location could be identified, while 20 studies were conducted on more than one continent.
When looking at the educational sectors, it gets clear that empirical research primarily focuses on the higher education sector (Figure 4): Approximately 71% of the studies that were examined can be assigned to the higher education sector. In comparison, only 14% originate in the school sector. Other educational sectors, such as vocational training (<1%) and further education (approx. 2.6%), tend to be on a marginal level. Crosseducational research could be recorded for about 13% of the studies and thus plays a substantial role in empirical OER research. This category includes reviews or studies covering a broad range of topics and does 8 / 18 not relate to a specific education area. It also includes studies that, for example, investigate the use of repositories and thereby do not collect data regarding the educational sector.

Figure 4. Distribution of studies to educational areas, in percent
The following allocation of the studies to subject areas and scientific disciplines is based on the subject classification system of the German Research Foundation (DFG 2020). The results clearly demonstrate that interdisciplinary studies account for more than half of the studies (almost 56%). Humanities and social sciences are followed by almost 24% and natural sciences with about 12%. About 5% of all studies were assigned to the life sciences, and almost 4% to engineering sciences (Figure 5).

Figure 5. Distribution of studies according to disciplines, in percent
Concerning the research methods used in the studies, quantitative methods are prevalent with slightly 63%. In comparison, qualitative methods account for a relatively small share with almost 20% and are at a similar level to mixed-method approaches with a share of around 17% (Table 3). When considering the three types of methods in more detail, it becomes clear that surveys and data analyses (descriptive and correlation analyses, experimental and quasi-experimental designs, data mining, learning analytics) are primarily carried out in the quantitative field. On the other hand, interviews and interpretative studies predominate in qualitative studies.

/ 18
After the analysis on the formal level, the content level analysed the studies main investigation focus and research topic. For the main investigation focus, only the leading focus was revealed for each study. However, this does not exclude the possibility that multiple investigation focuses might be present within one study.
When categorising the studies according to the main investigation focus (Figure 6), it becomes clear that the investigation of learners (learner focus) is the most frequent focus with around 39%, followed by teachers/course leaders (instructor focus) with 33%. Only about 14% of the studies could be assigned to the institutional level (institutional focus). In comparison, technical aspects (technical focus) were identified as the main investigation focus in only 12% of the studies.

Figure 6. Distribution of studies by primary focus of the investigation, in percent
Furthermore, the studies were differentiated regarding their research topic (Figures 7 and 8). For this category, different subcategories could be assigned to each study in order to be able to cover the entire spectrum of existing topics. As Figure 7 illustrates, no clear trends occur regarding the numbers and distribution of the research topics per year.
It is striking that almost 30% of the studies were being assigned to the category usage, adoption (Figure 8).
The comparatively high number of studies assigned to this category appears stable, respectively, slightly increasing over time (Figure 7). On the one hand, studies examining the creation and use of OER by teachers rarely by learners are, for example, using the OER life cycle (Schuwer & Janssen, 2018) or the OER pyramid (Baas et al., 2019). On the other hand, various studies present lessons learned from the creation and use of OER in learning settings (Hassan et al., 2019) or make evidence-based recommendations on integrating them (Sandanayake, 2019). Studies that focus on teachers/course instructors or learners frequently examine attitudes towards or perceptions of OER in addition to the creation and use of OER. While perception can be understood as a cognitive reaction to external stimuli, attitudes constitute an evaluation of a person towards an object or event that preludes action or behaviour (Otto, 2021).
Corresponding studies were assigned to the category perception, which applied to about 18% of all studies. On the one hand, this includes studies that aim to investigate perceptions of and attitudes towards OER, e.g., the associated opportunities or challenges and the willingness to share resources (Ozdemir & Bonk, 2017).
On the other hand, the category also includes studies in which the perceptions of and attitudes towards OER were surveyed after using them in a learning setting (Illowsky et al., 2016). Finally, studies that focused on learners repeatedly include evaluating learning outcomes achieved with OER -overall, almost 12% of the studies were being assigned to the corresponding category learning outcomes. These studies compared, for example, in experimental settings or using surveys or learning analytics, the extent to which learning  (Hilton, 2019). Many of these studies indicate that the use of open versus closed materials has no significant impact on learning outcomes (grades, completion rate, dropout rate). Accordingly, the state of research indicates that OER may well be an adequate substitute for closed content that can be used to save costs and improve the fit between learning materials and content (Cuttler, 2019).
In this regard, an analysis of the category effectiveness, to which 11% of the studies could be assigned, reveals that effectiveness mainly concerns learning outcomes and aspects such as costs and time efficiency. Many studies in this category evaluate open educational materials regarding their quality and suitability for teaching and learning contexts. In this context, various studies, especially from the U.S., investigate the use of open textbooks and their cost advantages compared to closed educational materials. This research focus is important as it evaluates the general assumption that OER use can reduce overall costs in education. In the U.S., respective studies can be located primarily in the higher education context. Frequently, these studies compare the quality and acceptance of open course materials with closed course materials and evaluate the substitution of paid materials with open equivalents. Approximately 6% of the studies examined had such a focus, which also becomes evident in the analysis of related concepts (Table 4). Here, it was investigated whether concepts thematically related to OER were subject to the studies. Remarkably, topics such as Open Pedagogy and Open Educational Practices (OEP) could only be identified in three, respectively, four studies. However, one cause for these concepts' marginal presence might be the limitation of searching only "Open Educational Resources" and not "Open Educational Practices" or "Open Pedagogy" specifically. Another explanation might be that the concepts are implicitly used in some of the studies but not explicitly mentioned due to the absence of an established definition in the literature (Cronin & MacLaren, 2018;Koseoglu & Bozkurt, 2018;Wiley & Hilton, 2018). It is also noteworthy that Open Access and especially Open Science appear not to play a notable role in empirical OER research. Further research will be necessary to explore whether this shortage is only valid for empirical research on OER or also for conceptual discussions. Originally, OER evolved in part out of the Open Access and Open Science movement.
The category quality of materials was found in 7% of all studies. One strand of the studies addresses the evaluation of open materials based on various criteria such as relevance, aesthetics, comprehensibility, and content quality (e.g., Cuttler, 2019). A second strand addresses the process of quality assurance in the (partly collaborative) creation and evaluation of open educational resources (e.g., Marín et al., 2019), while a third strand focuses on the analysis criteria for evaluating OER (e.g., Yuan & Recker, 2015).
Furthermore, it is striking that in the studies assigned to the research focusses mentioned above, primarily teachers or learners were investigated, while other actors, e.g., librarians or administrations, are hardly ever examined. This changes when looking at the categories technical infrastructure and strategy with around 10% respectively 5% of the studies. Here, the main focus is not on how OER are created, used, perceived or evaluated, or the effects of their use. Instead, it is investigated how the conditions for OER use can be created or improved on a structural level or how implementation strategies or policies can be developed or implemented. In addition to learners and teachers, other actors, e.g., technical support, libraries or nonteaching staff, also play a role.
The infrastructure category contains studies that focus on environments, systems, methods, or tools that enable or enhance finding and curating or creating, using, remixing, and sharing OER (Moundridou et al., 2019). Besides, this category includes studies examining library staff's role in helping faculty use OER (Braddlee & Vanscoy, 2019). Finally, the category strategy contains studies that, for example, deal with the factors that influence the widespread use or dissemination of OER from the perspective of an institution 12 / 18 (Masterman, 2016) or how to initiate and expand OER activities, taking into account various aspects such as institutional culture, overarching strategies and policies, and prevailing business models (cf. Jung et al. 2017).
In this respect, it is worth mentioning the examined barriers that undermine the use of OER. About 4.5% of the studies deal with this topic and can thus be assigned to the category barriers. The barriers stated in the studies are highly heterogeneous and range from a lack of policies, incentive systems, technological infrastructure and awareness of OER to difficulties in finding materials or insufficient knowledge or skills (Henderson & Ostashewski, 2018). Over 4% of the studies examined how to deal with motivational and other factors that influenced the use of OER and were thus assigned to the category incentives, motivation (Belikov & Bodily, 2016).
Remarkable results emerge when looking at the category main contribution in Figure 9, which clearly shows that most of the studies (almost 80%) focus on evaluating implemented projects, measures, or interventions.
In contrast, hardly any contributions develop concrete recommendations for further designing, researching, or promoting OER. Also scarcely available are contributions on theory building or the elaboration of teaching or implementation strategies. Despite limitations that might be rooted in the search process, this indicates that empirical research on OER is still in its infancy and those new and compelling theories and models for OER have still to be established. For this very reason, this hitherto understudied aspect -at least in the context of educational technology -forms an opportunity to focus more on theories of pedagogical action and teaching concepts (Wiley & Hilton, 2018).

DISCUSSION
Our article aimed to disclose the status of empirical research on OER by using a systematic mapping approach.
The results, prima facie, allow us to conclude that OER, in empirical terms, is in the emergence of a research field. This emergence is reinforced by the total number of 272 studies that met the scientific quality criteria and were included in our systematic mapping. The following findings could be derived which answer our initial research questions raised at the beginning: What are the focal areas in OER research so far? Can prevalence be identified concerning subject disciplines, educational sectors and countries?
Concerning the geographical distribution, empirical research on OER is primarily conducted in North America (Figure 3). The European and Asian regions follow at a significant distance. It is striking that few empirical studies are available for regions often considered in the literature to be the major profiteers of OER, namely countries of the global South (King et al., 2018;Otto & Kerres, In Press).

/ 18
The studies' distribution among educational sectors clearly shows higher education prevalence, followed by schools (Figure 4). However, it can also be noted that a considerable number of studies are conducted across educational sectors. The latter is especially the case for topics such as OER repositories or when surveys measure the acceptance and perception of OER.
The fact that OER cannot be assigned to a specific educational sector is also valid for allocating the studies to subject areas (Figure 5). More than half of all studies are undertaken in more than one subject discipline.
Future research would be important to expand the range of educational sectors, especially for further and vocational education. Moreover, there needs to be a shift from the dominant focus on North America to the global South, where the perception and implementation require further studies.

What are the research methods used in empirical OER studies?
In terms of methods, a clear emphasis is on quantitative studies, especially survey-based ones ( Table 3). The latter may follow from the fact that most of these studies' prominent research focuses is the investigation of the perception and acceptance of OER by different groups of actors, for example, whether teachers are aware of OER or how they assess and evaluate their quality, which is primarily assessed through surveys. Data analyses constitute a second focus in terms of used methods. This includes, for example, investigating OER repositories and the nature and quantitative availability of OER. Especially the latter, along with a lack of awareness, is one of the significant problems for the adoption of OER (Otto, 2019).
In contrast to quantitative methods, studies that use qualitative methods are available to a much lesser extent. Given that studies on OER usage are thematically the most common, qualitative approaches could be used to examine the creation and adaptation of materials in teaching and learning practices in more detail.
From a methodological point of view, the limited availability of meta-analyses or reviews underlines the fact that OER research is a field in the process of emergence. A central problem in this respect appears to be the limited availability of high-quality studies, especially on more subject-specific issues (Clinton, 2019).

What are the prevailing thematic empirical focuses on OER research?
Concerning the studies' leading empirical focuses, it became evident that they are mainly directed at learners or instructors (Figure 6). The primary objective here is to examine the perceptions and experiences of both groups with OER. It is noteworthy that OER are usually studied in contrast to conventional/closed educational materials and not as the exclusive object of investigation. The latter type of research is often based on individual case studies or field reports of teachers. On the other hand, studies that shed light on institutional or technical conditions for OER are underrepresented. This is also confirmed by other findings and reports, which state that so far, hardly any country has adequate political or structural conditions for the sustainable adoption of OER (Orr et al., 2017;UNESCO IITE, 2019). Overall, this corroborates the fact that follow-up questions on the technical and infrastructural standards for the construction and design of OER repositories are mainly unanswered, with few available empirical findings (Clements et al., 2015;Kerres & Heinen, 2015;Santos-Hermosa et al., 2017).
These distinctive tendencies in the research focuses are further substantiated by the results of the more differentiated thematic research topics (Figures 7 and 8).
Only a few studies address the development of institutional strategies and the technical infrastructure for OER. Instead, the dominant number of studies concentrates on factors directly related to OER: How are OER perceived or used? Can OER achieve comparable learning outcomes to traditional materials, and are they as efficient?
The last aspect manifests in numerous studies on the use of open textbooks (Table 4). Notably, these studies are exclusively located in the U.S., where the idea is receiving much attention, especially against the background of educational materials' exorbitant costs (Clinton, 2019;Hilton, 2016). To what extent the use of open textbooks is also interesting for other countries could be one aspect of further OER research. Comparatively underrepresented among related concepts are those that have developed based on OER and are the subject to vivid discussions, at least at the conceptual level. However, neither the idea of OEP, understood as the effects of the use of OER on established individual and institutional practices (Cronin & MacLaren, 2018), nor the concept of Open Pedagogy, which some authors have derived from the distinctive characteristics of OER (Wiley & Hilton, 2018) have been implemented in empirical studies of this systematic mapping to any substantial extent. Further research is, therefore, necessary to investigate both concepts in more detail.
The last point of the studies' primary contribution (Figure 9) reinforces the impression that empirical research on OER has concentrated primarily on broadening the corpus of available empirical studies. This corpus mainly encompasses individual measures evaluated and reported on, predominantly at a university or course level. The often criticised lack of theory-based approaches and/or contributions to theory development for the field of educational technology (Hew et al., 2019) can also be confirmed for the case of OER. Therefore, a desideratum for OER research could be to substantiate empirical findings by applying educational technology concepts and theories. This could also facilitate the systematising of the empirical findings available. On this basis, another underrepresented aspect, the concrete recommendations for OER implementation, could be investigated more in detail.

CONCLUSION
Concluding, research desiderata that should be addressed in future empirical OER research are the theoretical foundation of OER research designs. These are lacking in many studies that were analysed in the present paper. A theoretical foundation could lead to a better understanding of OER and more profound and systematic approaches to compare, replicate and validate research findings. In this manner, the key explanatory variables could be identified instead of being only vaguely stated and based on anecdotal evidence. In addition, an important emerging research topic that has rarely been explored is the usability or user-friendliness of OER repositories. OER repositories are the critical infrastructure for making OER available to users and allowing them to up-or download OER. Therefore, not surprisingly, numerous initiatives in different countries worldwide establish and enhance the use of OER repositories and could benefit from empirical data on this topic. Further research gaps that we identified are the little-investigated empirical effects of the use of OER on established pedagogical approaches as well as their potential impacts on conventional educational practices. Both are essential aspects for which, despite vigorous conceptual debates, hardly any reliable empirical studies are available.
To conclude, with our systematic mapping study, we aimed to provide the first impetus to spur the growth of an empirical research agenda for OER. The development of such an evidence-based ground for OER can help to render its potential and anchor OER as a critical aspect for education in the digital world.
Lastly, it must be stated that the implications of this systematic mapping have to be considered against the background of the different limitations described in the methodological section and the discussion of the results. However, these limitations pinpoint critical trajectories that require further and more in-depth research to deepen and expand findings on a hitherto under-researched topic.