Finding Free OER Textbooks Online: Untangling the Web

Abstract

Although Open Educational Resources (OERs) can help reduce costs and maximize access to instructional materials, academics face significant problems in identifying good OER textbooks. This can be traced, in part, to the low quality of many OER directories. This study evaluated more than 350 potentially relevant resources, identifying 95 multidisciplinary and 23 subject-limited OER directories that include a relatively high proportion of free textbooks rather than other OERs (syllabi, assessment materials, etc.). Comparative information is presented for each of the 118 directories, with special attention to those with high recall, high precision, explicit and meaningful selection criteria, and consistently good textbook quality. The results focus on the characteristics of the OER directories, the extent to which they support the discovery of textbooks, the particular directories that are likely to be most useful, and the ways in which the most useful directories are systematically different from the others. There are at least 24 high-quality OER directories, but three—the Open Textbook Library, the B.C. Open Collection, and LibreTexts Commons—are especially useful. By devoting more attention to directories such as these, we can overcome the greatest barrier to the adoption of OER textbooks—the difficulty of identifying titles that can replace conventional texts.

1. Introduction

1.1. Textbooks and Open Educational Resources

Textbooks, central to instruction in many fields, pose three significant problems for further and higher education. The first of these is their cost. In the US, the average price of a university textbook is USD 105, and the average annual rate of increase in price is about 6% [1]. In the UK, textbook prices are somewhat lower but have also been rising rapidly—about 80% over the past 10 years [2]. A second problem is that the mechanisms traditionally used to provide shared access, such as library course reserves, are prohibited or made ineffective by many publishers’ licensing agreements. For instance, many of the largest textbook publishers, both commercial and non-profit, will not provide access to their digital texts through any mechanism other than single-user licenses. Essentially, they will not sell to libraries [3]. Finally, and most importantly, high textbook prices have left many students without any reliable means of access to content that is central to their instructional programs. Students may intend to use their friends’ online textbooks, for instance, only to find that the publishers’ digital rights management restrictions do not allow the large-scale printing or copying of content [4]. Two-thirds of US university students report that they have gone without access to required textbooks because they could not afford them [1], and surveys suggest that 18–35% of UK university students are in a similar situation [5,6,7].

Open Educational Resources (OERs)—textbooks and other instructional materials that are freely available online—provide one means of circumventing the problems associated with conventional textbooks. In general, the term OER includes all resources that are intentionally made available by the authors or publishers without payment (e.g., distributed under a Creative Commons license) and are meant primarily for use in teaching/learning environments rather than specialized research settings. OERs include textbooks as well as lesson plans, syllabi, instructional guides, videos, online tutorials, handouts, teaching activities, homework assignments, assessment instruments, and other resources [8].

Although many authors have adopted a very broad view of OERs and open educational practices [9,10], it is OER textbooks that provide the most direct means of reducing barriers to access and learning. Nearly 90% of academics consider cost an important or very important factor when selecting textbooks. Likewise, cost to students is the single textbook attribute that instructors find least satisfactory [11]. Because OER textbooks can substantially reduce costs while maintaining educational effectiveness [12,13], we might expect a large-scale switch from conventional textbooks to OERs.

1.2. Deficiencies of Many OER Directories

In fact, however, OER textbook adoption rates are low. Although 70% of US postsecondary faculty require textbooks for their classes and 42% are aware or very aware of OERs, just 15% have adopted them [14]. A major obstacle to the more widespread adoption of OER textbooks is instructors’ difficulty in finding high-quality works that can replace the conventional textbooks currently in use. Surveys, interviews with academics, and reviews of the literature all reveal that poor discoverability is the single greatest barrier to adoption [11,15,16,17]. Worldwide, 58% of postsecondary instructors who use OERs face difficulties finding appropriate resources in their subject areas, and 53% do not know where to search for them [18].

The low quality of many directories (lists, databases, and catalogues) of OER titles is a particular concern. While hundreds of resources claiming to be directories of OERs can be found online, many of them are mislabeled or otherwise unhelpful. The process of compiling data for this study revealed two problems that are especially prevalent. First, a web resource that claims to be one thing is often another. A “directory of OER textbooks” may actually include many other items: instructional resources other than textbooks; Open Access books intended for research rather than teaching; lists of OER publishers, funding agencies, and publishing platforms; and guides to the creation or use of OERs. Some websites present individual OER titles alongside OER collections or directories, so a particular link may represent a single title or a database with thousands of titles.

Second, many OER directories present a small number of full-length textbooks along with a much greater number of lesson plans, syllabi, short instructional guides, handouts, teaching activities, exercises, homework assignments, and assessment instruments. In some cases, it is difficult to find the few real textbooks among the many short, ephemeral, and even trivial items included in the directory. According to Fernando Bleichmar, an executive vice president at Cengage, many OER resources are especially well suited to short, targeted instructional programs that focus on students’ immediate needs [19]. Their limited scope often makes them less appropriate as replacements for conventional textbooks. The difficulty of distinguishing between full-length textbooks and other resources is exacerbated by the limited or ineffective search functions available within many OER directories [20].

A number of lesser problems can also be identified. For instance, many OER directories:

• Present works about OERs or Open Education as if they were OERs appropriate for student use;
• Make no distinction between OERs and other Open Access books;
• Make no distinction between OER textbooks and other kinds of OERs (videos, lesson plans, class activities, etc.);
• Label resources in ways that misrepresent their content (e.g., using a textbook code/limiter for resources other than textbooks);
• Provide ineffective oversight for user-added items so that low-quality OERs are not excluded [21];
• Duplicate other OER directories, either entirely or partially;
• Include works that are out of date or no longer available;
• Place great emphasis on licensing terms and formal accessibility standards while ignoring important practical concerns such as ease of downloading and printing;
• Highlight locally produced resources (or those funded by the sponsoring agency) even when those resources are clearly less useful than those created elsewhere;
• Provide limited information on the methods used in their construction, so that users have no way to readily judge the completeness, reliability, or comparative utility of the directory or its contents.

1.3. Research Design

This study identifies and evaluates 118 OER directories discovered through a Google search. Although more than 350 potentially relevant resources were initially identified, just 118 proved to be legitimate OER directories that include a substantial number of textbooks. That is, just 118 met the criteria described in Section 2: inclusion of individual OER titles, listed separately; exclusion of other items; emphasis on works suitable for high school and university instruction; and inclusion of more OER textbooks than other OER materials.

The results present comparative information about each of the 118 directories, highlighting those with high recall, high precision, explicit and meaningful selection criteria, and consistently good textbook quality. The investigation also addresses four questions:

• What are the characteristics of the OER directories currently available online?
• To what extent do these directories support the discovery of OER textbooks (i.e., texts that can be used as the central reading material for an undergraduate course or module of at least 10–15 weeks’ duration)?
• Which particular OER directories are most likely to be useful to postsecondary instructors?
• In what ways are the most useful OER directories systematically different from the others?

The research design is described in more detail in Section 2.

1.4. Previous Research on OER Directories

Just a few studies have addressed questions similar to these. Santos-Hermosa and associates [22] evaluated 110 OER repositories, focusing on (a) the extent to which the repositories facilitate the reuse of OERs (revision, recombination, etc.) and (b) the extent to which they support specifically educational use through limiters, search mechanisms, and metadata. The report does not present results for any particular repositories, however; it includes only aggregate results, such as summary statistics. Santos-Hermosa and associates conclude that most OER repositories support the reuse and reconfiguration of materials but do not provide the tools that would make them truly effective for educational use. They also note that while 44% of the repositories have stated selection criteria, many of those criteria are very basic or incomplete. Just 20% require the review of OERs by the staff of the sponsoring organization, and just 10% use any kind of peer review.

Perifanou and Economides [23,24] present an in-depth analysis of 13 OER repositories and directories—websites that host OERs or provide links to the OERs available elsewhere. They report on nearly four dozen variables related to the sites’ basic characteristics, interface quality, content, popularity, use, and accessibility. Their analysis is valuable, especially with regard to accessibility and use metrics (visitors, links, time on site, bounce rate, etc.), but it covers just 13 websites and does not focus on textbooks, specifically. Every one of the thirteen resources includes a high proportion of non-textbook OERs.

Another study by the same authors [21] includes data on 20 OER textbook directories, evaluating their basic characteristics (six variables), their popularity (seven variables), their technical attributes (speed, mobile-friendliness, accessibility), and their visitors’ engagement (duration of visit, pages per visit, and bounce rate). Twelve of the directories examined by Perifanou and Economides are included in this study as well. Section 4 compares their main findings with the results of this investigation.

2. Materials and Methods

A Google search for directory “Open Educational Resources” (not limited by date or other characteristics) was conducted on 22 January 2024. It yielded 278 search results. For the first 139 results, I checked all parts of each website for (a) directories of individual OER titles (individual books, lesson plans, etc.) and (b) lists or directories that present OER-related resources other than individual titles (e.g., collections, publishers, and funding agencies) or that combine multiple types of resources in a single list. Resources of type (a) were added to a tentative list of OER directories. Resources of type (b) were searched or browsed for resources of type (a).

The 139 websites included more than 350 resources that initially appeared to be of type (a). Each was evaluated to determine whether it was truly a directory of OERs (single titles). Potentially relevant resources were excluded if they listed a substantial number of:

• Collections or databases rather than individual titles;
• Books other than OERs (e.g., Open Access books more appropriate for research than for teaching);
• Items that require a subscription or payment for access;
• Items available only to users affiliated with particular institutions;
• Items that are no longer available online (e.g., broken links);
• Non-text materials (videos, images, data), unless intended for language learning;
• Textbooks meant for grades K–8.

Directories were also excluded if they listed relatively many non-textbook OERs—if textbooks comprised less than 50% of the items in the directory.

In February 2024, I consolidated duplicate entries; standardized the directory and producer/sponsor names; and standardized, updated, and corrected the URLs. Altogether, the first 139 Google results led to a list of 118 OER directories that met all the criteria for inclusion in the study—95 broadly multidisciplinary directories and 23 subject-limited directories (those that cover just one subject or a set of closely related fields). The scope of the investigation was not limited by subject, so the subject coverage of the OER directories simply reflects the number that have been developed in each area.

Quantitative information, such as the number of textbooks in each directory, was compiled in February and March 2024. Additional qualitative information (e.g., notes on database construction) was compiled and verified in March and April (see Supplementary Tables S1 and S2).

The 24 most useful OER directories—17 multidisciplinary directories and 7 subject-limited directories—were identified based on their high recall (absolute number of textbooks included), high precision (percentage of included items that are textbooks), explicit and meaningful selection criteria, and consistently high quality of textbook content (a subjective assessment based on completeness and rigor, appropriateness for the intended audience, and effectiveness of presentation). The 24 directories that met at least two of these four criteria are coded 1 in column O (high overall utility) of Supplementary Tables S1 and S2. The details of the recall, precision, selection criteria, and textbook quality designations can be found in Section 3.2, which describes each column of the tables.

3. Results and Discussion

3.1. “Directories” That Are Not Helpful for the Identification of Textbooks

As noted in Section 2, more than 350 potentially relevant resources were initially identified, but only 118 met the criteria for inclusion in Supplementary Tables S1 and S2. Most of the resources that did not meet the criteria were mentioned on only a single website, and many were simply lists of OER-related resources (of multiple types) that might be of interest to the instructors at a particular institution. However, 34 of the directories excluded from the study were each mentioned on at least three different websites (

Table 1. Directories mentioned on at least three websites but excluded from Supplementary Tables S1 and S2.

Directory	No. of Website Mentions	Includes Many OA Books Other than OERs	Includes Many OERs That Are Not Textbooks
Creative Commons [25]	15	1	1
Internet Archive: Open Educational Resources [26]	11	1	1
HathiTrust Digital Library [27]	6	1	1
Google (with usage rights filter) [28]	2	1	1
DOAB: Directory of Open Access Books [29]	31	1
IntechOpen [30]	18	1
OAPEN [31]	15	1
Open Book Publishers [32]	15	1
Project Gutenberg [33]	14	1
Open Research Library [34]	11	1
Orange Grove [35]	7	1
EdTech Books [36]	5	1
Library of Congress Open Access Books [37]	5	1
Library of Congress World Digital Library [38]	5	1
National Library of Medicine Bookshelf [39]	5	1
Open Access Books on JSTOR [40]	5	1
Digital Public Library of America [41]	4	1
National Academies Press Open Books [42]	4	1
World Bank Open Knowledge Repository [43]	4	1
AU Press Remix [44]	3	1
Project MUSE: Open Access Content Results [45]	3	1
Unglue.it [46]	3	1
VIVA Faculty Textbook Portal [47]	3	1
Mason OER Metafinder [48]	67		1
Wikibooks [49]	18		1
Wisc-Online [50]	14		1
NSDL: National Science Digital Library [51]	12		1
MedEdPORTAL [52]	6		1
PhET [53]	6		1
HippoCampus [54]	5		1
Khan Academy [55]	5		1
OpenLearn [56]	4		1
Global Text Project [57]	3		1
nanoHUB [58]	3		1

) [25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58], and some of them appear often in online lists of OER-related resources.

As Table 1 reveals, at least 23 directories of Open Access books were excluded from the main analysis because they include many books that are not OERs—research monographs, for instance. See, for example, the Directory of Open Access Books, IntechOpen, OAPEN, Open Book Publishers, and Project Gutenberg [29,30,31,32,33].

At least 15 directories of OERs were excluded because they focus primarily on instructional materials other than textbooks. The most prominent of these is the Mason OER Metafinder [48], a metasearch engine that is often recommended by librarians [59] and that appears on at least 67 websites. Although the Metafinder searches 22 OER databases and provides access to more than 2.2 million items, the results of multiple searches suggest that fewer than 5% of those items are textbooks. Moreover, the search limiters provided by the Metafinder (material type: book, for instance) are error-prone and of limited value to instructors looking for alternatives to conventional textbooks. The Metafinder has high recall but very low precision. Other well-known sites that focus on OERs other than textbooks include Wikibooks, Wisc-Online, and the National Science Digital Library [49,50,51].

Finally, at least four resources that are sometimes listed as OER directories actually include both (a) more general-purpose OA books than OERs and (b) more non-book OERs than OER textbooks [25,26,27,28].

3.2. Survey of OER Directories Included in the Analysis

The directories that did meet the criteria for inclusion in the main analysis are presented in Supplementary Tables S1 and S2. Supplementary Table S1 shows the 95 OER textbook directories that are broadly multidisciplinary in scope. Supplementary Table S2 presents 23 others that each focus on a small number of related subjects. Both tables include the same columns (variables), some of which were suggested by previous research [21,22,23,24]. Specifically, columns A–E indicate:

• A: the name of the directory;
• B: the producer or sponsor;
• C: the producer/sponsor type: university, government agency, for-profit corporation, consortium of universities, scholarly society, or other non-profit organization;
• D: the URL;
• E: the broad subject emphasis (Supplementary Table S1) or the subjects included in the directory (Supplementary Table S2).

Columns F–J provide basic descriptive information:

• F: the total number of items in the directory, rounded to the nearest 10;
• G: the limiter/filter, if any, that restricts the results to textbooks rather than other types of OERs;
• H: the number of items that are retrieved when the textbook limiter is applied, rounded to the nearest 10;
• I: recall; the estimated number of textbooks in the directory, rounded to the nearest 10; for each directory, this is based on a review of at least 60 OERs—those ranked highest, by relevance, on the results pages—or of all the titles in the directory, if fewer than 60; in each case, I searched for subject areas where it would be easy to distinguish between textbooks and other OERs—chemistry, economics, and statistics, in particular; the column I value is generally different from the column H value because the directories’ textbook designations are often applied to resources other than textbooks;
• J: precision; the percentage of directory items that are textbooks, rounded to the nearest 5%; this is based on all the items in the directory, if there is no textbook filter, or on all the items that remain after the filter is applied.

Columns K–O indicate which directories are likely to be especially useful:

• K: high recall; coded 1 if the directory includes an especially high number of textbooks as well as a reasonably high percentage of textbooks—for Supplementary Table S1, a value of at least 500 for column I and a value of at least 80% for column J; for Supplementary Table S2, a value of at least 40 for column I and a value of at least 80% for column J;
• L: high precision; coded 1 if the directory includes an especially high percentage of textbooks as well as reasonably many textbooks—for Supplementary Table S1, a value of at least 90% for column J and a value of at least 100 for column I; for Supplementary Table S2, a value of at least 90% for column J and a value of at least 20 for column I;
• M: coded 1 if the directory has explicit and meaningful selection criteria;
• N: coded 1 if the textbooks themselves are of consistently high quality—a subjective assessment based on completeness and rigor of content, appropriateness for the intended audience, and effectiveness of presentation (writing, tables, and images); these criteria are consistent with those set forth in previous research [60,61,62];
• O: high overall utility, coded 1 if the sum of columns K–N is two or higher.

Columns P and Q represent the prominence of each directory—the extent to which it can be readily found by those who seek to use OER textbooks:

• P: the number of times the directory was mentioned on the websites evaluated for this study;
• Q: coded 1 if the directory is among those that US librarians most often recommend to faculty and students [59].

Columns R–X indicate how the textbooks in the directory are identified and added to the database:

• R: developed and submitted by OER authors commissioned by the organization responsible for the directory;
• S: submitted by other OER authors;
• T: found online by the staff of the organization;
• U: submitted by users of the directory; this sometimes requires user registration and generally requires staff approval of the submitted/recommended items;
• V: found within (taken from) other directories of OERs, usually on a selective basis;
• W: submitted by agencies that have partnered with the organization that produces the directory;
• X: coded 1 if all, or nearly all, of the textbooks in the directory are written by authors associated with the organization that produces the directory.

Columns Y–AC identify

• Y: the directories that are themselves publishing platforms (i.e., those that host OER content), even if only some of the OER textbooks in the directory are actually hosted there;
• Z: those that are subsets or spin-offs of a university’s institutional repository;
• AA: those associated with the BCcampus software/platform (any version);
• AB: those associated with OER Commons (any version);
• AC: those associated with Pressbooks (any version).

Finally, column AD presents notes related to the scope, format, organization, or use of each directory.

3.3. Multidisciplinary Directories

3.3.1. Overview

Several important relationships can be seen in Supplementary Table S1 (Also see

Table 2. OER directories that are broadly multidisciplinary in scope and that include more textbooks than other types of OERs.

Col. A: Directory.	Col. F: Total No. of Items	Col. H: No. of Items with Limiter	Col. I: Actual No. of Textbks.	Col. J: Pctg. of Items That Are Textbks.	Col. K: High No. of Textbks.	Col. L: High Pctg. of Textbks.	Col. M: Meaningful Selectn. Criteria	Col. N: Good Textbk. Quality	Col. O: High Overall Utility	Col. P: No. of Website Mentns.	Col. Q: Rec. by Librarians [59]
Open Textbook Library [63]	1400		1380	100	1	1	1	1	1	127	1
B.C. Open Collection [64]	1520	370	360	95		1	1	1	1	100
LibreTexts Commons (LibreCommons) [65]	7330	2620	2360	90	1	1			1	72	1
Open Culture: 200 Free Textbooks [66]	210		170	80			1	1	1	8
OpenALG: Affordable Learning Georgia [67]	140		130	90		1	1		1	7
MOST: Maryland Open Source Textbook Commons [68]	4370	850	750	90	1	1			1	6
OhioLINK Open Course Content Library [69]	3470	970	870	90	1	1			1	5
Open Textbook Guide [70]	290		280	95		1		1	1	5
OpenEd CUNY [71]	6380	620	560	90	1	1			1	3
OER by Discipline Directory [BCcampus] [72]	610		580	95	1	1			1	2
OER By Discipline Guide Version 2 [73]	190		190	100		1		1	1	2
HBCU Affordable Learning Community Portal [74]	180		180	100		1	1		1	1
Now is the Time for Open Educational Resources [75]	240		240	100		1		1	1	1
OER by Discipline [San Bernardino] [76]	680		650	95	1	1			1	1
OER by Discipline Directory [Victoria] [77]	640		580	90	1	1			1	1
OER by Subject Directory [78]	1000		950	95	1	1			1	1
University of Regina OER by Subject Directory [79]	710		670	95	1	1			1	1
OpenStax [80]	60		60	100				1		129	1
MERLOT [81]	104,680	9970	8680	85	1					113	1
OASIS: Openly Available Sources Integrated Search [82]	455,500	4350	3700	85	1					81	1
Milne Open Textbooks [83]	50		40	90			1			60	1
GALILEO Open Learning Materials [84]	740	150	130	85			1			21	.
eCampusOntario Open Library [85]	8120	480	460	95		1				17	.
COOL4Ed [86]	180	100	90	90			1			11	1
Open Oregon Educational Resources [87]	800		560	70			1			10
New Prairie Press Open Access Textbooks [88]	20		20	95				1		5
OpenNJ [89]	550	40	30	75			1			5
UBC OER Collection [90]	90	40	30	80				1		5
Open College-Level Textbooks [91]	80		70	90			1			4
Open Educational Resources [Oregon State] [92]	50		50	95				1		4
OPENPRESS.USASK.CA Catalog [93]	40		30	85				1		4
University of Minnesota Libraries: Textbooks [94]	20		20	100				1		4
Affordable Learning LOUISiana OER Repository [95]	7070	2590	2070	80	1					2
Open Education Alberta [96]	60		50	75			1			2
Open Textbooks for Hong Kong [97]	100		100	100		1				2
Textbooks and OERs [98]	20		20	100				1		2
Adoption Finder [99]	720		580	80			1	.		1
CommunityArchive@LBCC [100]	120		110	90		1				1
Disciplinary Open Educational Resources [101]	480		410	85				1		1
OER by College and Academic Unit [102]	150		140	95		1				1
OER by Discipline Guide: University of Ottawa [103]	920		790	85	1					1
OER by Discipline: University of Manitoba [104]	1050		890	85	1					1
OER by Subject [IUP] [105]	120		120	100		1				1
OER Discipline Resource Guide: Concordia [106]	330		320	95		1				1
Open Educational Resources [WPI] [107]	170		160	95		1				1
Open Educational Resources by Discipline [108]	900		700	80	1					1
Open Educational Resources: OER by Discipline [109]	180		170	95		1				1
Open Educational Resources: OER by Subject [110]	200		180	90		1				1
Tennessee Open Education Hub [111]	>10,000	3440	2060	60			1			1
OER Commons: Open Textbooks [112]	6963	4556	2440	55						135	1
Pressbooks Directory [113]	6410		4150	65						52
MIT OpenCourseWare [114]	>10,000	10	10	100						49
Saylor Academy Open Textbooks [115]	90		90	100						36
Lumen Learning [116]	50		50	100						29	1
Open Michigan: Find Open Educational Resources [117]	430		220	50						21
Teaching Commons [118]	25,830	600	420	70						16
Skills Commons [119]	16,760	2160	1610	75						15
AMSER [120]	>10,000	190	140	75						9
OERTX Repository [121]	>10,000	1940	970	50						7
SUNY Digital Repository: Open Courses [122]	10		10	90						6
PDXOpen: Open Educational Resources [123]	40		30	75						5
OAsis: COL’s Open Access Repository [124]	5270	210	170	80						4
ScholarWorks@GVSU [125]	22,080	30	30	90						4
#GoOpenVA [126]	>10,000	330	180	55						3
ASCCC OERI Resources [127]	60		40	70						3
Boundless [128]	20		20	100						3
OER Textbooks [129]	70		70	95						3
openNCCC [130]	>10,000	1150	690	60						3
Scholarly Commons: Open Educational Resources [131]	30		30	85						3
Virginia Tech Open Textbooks [132]	50		30	60						3
BCIT Institutional Repository [133]	40		20	60						2
OERSI: Open Educational Resources Search Index [134]	81,210	9510	6850	70						2
PA-ADOPT Bookshelf [135]	10		10	100						2
Sask DLC [136]	>10,000	120	90	75						2
University of Wisconsin Pressbooks Catalogue [137]	60		30	50						2
VIVA Open! [138]	4960	2320	1500	65						2
#GoOpenMichigan [139]	>10,000	500	400	80						1
Digital Commons Textbooks Collection [140]	10		10	85						1
Finding Open Educational Resources [141]	170		140	85						1
KU Libraries Open Textbooks [142]	20		10	75						1
Manifold@CUNY [143]	300		190	65						1
OER by Discipline [Staten Island] [144]	50		40	95						1
OER by Discipline Guide: University of Calgary [145]	1130		770	70						1
OER By Subject [Washington] [146]	50		50	100						1
Open Education: Find OER (by discipline) [147]	70		60	85						1
Open Educational Resources (OER) [148]	50		40	70						1
Open Educational Resources [Mount Saint Vincent] [149]	280		220	80						1
Open Educational Resources Guide: OER by Subject [150]	70		50	65						1
Open Range Wyoming [151]	9710	140	110	80						1
ROAM Repository of Open and Affordable Materials [152]	260		150	60						1
Tennessee Open Education Hub: Tennessee Collection [153]	430		220	50						1
Textbook Affordability: OER [154]	210		180	85						1
U of R Open Educational Resources [155]	10		10	80						1
WISELearn Resources [156]	>10,000	460	400	85						1
ZTC/OER Textbooks [157]	60		50	85						1

Note: The rows (directories) are ordered so that those with the highest overall utility appear near the top. Specifically, the rows are ordered by the sum of columns K–N, then by column P. Yellow shading indicates directories with high overall utility (column O). See the text for detailed information about each column. For additional variables, see Supplementary Table S1.

[63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157], an abridged version of Supplementary Table S1 that presents just some of the variables).

First, the 95 multidisciplinary directories vary greatly in size (column F), from just 10 items to more than 455,000. Although the average size is 9329 (with values of >10,000 interpreted as 10,000), the median size is just 280. This indicates that the size distribution is heavily right-skewed. Two-thirds of the directories have fewer than 920 items (OERs), and 75% have fewer than 5000. At the same time, there are fifteen directories with 10,000 or more items, including three in the 16,000–26,000 range, one with 81,210 items (OERSI [134]), one with 104,680 items (MERLOT [81]), and one with 455,500 items (OASIS [82]).

The number of OER textbooks (column I) also varies substantially, from 10 to 8680. The mean is 597 and the median is 170. Because directories devoted chiefly to non-textbook OERs were excluded from the table, the percentage of items that are textbooks (column J) varies from 50% to 100%, with a mean of 83% and a median of 85%.

Notably, the directories with the most items (OERs) are not necessarily those with the most OER textbooks. Among the directories shown in Supplementary Table S1, the correlation (Pearson’s r) between number of items and actual number of textbooks is just 0.48, and the correlation between number of items and percentage of items that are textbooks is negative: −0.03. Some of the directories with many items include relatively few textbooks; see, for example, the OERTX Repository [121], #GoOpenVA [126], openNCCC [130], and the Tennessee Open Education Hub [153]. Likewise, the directories with a high proportion of textbooks (90% or more) tend to be smaller, with an average size of 1,664 items and a median of just 180.

Finally, Supplementary Table S1 and Table 2 reveal that Canadian universities and government agencies are especially well-represented as producers of OER directories. This is consistent with the central role of Canadian provincial governments in many OER initiatives [158,159].

3.3.2. The Top 17 Multidisciplinary Directories

Instructors searching for OER textbooks are likely to want directories with both high recall (many OER textbooks) and high precision (a high proportion of OER textbooks among the items in the directory). These two constructs are represented by columns K and L, respectively. Columns M and N also represent favorable qualities: meaningful selection criteria and consistently good textbook quality.

Column O combines recall, precision, selection criteria, and textbook quality in a single indicator, high overall utility, coded 1 if the sum of columns K–N is two or higher. Seventeen directories are coded 1 in column O, but three stand out above all the others: the Open Textbook Library, the B.C. Open Collection, and LibreTexts Commons [63,64,65].

Table 3. Characteristics of the top 17 multidisciplinary directories and the other 78—percentage of directories in each category.

Category	Top 17	Others
Producer/sponsor type (Col. C):
University	71	68
Government	0	11
Other non-profit	6	9
For-profit	12	6
Consortium	12	5
Scholarly society	0	0
Usefulness (Cols. K-O):
High no. of textbooks	59	8
High pctg. of textbooks	94	12
Meaningful selection criteria	29	12
Good textbook quality	35	10
High overall utility	100	0
How identified or added (Cols. R–X):
Commissioned authors	12	19
Other authors	29	50
Directory staff	65	35
Directory users	35	27
Other directories	18	3
Partner agencies	0	5
Affiliated authors only	6	26
Platform characteristics (Cols. Y–AC):
Publishing platform	29	55
Institutional repository	0	10
BCcampus	18	1
OER Commons	18	15
Pressbooks	35	13

shows the differences between the top 17 directories—those coded 1 in column O—and the other 78. Although there are no clear differences with regard to producer/sponsor type, the top 17 directories are distinctive in the mechanisms used to identify and select OER textbooks. In particular, the top 17 directories are especially likely to be compiled by the staff of the sponsoring organizations or from items found within (and selected from) other directories of OERs. Conversely, the top 17 are unlikely to include OERs submitted by the authors themselves or by the sponsors’ designated partner agencies. Of the twenty-one directories that accept submissions only from affiliated authors—faculty at the sponsoring university, for example—only one, OpenEd CUNY [71], appears among the top seventeen.

Perhaps surprisingly, the directories that double as publishing platforms—those that themselves host OER content—are especially unlikely to appear among the top 17. The same can be said of the directories hosted in universities’ institutional repositories. However, the top 17 directories do tend to make use of the BCcampus and Pressbooks platforms, two of the three major software systems designed to facilitate the cataloguing and hosting of OERs.

3.3.3. Prominence versus Utility

Column P of Supplementary Table S1 (and Table 2, above) shows the number of times each directory was mentioned on the websites reviewed for this study. Within the set of 95 multidisciplinary directories, the column p values range from 1 to 135, with a mean of 13 and a median of 2. That is, some directories were mentioned up to 135 times, but the majority (50 of the 95) were mentioned just one or two times. The values for the top 17 directories—those coded 1 in column O—are roughly similar, ranging from 1 to 127 with a mean of 20 and a median of 3.

Notably, the directories most often mentioned online are not necessarily the most useful for identifying OER textbooks. Supplementary Table S1 includes six directories that were each mentioned more than 20 times online but have no codes of 1 in columns K–N. OER Commons [112], the Pressbooks Directory [113], and Open Michigan [117] list relatively few textbooks, and each has an ineffective or unwieldy user interface. MIT OpenCourseWare [114] and Lumen Learning [116] each include no more than 50 textbooks, and most of the Saylor Academy Open Textbooks [115] were published in 2012 or earlier.

Conversely, eight directories appear among the top 17 (column O) but are mentioned just once or twice on the websites evaluated for this study [72,73,74,75,76,77,78,79]. All eight include a high percentage of textbooks (at least 90%) among the OER items they list, and five include more than 500 textbooks, overall. Two are notable for their consistently good textbook quality, and one for its meaningful selection criteria. All eight are compiled at least partly by the staff of their sponsoring universities, and six are hosted on the Pressbooks platform. None of the eight include more than 1000 total items, however. This suggests that website mentions are directly correlated with overall directory size—an assertion that is supported only to a modest extent (r = 0.31).

Finally, there are three multidisciplinary directories with both high overall utility (column O) and more than 70 website mentions (column P): the Open Textbook Library, the B.C. Open Collection, and LibreTexts Commons [63,64,65]. As shown in Supplementary Table S1 and Table 2, the Open Textbook Library scores high on all four criteria (columns K–N) and was mentioned online at least 127 times.

Just as overall utility is sometimes unrelated to the number of website mentions, the top 17 directories are not necessarily those that librarians most often recommend. The directories most often recommended by librarians (column Q) [59] include just two of the top 17 directories—the Open Textbook Library [63] and LibreTexts Commons [65]—along with two other directories that satisfy none of the criteria represented by columns K–N: OER Commons [112] and Lumen Learning [116].

3.4. Subject-Limited Directories

Although 95 of the 118 OER directories are multidisciplinary, 23 cover just a single subject or a small number of related subjects. These can be seen in Supplementary Table S2 and in

Table 4. OER directories that focus on a small number of related subjects and that include more textbooks than other types of OERs.

Subject/Col. A: Directory	Col. F: Total No. of Items	Col. I: Actual No. of Textbks.	Col. J: Pctg. of Items That Are Textbks.	Col. K: High No. of Textbks.	Col. L: High Pctg. of Textbks.	Col. M: Meaningful Selectn. Criteria	Col. N: Good Textbk. Quality	Col. O: High Overall Utility	Col. P: No. of Website Mentns.
Art and art history
Smarthistory Books [160]	10	10	100				1		3
History of Art OER [161]	10	10	100						1
Computer science
Green Tea Press [162]	20	20	95		1		1	1	2
FreeTechBooks [163]	1240	930	75						2
Open-Source, Open-Access Literature [164]	10	10	80						1
Economics and business
Lyryx Learning Textbooks [165]	10	10	100				1		4
CORE Econ ebooks [166]	20	20	90		1				1
Economics Network: Text and Notes for Economics [167]	320	160	50						3
Engineering
Open Textbooks for Engineering [168]	280	240	85	1					3
Open Educational Resources [Alberta] [169]	10	10	90						1
English composition
Composition Reading Bank [170]	260	240	90	1	1			1	1
Open-Access Textbooks [171]	20	20	95		1				3
Geography
Open Geography eTextbook Resources [172]	10	10	100						2
Languages
COERLL [173]	10	10	100						5
Indigenous Storybooks [174]	10	10	100						1
Law
CALI: eLangdell Bookstore [175]	70	70	100	1	1		1	1	5
Mathematics and statistics
AIM Approved Textbooks [176]	70	70	100	1	1	1	1	1	14
OpenTextBookStore [177]	40	40	100	1	1	1	1	1	12
Open Access Texts [178]	120	120	100	1	1			1	1
Lyryx Learning Textbooks [165]	10	10	100				1		4
FreeTechBooks [163]	1240	930	75						2
Nursing
Open RN: Open Resources for Nursing [179]	10	10	100				1		4
Physics
Open Access Texts [178]	120	120	100	1	1			1	1
Psychology
Noba [180]	10	10	100						9
OpenPSYC [181]	1	1	100						1
Social work
Open Social Work [182]	70	60	90	1	1			1	4

Note: Directories that deal with multiple subjects appear multiple times, under each relevant subject heading. None of these directories use limiters or filters to distinguish between textbooks and other OERs. None appear among the directories that librarians most often recommend [59]. Yellow shading indicates directories with high overall utility (column O). See the text for detailed information about each column. For additional variables, see Supplementary Table S2.

[160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182], which presents just some of the variables.

As Table 4 illustrates, the availability of OERs (and OER directories) varies by subject. There are at least five subject-limited directories in mathematics/statistics; three each in computer science and economics/business; two each in art/art history, engineering, English composition, languages, and psychology; and one each in geography, law, nursing, physics, and social work.

The subject-limited directories are far smaller than the multidisciplinary directories, with an average size of just 114 items (column F). Likewise, the average number of textbooks is 91, with a median of 20. Another major difference can be seen in the proportion of items that are textbooks. Most of the subject-limited directories are essentially textbook directories, and 13 of the 23 include nothing but OER textbooks—no lesson plans, syllabi, handouts, teaching activities, exercises, homework assignments, or assessment instruments.

In general, the subject-limited directories are more selective—more carefully curated—than the multidisciplinary directories. They also tend to have more straightforward interfaces and to rely less on limiters/filters for the identification of particular subfields and types of OERs. Just 18% of the multidisciplinary directories, but 30% of the subject-limited directories, have codes of 1 in column O (high overall utility).

The seven directories coded 1 in column O are different from the others in just a few respects. They are more likely to be compiled by the directory staff (71% vs. 31%) but less likely to include textbooks commissioned and published by the sponsoring organization (29% vs. 63%). Of the top seven directories, only two (29%) function as publishing platforms; among the other sixteen directories, the percentage is 69%.

As Table 4 shows, the top seven directories (based on column O) represent six different subject areas. However, only one subject area—mathematics/statistics—includes more than one of them. Specifically, the AIM Approved Textbooks [176] and the OpenTextBookStore [177] each score high on all four criteria: high recall, high precision, meaningful selection criteria, and consistently good textbook quality (columns K–N). A third mathematics directory, Open Access Texts [178], scores high on two of the four criteria.

Unlike some of the multidisciplinary directories, none of the subject-limited databases appear to be widely known. None appear on more than 14 of the websites evaluated for this study, and none are included among the directories that librarians most often recommend [59].

4. Conclusions

Of the more than 350 potentially relevant resources identified online, only 118 meet the criteria for inclusion in Supplementary Tables S1 and S2. That is, only 118 are directories of OER titles for which textbooks comprise more than half the items listed. Many websites billed as OER directories are not directories of individual titles at all, and others combine records for individual titles with lists of OER publishers, lists of funding agencies, tips for publishing and modifying OERs, and other resources not intended for student use. Moreover, many OER directories include few full-length textbooks, and some provide no easy means of distinguishing between textbooks and other items—lesson plans, syllabi, homework assignments, in-class instructional activities, and lab manuals, for instance.

Although 118 directories meet the minimum criteria for usefulness as directories of OER textbooks, just 24 satisfy two or more of the four utility standards set forth here: high recall (based on the number of textbooks in the directory), high precision (based on the proportion of retrieved items that are textbooks), explicit and meaningful selection criteria, and consistently good textbook quality. This research confirms that many OER directories are of limited value for instructors seeking free textbooks online. It also shows that the top 24 directories are systematically different from the others in (a) the methods used to identify/add resources and (b) the publishing (software) platforms that support them.

High-quality directories of OER textbooks are likely to be useful to faculty looking for free online textbooks for their courses, to authors and publishers seeking to identify the fields in which new OER textbooks are needed, and to students looking for free textbooks for self-study or as supplements to the texts their instructors have assigned. Individuals interested in finding the best OER textbooks may want to consider the multidisciplinary directories that meet two or more of the four utility standards set forth here. (See column O in Supplementary Table S1 and Table 2). Three directories, in particular, seem especially useful: the Open Textbook Library, the B.C. Open Collection, and LibreTexts Commons [63,64,65].

The best subject-limited directories are also likely to be helpful. Faculty and students in mathematics/statistics, computer science, economics/business, art/art history, engineering, English composition, languages, psychology, geography, law, nursing, physics, and social work may want to consider the OER directories listed in Supplementary Table S2 and Table 4.

An earlier investigation [21] evaluated 12 of the 118 directories included in these results. Based on characteristics such as the number of OER textbooks in each directory, the breadth of subject coverage, the presence or absence of expert evaluations, and the effectiveness of the directories’ search mechanisms, Perifanou and Economides identified two especially useful directories: the Open Textbook Library [63] and OER Commons [112]. Notably, the Open Textbook Library emerged as the highest-ranked directory in the current study as well. OER Commons is ranked much lower, however, due to its relatively high percentage of non-textbook items, its absence of clearly documented selection criteria, and its inconsistent textbook quality (See Table 2). Aside from the Open Textbook Library, the other directories with favorable rankings in both studies include the B.C. Open Collection [64], LibreTexts Commons [65], OpenStax [80], and eCampusOntario [85].

Further research should incorporate a wider range of variables—those presented here as well as the interface-, popularity-, and use-related variables considered by other authors [21,22,23,24]. More comprehensive, systematic, and in-depth evaluations of selection criteria, evaluation processes, and textbook quality would also be helpful. The assessments presented in this study are data-based but also subjective and somewhat superficial—a situation made necessary by the goal of covering a large number of directories.

A strong emphasis on OER textbooks is also important, since poor discoverability is the single greatest barrier to their adoption and use [11,15,16,17,18]. An instructor looking for a lab activity to demonstrate that carbon dioxide and water result from hydrocarbon combustion, for example, will have no difficulty finding one through a simple Google search. It is much more difficult to find a free online resource that can be used in place of a conventional chemistry textbook—one that is appropriate in terms of scope, rigor, level of presentation, and attention to detail. While there is a tendency to expand the definitions of OER and open educational practices to include an ever-wider range of resources, policies, and activities [9,10], a more focused approach may be helpful to instructors in need of textbooks that are readily accessible to students. After all, every document, website, individual, physical artifact, and process in existence can be regarded as an educational resource—but few of them offer the comprehensiveness, organization, intentionality, academic orientation, and user-centric focus that characterize high-quality textbooks. In building better directories of OER textbooks (rather than OER resources in general), we are simply diverting resources to the need that is greatest.

Finally, authors and publishers are advised to ensure that their OER textbooks appear in the most useful textbook directories, to promote those directories as superior discovery mechanisms for OER textbooks, to offer the same value-added features as conventional textbook publishers, and to market their textbooks as substitutes for particular titles that cater to the same target audiences. Arguments in favor of Open Access principles often overlook the substantial variations in quality between the most and least effective OER resources. As a result, faculty and students exposed to low-quality OERs may end up with a negative opinion of all OERs. Conversely, those who have used the best OERs may be more likely to support OA principles as a result of their experiences with particular high-quality texts.