In This Section:
By the end of this module, you should be able to:
In the previous module, Finding and Adopting OER, you focused on organizing your search and finding relevant OER. This module will focus on elements of evaluating and assessing OER.
Watch the last few minutes of this video, which discusses OER evaluation:
Once you've found an OER you might want to use, the first step in evaluating that OER is asking yourself what you want to do with it. Do you want to adopt and use it as-is? Or do you want to adapt and modify the content to meet your needs? If you found an OER that matched your learning outcomes perfectly, but some modification was required, does the license on that resource allow you to modify? Or, is it licensed in a way that does not allow for modifications or derivatives? You'll learn more about open licenses in Module 4. If modifications are not allowed, you may want to consider another resource. So first, before diving into rubrics, consider the license for the OER and what the permissions allow.
There are plenty of rubrics and evaluation tools available. Your department may already use one for evaluating other course material or textbooks for adoption, and if they do, use that! Apart from considering which of the 5Rs you want to exercise and whether the license on the resources allows for it, evaluating OER should not be any different than evaluating other course material under consideration for adoption.
A common rubric for evaluating OER comes from the Open Textbook Network and establishes 10 criteria to use as a guide:
More evaluation checklists:
Another successful approach used for evaluating OER is to track various aspects of the resource with a course map or curriculum map. You likely have already used or are familiar with course mapping (in fact, this type of mapping is very similar to the exercise you completed in the Module 2 Knowledge Check). A course map can help you decide if the resource matches your learning outcomes. In addition to charting the usual charcteristics, such as learning outcomes and assessment methods, an OER course map adds fields for documenting the resource license, where the resource lives online, and any resource-specific comments. Course maps are especially useful if you are combining multiple OER to deliver course content.
The ACC Learn OER course map template is a good starting place.
OER publishers have worked to ensure the quality of their resources. Many open textbooks are created within rigorous editorial and peer-review guidelines, and many OER repositories allow faculty to review (and see others’ reviews of) the material. There is also a growing body of evidence that demonstrates that OER can be both free of cost and high quality—and more importantly, support positive student learning outcomes.
One of the most recognized open textbook publishers, OpenStax created a library of more than 40 peer-reviewed, professional grade open textbooks for the highest enrollment college courses. These books are kept up to date through a centrally-controlled errata process, and 70% of U.S. higher education institutions use at least one OpenStax textbook.
The Open Textbook Library is a collection of over 1,000 open textbooks. Prospective users can read public reviews of the books written by faculty, which assess the text through a star rating and the 10-point rubric mentioned above. Reviewers are faculty working at institutions and consortia that are members of the Open Education Network (OEN). While not traditional peer review, the review process is controlled and often result in very thorough critical analyses.
BCcampus received government funding in 2012 to make open textbooks available for highly enrolled first and second year courses in post-secondary institutions in Canada’s westernmost province. BCcampus sought reviewers for existing open textbooks, reasoning that peer evaluation would prompt more adoptions by faculty. All reviewed textbooks are given a score. Reviews and the average score are posted along with the textbook for educators thinking of adopting.
Often, in conversations surrounding the evaluation of OER, common questions emerge related to quality. There are many aspects of quality that are subjective, since the right material for the course depends on the student, the professor, and the learning outcomes. This is true for both open and closed materials, and is evidenced in the traditional textbook market with multiple titles offered for the same subject.
As awareness and adoption of OER continue to grow, so does the body of evidence that faculty are as satisfied with OER as with non-OER. In fact, in a 2020 survey of more than 3,200 higher education faculty, respondents who used an OER primary text in their course actually had a higher satisfaction rate (89%) than respondents who used a non-OER text (85%) (Seaman & Seaman, 2020).
Questions about OER quality are actually questions of evaluation. A typical question might be: Is the quality of the OER as good as commercially produced, copyrighted course material? As you find and evaluate OER, challenge yourself to consider HOW quality is defined and measured.
Please read the short blog post On Quality and OER by David Wiley. In it, he states, "For educational materials, the degree to which they support learning is the only meaning of quality we should care about." Do you agree or disagree, and why?
The deadline for completing this module and responding to the discussion question in Teams is Friday, November 3.
Information in this module was adapted from:
"Carrie Gits SPARC Capstone - ACC Learn OER" by Carrie Gits licensed under CC BY 4.0.
OER Mythbusting, created by SPARC under a CC-BY 4.0 license.
SPARC Open Education Primer, created by the SPARC Open Education Leadership Program under a CC-BY 4.0 license.
Seaman, J. E. & Seaman, J. (2020). Digital Texts in the Time of COVID: Educational Resources in U.S. Higher Education, 2020 (p. 44). Retrieved from https://www.bayviewanalytics.com/reports/digitaltextsinthetimeofcovid.pdf.
