Open Education 2017

In recent student focus groups, Ohio State's Affordable Learning Exchange team discovered that required textbooks were assigned not because the texts were referenced heavily in class, but primarily for access to test banks used to deliver quizzes throughout the semester. These test banks are a significant barrier to open textbook adoption in large enrollment courses at our institution, and the questions included are often of low quality. To address this barrier, we're piloting a process to bring faculty from across several institutions to collaboratively author and review large test banks in three disciplines: Macroeconomics, Biology and Sociology. In this pilot, we are examining the time, tools and resources required to create not only a test bank useful in one course with a specific textbook, but one that can be used and customized by many instructors in an iterative, sustainable process in many courses across institutions. The project began in March 17 and will produce test banks ready for use beginning in fall 17. Questions we will attempt to answer in this session include: How do authors approach development of learning objectives that can be applied across many courses, using differing course materials? How do we integrate a robust peer review process into this process, to create a higher-quality finished product? What process and support framework should be in place for each content area? How many contributors are needed to create a robust test bank, and how do needs change based on discipline? Which part of the workflow can be managed within an authoring tool? What is the best way to deliver question/answer sets for easy remix and reuse? We will share Content Camp project and process documentation and resources and report on our lessons learned and future plans. The audience will be actively engaged in the conversation, with plenty of time for questions and feedback that we'll take back home and incorporate as our Content Camp model grows beyond the pilot.

We'll discuss a little bit of the history of Open Embeddable Assessments and then discuss how it is currently being used by various organization. We'll share a little bit of information about the various Open Source versions of the project for those who want to dig into code. We'll then discuss the future of Open Embeddable Assessments and how we can still achieve the vision of sharing assessments as OER.

There are a few repositories that have been created to provide increased access to open educational resources (OER) such as OpenStax (openstax.org), FlatworldKnowledge (flatworldknowledge.com), or LibreText (chem.libretexts.org). There are a few tools doing similar things in the assessment space, such as MyOpenMath (myopenmath.com), Oppia (oppia.org), H5P (h5p.org), or Open Assessments (openassessments.org). These tools provide platforms to facilitate the creation of resources. However, discoverability, reuse, moderation, and remixing of existing content could be improved in these tools. In addition, these tools do not address the issue of high stakes open assessment. For open assessment items, one of the challenges to overcome is student cheating. If assessments are open, students will have access to the questions in an online format (online quizzes or online homework assignments using open questions) which encourages student cheating.



Our proposed model and solution for open assessments is as follows:

1. There are two communities of openness, a public community and a teacher-only community.

a. The public community is open to everyone. This community contains formative assessment items that can be used anywhere as practice questions.

b. The teacher-only community is tightly restricted to teachers. This community contains summative assessment items that can be used in homework system problems, quizzes, or exams.

2. Interoperability standards are used

a. Quizzes can be imported using the quizzing and testing interoperability (QTI) standard

b. Quizzes can be taken from an LMS using learning tools interoperability (LTI)

c. Data is tracked using the Experience API (xAPI) data format

d. Practice questions can be embedded as an HTML5 object in textbooks, blog posts, webpages, etc.

3. Formative questions, quizzes, homework system sets, and exams can easily be shared, modified, and incorporated into existing courses within either community

We'll discuss a little bit of the history of Open Embeddable Assessments and then discuss how it is currently being used by various organization. We'll share a little bit of information about the various Open Source versions of the project for those who want to dig into code. We'll then discuss the future of Open Embeddable Assessments and how we can still achieve the vision of sharing assessments as OER.

There are a few repositories that have been created to provide increased access to open educational resources (OER) such as OpenStax (openstax.org), FlatworldKnowledge (flatworldknowledge.com), or LibreText (chem.libretexts.org). There are a few tools doing similar things in the assessment space, such as MyOpenMath (myopenmath.com), Oppia (oppia.org), H5P (h5p.org), or Open Assessments (openassessments.org). These tools provide platforms to facilitate the creation of resources. However, discoverability, reuse, moderation, and remixing of existing content could be improved in these tools. In addition, these tools do not address the issue of high stakes open assessment. For open assessment items, one of the challenges to overcome is student cheating. If assessments are open, students will have access to the questions in an online format (online quizzes or online homework assignments using open questions) which encourages student cheating.



Our proposed model and solution for open assessments is as follows:

1. There are two communities of openness, a public community and a teacher-only community.

a. The public community is open to everyone. This community contains formative assessment items that can be used anywhere as practice questions.

b. The teacher-only community is tightly restricted to teachers. This community contains summative assessment items that can be used in homework system problems, quizzes, or exams.

2. Interoperability standards are used

a. Quizzes can be imported using the quizzing and testing interoperability (QTI) standard

b. Quizzes can be taken from an LMS using learning tools interoperability (LTI)

c. Data is tracked using the Experience API (xAPI) data format

d. Practice questions can be embedded as an HTML5 object in textbooks, blog posts, webpages, etc.

3. Formative questions, quizzes, homework system sets, and exams can easily be shared, modified, and incorporated into existing courses within either community

Open Embedded Assessments (OEA) has come a long way from their humble beginnings as a seed of an idea from the Center for Open and Sustainable Learning. From the proof of concept produced in 2013 and presented at OpenEd in 2014 to today, we've built robust, open source software in use by Lumen Learning in the United States and by MIT and partners in India. Thousands of learners have taken hundreds of thousands if not millions of assessments to help understand their learning in English and STEM topics.



This presentation will showcase the authoring and player environments developed by MIT and Atomic Jolt and used in the Connected Learning Initiative in India. These CLIx assessments are used by learners in Grade IX in about 400 government schools across four states in India. Learners engage with English, Math and Science learning modules in which OEA formative assessments are provided as activities embedded in the content modules. Learners interact with these assessments either on stand-alone computers or ones served by a local server in their computer labs. Authoring is accomplished by curriculum teams in India on cloud-based servers. Currently OEA supports a number of common item types (multiple choice, short answer, numerical answer with a tolerance, etc.) and a number of customized user interfaces on other common types (drag and drop, fill in the blank, audio response / file upload, etc.).