Helping instructors design courses with a learning science approach at Open Learning Initiative

Open Learning Initiative

The Open Learning Initiative (OLI) is the flagship product under The Simon Initiative. It is a collection of tools for guiding instructors to design courses using an evidence-based approach that boosts student success.

Opportunity overview

An instructor’s course design is a hypothesis for how students might best engage with and absorb content. Instructors have a wealth of knowledge, but packaging that knowledge into a productive course that can effectively communicate it to students can be an overwhelming and stressful process.

Over the summer of 2020, as the Product Management Intern at OLI, I set out to explore instructor use cases and create a tool that can help them design courses using a learning science mindset.

My role

• Uncover instructor pain points through interviews and secondary research

• Propose and prototype a new onboarding tool that enables instructors to adopt learning science practices

The Simon Initiative

Is a learning engineering ecosystem at Carnegie Mellon University helping faculty improve their educational practice, and improve student outcomes. Learn more here.

Learning Science

Learning science is an iterative outcome-based approach to instructional methodology. In this approach, student outcomes are clearly documented ahead of time, and course materials are evaluated based on how well they enabled students to meet those outcomes. By iteratively improving a course design, instructors deliver a progressively more valuable learning experience for students.

Problem

Most instructors are used to lecturing students, but are not used to creating a strategic plan to measure the effectiveness of their teaching. Learning science is an approach to teaching that was developed to help these instructors, focused on measuring student outcomes and iterating upon course materials to improve student learning. However, even those instructors who are already aware of learning science don’t know where to begin incorporating it into their lecturing practice.

Action taken

I reviewed existing course designs on OLI to understand existing best practices for measuring student performance in accordance with learning science methods. I interviewed instructors to understand why they seek out OLI for course authoring, and their current pain points when it comes to adopting learning science principles, and teaching students in general.

Outcome

I created a course template that helps instructors adopt the mindset of a learning scientist. The template provides a scaffolding for instructors to document a teaching hypothesis, set up learning objectives, and establish measurable skills that students will learn. Together, these form the basis for a course design that can be iterated over time, helping students meet the course’s clearly articulated goals more and more effectively.

The process

I interviewed users from OLI’s target user group: instructors that had either authored courses before, or were interested in course authoring in the future. From these interviews, I discovered two main goals that were consistent between both groups: while all instructors wanted to teach more effectively, some instructors wanted to author courses from scratch, while other instructors wanted to start with existing course material and incorporate them into their course design.

Identifying instructor pain points in use cases

All instructors I interviewed had some familiarity with learning science. However, they struggled to actively apply the principles of learning science to the courses they were designing. Many instructors failed to develop a teaching hypothesis; an essential part of documenting an instructor’s initial assumptions about their teaching methodology, and the first step of applying learning science to course design. For many instructors, testing and learning about teaching with a scientific methodology is not an intuitive approach to course design. This challenge was a consistent theme throughout all the pain points I discovered. Instructors articulated the following challenges that they struggled to overcome:

Use case 1: Course design with existing course material

• I don’t have a process for figuring out where I’d want to make edits to existing course content

• There is no organized way to think about where I want to insert student engagements in my course content

Use case 2: Course design without existing course material

• I’m not sure how to start designing an effective course

Common pain points

• I’m not sure how to incorporate learning science principles into my course design.

• It’s difficult to share, collaborate, and store feedback on course design - especially now that my colleagues and I are working from home.

• I’m not sure what iterating my course design means, since it is unclear what “metrics of success” means for my course design.

Design Principles

Based on the pain points instructors articulated during their interviews, I created a set of design principles that would guide further exploration of the solution space.

• The solution should help the instructor step into the mindset of a learning scientist.

• The solution should allow the course design to be easily replicable, so that it might be easy to iterate upon, or to use as a template for another course.

• The solution should align with the way instructors currently think about developing course materials, so that it is simple to use and easy to understand

• The solution should be customizable for instructors teaching different subjects.

• The solution should be quick to implement and lightweight. OLI has limited engineering resources to develop a new solution for the problems uncovered during this project, as the majority of the OLI team is currently focused on launching a new platform.

Identifying the best solution medium

During the interviews, instructors consistently expressed a desire for a tool that would help them adopt a learning science approach to course development, even before launching a authoring project in OLI. Instructors were intimidated by the number of tools available for authoring, and did not know which stage of course design each tool was intended for.

I decided that a primer to help instructors with with content creation would be the best solution. OLI does not currently provide any formal guidance on how to approach tools and features in its authoring platform. Most instructors interested in using OLI for authoring are new to learning science, and don’t know how to begin restructuring their courses to apply its principles.

Since I did not have dedicated engineering resources for this project, I researched different platforms and ranked them by their ability to support instructor needs:

Based on this matrix, I elected to focus on Miro and Trello, as they delivered consistent benefits across all six user needs.

Iterating content templates and gathering feedback from instructors

I created draft versions of a primer template in both Miro and Trello, and asked instructors to think aloud as they independently explored the templates. Instructors were delighted to see learning science principles laid out in a digestible format. One instructor noted:

“I ask students to document their hypothesis in their lab work all the time, not sure why I’m not doing the same thing myself, and treating teaching like a scientific experiment.”

From the first round of user testing, I discovered that the Trello board was more suitable for instructors working with existing content, because it gave them a bird-eye view of their course layout. It was easy for them to figure out where they should slot-in student engagements and data collection opportunities. For instructors authoring from scratch, Miro proved more effective: they needed a medium that had flexibility in tagging, brainstorm space, and non-linear organization.

Based on these results, I created a second iteration of both templates. For Trello, I created color labels for the different kinds of student engagement strategies that instructors could incorporate into their lessons. This made it easy for instructors to quickly see the distribution of different student engagement strategies within a course, and evaluate whether there is enough diversity and frequency in engagement strategies incorporated. For Miro, I created more space for content brainstorming and added tags to help instructors draw links between learning objectives. Learning objectives that relate and build off of one another help students aborb content in a organized manner. User testing subjects found that the templates were a practical framework to apply to any course subject, fulfilling the universality design principle.

The Output: Course design templates

The content template is a starting point for instructors using OLI to create courses that intentionally lay out paths towards learning outcomes. The template also provides structure and guidance for iterating that course over time. As students take a course developed with this template, they are regularly benchmarked on their learning outcomes through activities, assessments, and skill-building. The template provides guidance for instructors on how to use these insights into student performance to improve their course design.

Two templates for two instructor use cases 

Echo BlueSky Template on Miro - designed for instructors who are beginning to design their course from scratch, and need a well-defined structure to guide their authoring process. 

The Echo Lite Template on Trello - designed for instructors who already have course content to work with, and need help planning for content restructuring, students assessments, and activities.