CC BY-NC-ND
107
Practices for Student Success:
From Face-to-Face to At-Scale and Back
Deborah Keyek-Franssen
Received in Deborah Keyek-Franssen
July 2018 Ph.D., Germanic Languages and Literatures, Associate Vice President for Digital Education and Engagement, University of Colorado System. Address: 1800 Grant Street, Suite 800 Denver, CO 80203 USA. Email: deblkf@cu.edu
Abstract. U.S. higher education is experiencing a time of shifting landscapes, of new technologies, and of unfamiliar competitors. These and other factors, including decreasing public support for colleges and universities, mean that student success is increasingly paramount as a strategic goal for postsecondary institutions. While institutional-level activities such as increased funding for and emphasis on student advising and predictive analytics are crucial, they are insufficient for postsecondary institutions to realize broad and consistent student success. Instead, institutions can look to practices at the curriculum and course level to further student success. This article examines those learning design and teaching practices that constitute the overlap between a) higher educa-
tion research and trends and b) the lessons learned from at-scale learning experiments. Postsecondary research has shown the effectiveness of practices supported by longitudinal data (high-impact practices), represent a confluence of effective learning design and teaching practices (high-impact teaching practices), and focus attention on lowering the costs of education, thereby making access to postsecondary education at least somewhat more equitable (open-educational resources). An analysis of at-scale learning experiments at the University of Colorado allows the layering of relevant and timely examples of specific MOOC design practices on top of the higher-education research and trends framework, illustrating the ways these two strands of student-success practices mutually reinforce one another. Keywords: student success, learning design, MOOC, high-impact practices, high-impact teaching practices, open-education resources, scholarship on teaching and learning, University of Colorado.
DOI: 10.17323/1814-9545-2018-4-116-138
Higher education is in a turbulent time, one of shifting landscapes, of new technologies, and of unfamiliar competitors. It is also a time in which student success is increasingly paramount as a strategic goal for postsecondary institutions large and small. Over the past few years, attention to student success at the institutional-level may have translated into modest gains in persistence, learning outcomes, and degree completion in the United States [National Student Clearinghouse Research Center 2017]. While institutional-level activities such as increased funding for and emphasis on student advising and predictive analytics are crucial, they are insufficient for postsecondary
institutions to realize broad and consistent student success. Universities and colleges should concomitantly, intentionally, and systemi-cally expand exploration and support for course-design techniques and pedagogies shown to promote student success. We likely know enough already about these practices to have a positive impact on student success, but I argue that the practices have not yet gained sufficient traction on our campuses. Because they are mutually reinforcing, I contend that investigating two strands of research about learning design and teaching activities can provide a stronger foundation and motivation for faculty and administrators wishing to implement widely the course-level practices that lead to student success.
This article examines learning-design and teaching practices that constitute a conceptual overlap between a) higher-education research and b) the lessons learned from at-scale learning experiments. I begin by providing context about the United States postsec-ondary environment and then move to review three higher-education trends: high-impact practices (HIPs), high-impact teaching practices (HITPs), and open-educational resources (OER). I have chosen these three trends because they are supported by longitudinal data (HIPs), represent a confluence of effective learning design and teaching practices (HITPs), and focus attention on lowering the costs of education, thereby making access to postsecondary education at least somewhat more equitable (OER). I then analyze at-scale learning experiments to layer relevant and timely examples of effective practices on top of the higher-education research and trends framework, illustrating the convergence of these two strands of student success practices. Finally, I offer a call to action to encourage the broader recognition and adoption of these mutually reinforcing practices across our institutions, mutually reinforcing because at-scale learning benefits from the design and activities it borrows from traditional (face-to-face or online) design and teaching practices and in turn reinforces and validates those practices with at-scale data.
1. The United U.S. postsecondary education has rarely seen such relentless change States context and tumult as in the past decade. U.S. universities and colleges are subject to declining public support for higher education, with deep splits along urban-rural, gender, and political party lines, even as data continue to show the long-term career and economic benefits of degree completion.1 State-supported institutions in individual states such as Colorado have seen an erosion of financial support from public funds. Additionally, an expected, long-term decline in high school graduates has already begun, leading to the potential for a precarious
1 Peterson J., Rudgers L. (2018) Saddle Up: 7 Trends Coming in 2018. Inside Higher Ed, January, 2. https://www.insidehighered.com/views/2018/01/02/ predictions-higher-education-coming-year-opinion
decrease in enrollments in some areas of the country, for some institutional types, exacerbating the fact that a majority of funding for post-secondary institutions comes from student tuition and fees [Grawe 2017]. Student demographics continue to shift to include more students from historically underserved groups—racial minorities, low-income, and first-generation university students—which puts added pressure on universities and colleges to change to meet those students' needs and evolve campus environments to support both diversity and inclusivity. Simultaneously, institutions are under increasing workforce demands for postsecondary and post-baccalaureate learning opportunities and credentials, especially for working adults. In short, "it's tough out there."2
Within this context, growing numbers of U.S. institutions are adopting strategic goals for student success. Those goals include increasing degree-completion rates, making certain that learning outcomes are met and that students have educational pathways to future employment, and ensuring that learners from historically underserved groups realize both access to postsecondary education and successful completion of degrees. In this article, I refer to these goals collectively as student-success.
I argue that employing mutually reinforcing learning design and teaching practices from different learning modalities can improve institutional ability to meet those goals.
2. Student success research and practices
The United States has a rich body of student success scholarship that stretches back decades and continues to grow in both volume and impact. Scholarship on effective teaching practices, both general and discipline-specific, is also well-established and influential. Most universities and colleges have faculty development or teaching and learning centers that expose faculty to effective teaching practices and course design. Three distinct groups of research-based practices for face-to-face or "traditional" online teaching overlap with at-scale findings: high-impact practices, high-impact teaching practices, and the use of open educational resources.
2.1. High-impact Although most instructors contend that they generally employ prac-practices tices that lead to student persistence, learning, and completion, they have largely anecdotal, and rarely longitudinal data to describe their individual classroom successes. Rigorous research over the past decade, however, has identified a set of specific, well-defined practices that lead to student success when implemented deliberately. These 10 research-based high-impact practices (HIPs) improve student suc-
Peterson J., Rudgers L. (2018) Saddle Up: 7 Trends Coming in 2018. Inside Higher Ed, January, 2. https://www.insidehighered.com/views/2018/01/02/ predictions-higher-education-coming-year-opinion
cess especially for those students from historically underrepresent-ed groups.3 Researcher George D. Kuh first described HIPs after analyzing data from the National Survey of Student Engagement (NSSE) in 2008.4 In 2013, he and Ken O'Donnell noted that these practices, when implemented well, share key elements, such as high-performance expectations, investment of student time and effort over an extended period of time, and frequent, timely, and constructive feedback [Kuh, O'Donnell 2013].
Although continued research on HIPs show that they benefit all students, and especially historically underrepresented students, institutions can and should make improvements in how they are implemented. Jillian Kinzie and George Kuh recently reiterated the need to go beyond just making high-impact practices available to students.5 Rather, they argue, for students individually and collectively to benefit from them, institutions must ensure that HIPs are implemented broadly, equitably, and with fidelity to the characteristics that made their initial instances high-impact in the first place.
Hundreds of universities across the country are working together to integrate HIPs more widely at the campus level, to ensure equitable, high-quality opportunities for all students, and to build an extensive community of faculty, staff, and administrators committed to the expansion of HIPs nationally. HIPs constitute one of the three interventions in the Taking Student Success to Scale (TS3) initiative of the National Association of System Heads (NASH), whose member institutions—public higher education systems like the University of Colorado System—collectively enroll 75% of the undergraduates in the United States.6
3 The Association of American Colleges & Universities provides information about high-impact practices and their benefits to students, and calls for campuses to be more systematic in their integration of HIPs into learning environments. https://www.aacu.org/leap/hips and https://www.aacu.org/ sites/default/files/files/LEAP/HIP tables.pdf.
4 The National Survey of Student Engagement (NSSE) annually surveys first-year and senior year students' at hundreds of four-year colleges and universities about their participation in programs and activities. See http://nsse. indiana.edu/html/about.cfm for more details about the survey and how institutions use its results. See: [Kuh 2008].
5 https://www.insidehighered.com/views/2018/05/01/kuh-and-kinzie-re-spond-essay-questioning-high-impact-practices-opinion
6 See the Nash Website, "Taking Student Success to Scale (TS3)," which reports it is "a degree completion initiative led by a collaborative of higher education systems and campuses. Encompassing over 75 percent of the U.S. undergraduate student body in four-year institutions of great diversity, our member systems and campuses have the scale, influence, and desire to impact college completion in unprecedented ways. Based on the collective wisdom of our member system heads and chief academic officers, as well as research, three interventions have been identified as a starting point for
Many of the high-impact practices evidence themselves at the campus or programmatic level: first-year seminars and experiences, common intellectual experiences, learning communities, coordinated writing intensive courses, collaborative assignments and projects, and internships. I describe here those that overlap with at-scale learning design and teaching practices; examples of how they manifest in at-scale learning at the University of Colorado are provided in a later section.7
1. Undergraduate Research—opportunities for students to actively engage in research with faculty, and the concomitant changes to especially undergraduate courses to better support student understanding of concepts they will encounter in research projects.
2. Diversity/Global Perspectives—courses and programs that help students explore cultures and worldviews different from their own and that may include experiential learning in local communities or study abroad opportunities.
3. Service Learning, Community-Based Learning—experiential opportunities with community partners in which students apply learning in real-world settings by serving local communities.
4. Capstone Projects and Courses—culminating projects that require students to integrate and apply learning in a final research paper or public presentation.
Longitudinal research shows unquestionably that these practices lead to student success. Students who engage in one or more high-impact practices have been shown to persist longer in degree programs, have higher GPAs and higher 6-year completion rates than students who do not. Responding to research findings, postsecondary institutions are taking steps to ensure that high-quality HIPs are widely available, deliberately and systemically implemented, and that historically under-represented groups are encouraged to participate in them.
2.2. Discrete teaching practices
HIPs focus primarily on providing learning experiences integrated at the curricular or institutional level. I now turn to discrete teaching practices at the level of course design and in-classroom pedagogies for either face-to-face, hybrid, or fully online courses. A growing strand of postsecondary scholarship, the scholarship on teaching and learning (SOTL), the continued importance of campus teaching and learning centers, and the emergence of the instructional design and learning experience design professions focus needed attention
this holistic and collective approach. These interventions are: Guided Pathways Using Predictive Analytics, Redesigning the Math Pathway, and High Impact Practices for All Students." http://ts3.nashonline.org/ 7 Definitions are provided by the Association of American Colleges & Universities on their HIPs resource page: https://www.aacu.org/leap/hips.
on these course level practices, especially on those that have been shown to lead to and deepen significant learning.8 SOTL research focuses dually on discipline expertise and effective pedagogy, while instructional and learning experience design combine learning science with user experience design and educational technology tools.
Because the research base is so broad, I highlight one author, L. Dee Fink, whose easily accessible guidelines for course design and corollary high-impact teaching practices are reflective of the SOTL and other pedagogical research of the past decades. In Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses and A Self-Directed Guide to Designing Courses for Significant Learning, Fink recommends a learner-centered course design process with four main steps [Fink 2013; 2005]:
1. Articulating the situational context of the course for learner and instructor;
2. Defining the learning outcomes by describing how students will be different after completing the course;
3. Developing "educative" assessments that are learning experiences in and of themselves and that provide quick, frequent feedback; and
4. Designing active, experiential learning experiences.
There is a convergence even within the postsecondary scholarship on student success: Fink used HIPs research as motivation to translate his design principles into a list of high-impact teaching practices (HITPs). He contends that, over time, HITPs can radically transform postsecondary teaching for the better, moving attention from the institutional level to course and classroom level, and shifting focus from teaching to learning [Fink 2016]. Unlike HIPs, which often require coordinated, campus-level effort, HITPs are especially beneficial because instructors can quickly integrate them into day-to-day learning activities. Building on his principles of course design, Fink's HITPs include incorporating activities that encourage a "growth mindset" on the part of students (as opposed to a "fixed mindset"), providing structured team-based learning opportunities, and engaging students in both service learning and reflection.9
2.3. Open educational resources to increase learning and degree completion
Combining HIPs at the campus and programmatic level with high-impact teaching practices at the course and classroom level can provide a double dose student success practices. The increase in adoption of open-education resources (OER) represents another trend in U.S.
8 For an overview of SOTL, including methodologies and references, see: https://www.stlhe.ca/sotl/what-is-sotl/
9 See https://www.mindsetworks.com/Science/ for the science behind mindsets, as well as practices and case studies.
Figure 1: Percentage increase in U.S. textbook cost since 2006.
200
College text booksl
80
60
College tution and fees
40
20
00
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
postsecondary education practice that may also lead to student success, as well as to equitable student access to learning materials. The William and Flora Hewlett Foundation has defined open educational resources as "teaching, learning and research materials in any mediumdigital or otherwise—that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions."10
Although the OER movement includes many practices, including sharing and co-creation of pedagogies and learning objects, postsecondary and mainstream media most frequently cover the use of OER as textbook replacements. When faculty switch from expensive textbooks to open, sharable versions, especially in large undergraduate courses, the cumulative savings for students is high and plays at least a small a role in lowering the cost of education.
Many faculty in the U.S., and especially at two-year community colleges, are turning to open digital textbooks provided by organizations such as the Open Textbook Network or OpenStax.11 Although the cost of textbooks represents only a fraction of the total cost of attendance for U.S. students, textbook costs rose dramatically between 2006 and 2016 (see Figure 1), nearly doubling in price in this period, rising even more steeply than increases in
10 https://www.hewlett.org/strategy/open-educational-resources/
11 Both Open Textbook Network https://open.umn.edu/opentextbooks/ and OpenStax https://openstax.org/subjects offer free, peer-reviewed textbooks and work with both individual faculty and whole institutions interested in adopting open textbooks. A group from the California State University System created MERLOT (https://www.merlot.org/merlot/index.htm) as a curated repository for open online learning and support materials.
tuition.12 Additionally, two recent studies show that between 66% and 85% of students delay purchasing a textbook or forego purchasing one altogether. One study found that fully 91% of students who did not purchase a textbook cited cost as the reason and that half of those asserted that their learning and grades suffered as a result. 13
Increased adoption of OER can counter the disadvantages of high textbook prices and the propensity of students not to purchase them. In addition to providing financial benefit to students, OER allows all students to have access to learning materials on the first day of classes. OER adoption as textbook replacement, therefore, has the potential to ensure equitable learning opportunities for all students. Nascent research on OER adoption indicates that students in courses with OER enroll in more credit hours and enjoy similar or even improved learning outcomes than in courses that do not use OER, thereby providing the potential to lower time-to-degree and raising completion rates [Hilton 2016].M Because of these benefits, states throughout the country like Colorado are investing heavily in OER initiatives, in part by providing grant funding to faculty, promoting the use of OER through faculty and staff professional development opportunities, and evaluating the impact of OER adoption.15
3. Converging In short, HIPs, high-impact teaching practices, and OER together student success point to a set of common practices that can lead to student success practices and that overlap with at-scale learning design principles, including:
1. Employing learner-centric design that focuses scrupulously on learner outcomes;
2. Designing educative learning assessments that are frequent, provide quick feedback, and offer a capstone or reflective experience;
3. Providing significant experiential learning that includes research, service or community assignments, and interaction with and between students from different backgrounds or countries; and
4. Incorporating open resources, including textbooks, learning materials, or even courses, to ensure equitable access to learning opportunities.
12 https://www.bls.gov/opub/ted/2016/college-tuition-and-fees-increase-63-percent-since-ianuarv-2006.htm
13 https://campustechnology.com/articles/2016/08/24/report-stu-dents-shun-new-textbooks-to-reduce-education-expenses.aspx and https://www.insidehighered.com/quicktakes/2017/09/20/study-high-text-book-prices-lead-poor-grades
14 See also the study of Virginia State University's School of Business concluded that higher grades were correlated with courses that used OER: [Feldstein et al.].
15 https://medium.com/@CoHigherEd/the-brave-free-world-of-open-educa-tional-resources-16446868791b
4. At-scale While the research base for student success is relatively well-estab-learning design lished, scholarship on at-scale learning is necessarily in its infancy.
Much of that research analyzes learner demographics, participation, and completion of courses. In general, critics of at-scale learning correctly note that completion rates are very low for open, at-scale learning, as low as 4% in one study, although completion rates rise when learners pay for enhanced learning opportunities and certificates [Chuang, Ho 2016]. Demographic research indicates that major platforms such as EdX and Coursera might not be as egalitarian as initially hoped: they tend to reach learners who already have postsecondary degrees and are working professionals from developed countries. One study, however, showed tangible career and economic benefits to learners who completed at-scale courses, noting that learners with lower levels of socioeconomic status and education in developing countries were "significantly more likely to report tangible career benefits."16
Because research on at-scale learning is relatively new, there exists little that evaluates the impact of design principles that might lead to student success (again: persistence, learning, and completions) both at-scale and, if transferred to them, for more traditional modalities such as face-to-face, hybrid, or "traditional" online. For that reason, sources different than traditional research studies reveal the lessons learned about at-scale learning detailed below: direct experience guiding faculty to design effective massive open online courses (MOOCs) and specializations; effective practices developed by Coursera through data analysis of existing courses and formalized in design workshops, documentation, and practices; and learner feedback to University of Colorado faculty and staff responsible for the continued success and improvement of courses.
The University of Colorado System (CU) comprises four separate universities: the University of Colorado Boulder, the University of Colorado Anschutz Medical Campus, the University of Colorado Colorado Springs (UCCS), and the University of Colorado Denver. The CU System is a public university that is provided some financial support by the state of Colorado and is funded largely through tuition and fees.
The University of Colorado has been a Coursera partner since 2013 and has launched nearly 100 MOOCs and 18 specializations (a series of four-to-six short, roughly one-month courses) with over two million enrollments combined.17 Coursera's evolving business model means that this content is more accurately described as at-scale learning op-
16 Zhenghao C. et al. (2015) Who's Benefitting From MOOCs, and Why. Harvard Business Review, September, 22. https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why
17 See https://www.cu.edu/mooc for current data on CU's Coursera content.
5. The University of Colorado context
portunities: only videos and discussion forums remain open, while assessments are reserved for learners who pay a fee for a course or specialization certificate.
Faculty from all four campuses may opt to teach at-scale on the Coursera platform for a variety of reasons. Some experiment with new pedagogical models or engage in SOTL research. Others are evangelists for their disciplines. Still others see the potential to recruit students into their traditional programs or to contribute to workforce development needs. Whatever the motivation for commencing, most CU instructors report changing their practices in their face-to-face or online teaching once they have undergone the intense experience of designing for at-scale learning.
6. Lessons learned As the Coursera point-of-content for three of the CU campuses, I have from at-scale had the opportunity to support faculty in their course and specializa-design tion design, write and evaluate proposals for at-scale courses and specializations, participate in several Coursera design workshops, speak formally and informally with faculty about their experiences, and review beta testing and course data with an eye toward continuous improvement. My observations of course development across many fields, combined with opportunities to reflect on those observations in the context of teaching and learning trends in postsecondary education, suggest a set of four at-scale design and teaching practices. These practices are especially in evidence in those courses that attract the greatest number of enrollments, completions, and payments and/or the highest ratings—or most passionate learners.
7. Unwavering attention on the learner and learning outcomes
Instructors developing content for the Coursera platform begin by considering the learner—who they are, how they might discover and enroll in a course, and what their life experiences might be. For many faculty, teaching on the Coursera platform is their first exposure to significant numbers of non-U.S. learners and, for some, their first time designing learning experiences for working professionals, both of which require a new orientation toward structure and presentation of content. Early in the design process, faculty are encouraged to create a few personas about hypothetical learners and detail potential background stories, behaviors, life situations, attitudes, goals, and skills. Foregrounding these personas through design, launch, and continuous improvement helps instructors maintain a learner-centric approach.
Throughout the design process, faculty consider how a student changes during and because of the course—in skills, beliefs, knowledge—and how assessments can provide evidence of that learning. Because learners on the Coursera platform are largely working professionals, faculty creating many of CU's courses and specializations design for a learner seeking tangible benefits from a course: a new job,
a promotion, a transition to a new field, skills to improve work performance. They consider how course completion may align with professional credentials, such as continuing medication education credits.
Fink emphasizes the importance of articulating the situational context of learners, especially with regard to content and encouraging in students a growth mindset [Fink 2005]. Providing context, continuity within and between courses, and coaching are critical practices for student success.
These practices in face-to-face classrooms are often implicit or intuitive: faculty sense when a student might not understand and might instinctively let students know when a difficult section of the course is approaching. Effective instructors provide context for their students and coach them through the course, often without even having to think about doing so. These practices are more difficult to integrate into at-scale learning because the implicit must be made explicit. Context, continuity, and coaching must be woven into videos and assignments. All faculty designing at-scale courses are encouraged to frequently ground learning in the larger context of the learners' professional goals or the discipline itself, and to provide explicitly continuity from one module to the next. Most videos begin with a few sentences about what learners will be able to do at the end of the module and end with a few words about the next video. The latter practice in particular seems to increase persistence from one video to the next. To provide context to his learners, Professor Tim Chamillard (UCCS) gives a graphic representation of the flow of the courses in his "C# Programming for Unity Game Development" specialization, allowing the students to see in graphical form (a sine wave) when the workload will be heavier and lighter. He also encourages them in his videos, explicitly acknowledging difficult concepts or weeks, while reassuring learners that they will succeed and have an easier following week.
8. Educative The Coursera platform offers a significant benefit to faculty and learn-assessments that ers: automatically- and peer-graded assessments in a mastery learn-provide frequent, ing environment. These two features allow the large enrollments in quick feedback courses without an overwhelming amount of work or monitoring by faculty. Automatic grading allows frequent formative and summative assessments—from weekly quizzes, to final texts, even to a one-question, in-video knowledge check—without the burden of faculty grading. When both correct and incorrect responses are annotated, even a multiple choice quiz becomes a learning experience in and of itself, especially when taken more than once. Similarly, peer-graded assessments with detailed rubrics promote reflection on learning and offer an opportunity to receive rich feedback about writing or other projects.
Several University of Colorado faculty design intentionally educative assessments and programming projects for their learners. In particular, Dr. Jay Lemery (Anschutz Medical Campus) incorporates
a case study as the final assessment for his "Global Health Respond-ers" course. Learners write memos as a global health responder assisting in a refugee crisis planning. Similarly, Greg Williams (UCCS) uses a case study project at the end of his "Detecting and Mitigating Cyber Threats and Attacks" course to propose detection and mitigation strategies that could have been used by a company that suffered a data breach. The final assessment of his "Proactive Computer Security" course allows students to test their cyber security detection and mitigation skills in a safe computing environment.
9. Experiential While at-scale courses feature rich videos and frequent, automati-learning cally-graded assessments, they run the risk of being little more than television programs with quizzes. To counter this risk, many at-scale courses incorporate learning opportunities that align with the HIPs listed above. University of Colorado faculty have incorporated research opportunities, global learning, community-based learning, and capstone projects in their Coursera courses.
Research Opportunities Roger Martinez (UCCS) has created a "Deciphering Secrets" series that empowers learners to conduct primary research as co-creators of new knowledge. After providing learners the opportunity to explore the historical, religious, and social contexts of walled cities in medieval Spain, Professor Martinez teaches paleography, the deciphering of medieval manuscripts, and then offers the opportunity for students to transcribe and translate newly digitized manuscripts from the archives of these walled cities. Professor Martinez estimates that the learners the initial offering of his first at-scale course completed in 6 weeks the work it would have taken a researcher 10 years to finish. The opportunity to work with primary documents and to contribute to the larger body of knowledge has proved to be an engaging experience for learners that encourages both persistence and passion.
Diversity/Global Learning All learners on the Coursera platform are necessarily exposed to worldviews different from their own: learners come from all around the globe, and bring with them diverse political, religious, cultural, and economic experiences and views. The challenge for faculty is to be aware of any implicit ethnocentric frames inherent in their classes and to be prepared to address any cultural misunderstandings. Setting expectations for behaviors in the discussion forums is critical, as is the willingness for faculty to intervene in the case of altercations. Many CU faculty teaching on the Coursera platform recruit "Community TAs" from around the globe to assist with monitoring discussions, with an eye toward recognizing and addressing cultural misunderstandings.
Service Learning/Community-Based Learning While traditional service learning in the HIPs literature has a component of giving back to a student's community through deliberate, structured service, community-based learning also can be part of at-scale courses. In an upcoming palliative care specialization created by a team on the Anschutz Medical Campus, learners apply what they have learned in their own communities. Using a "Nature of Suffering Evaluation Form," learners interview a person with a chronic or fatal illness, practicing the very skills they will need as professionals in the palliative care field. Reflecting on their experiences in applying new knowledge in real-world settings both promotes deep learning and prepares them for professional work.
Capstone Projects In the HIPs literature, capstone projects are a culminating experience at the end of a full degree. At-scale learning is modularized, so capstone projects on the Coursera platform are featured at the end of specializations, which take a period of time roughly equivalent to a single semester to complete. The "Data Warehousing for Business Intelligence" specialization created by Professors Mike Mannino and Jahan Karmini (Denver) provides a culminating experience that allows learners to integrate and apply what they learned in the specialization. Using a case study approach and building on the prior courses, learners design and build a data warehouse, integrate data, and write analytical queries as the basis for data visualization and dashboard design.
10. Open educational resources at-scale
While Coursera and other MOOC providers' courses could be considered free educational resources, in 2012 and for a few years after, they stopped being truly open. That is, although the access was free, there were and are restrictions on use, adaptation and redistribution by others. Additionally, since some of the most important aspects of the courses—assessments, feedback, and credentials—are behind a paywall, at-scale courses on the Coursera platform have moved even further away from being open.
Although a Coursera paywall exists, learners are not considered regular university students. Faculty are therefore discouraged from claiming fair use exemptions for any copyrighted materials they wish to post in their at-scale courses. This provides motivation for faculty to turn to open educational resources instead and many learn about OER options for the first time through working with librarians to choose materials for their at-scale courses.
11. Convergent, mutually reinforcing practices for student success
Lessons learned through at-scale course design and research-based teaching overlap conceptually with practices gleaned from student success and SOTL scholarship. In fact, a strong research base com-
Figure 2: Convergent Practices for Student Success.
^Success,,
Practices
Learner-centric design, with attention to outcomes, context cotinuity, and coaching Frequent, quick feedback through educative assessments
Experiental learning opportunities
Use of open educational
%sLear„M^
U.S. Context
Declining public support Increased need for postsecondary credentials Need to increase completition rates
Increasing student body diversity
bined with analysis of learner behavior on the platform constitute the foundation for many of Coursera's recommendations for at-scale learning design:
1. Learner-centric design, with attention to outcomes, context, continuity, and coaching;
2. Frequent, quick feedback through educative assessments;
3. Experiential learning opportunities; and
4. Use of open educational resources.
Figure 2 encapsulates the idea of converging, mutually reinforcing sources for student success practices. The U. S. postsecondary context provides the urgency for ensuring that these practices are adopted broadly.
12. A modest call to action: from conceptual overlap to mutually reinforcing
One of the least touted benefits of at-scale learning is the ability to know immediately and at full volume when a part of a course is unsuccessful. Discussion forum posts critical of course content, though representing a very small percentage of learner voices, nonetheless motivate faculty to make changes in the course. Negative comments from even a very small percentage of a very large number of learners still represent more complaints than a faculty member might have received in an entire professional career.
More importantly, data analysis on the Coursera platform can help pinpoint when learners drop a course, help understand which modules might have mismatched assessments and content, which videos or assignments result in greater engagement. CU faculty teaching on
Coursera have been eager to make adjustments to their courses in response to learner feedback and data analysis. Even more importantly still, most have reported changing their face-to-face and "traditional" online practices to better align them with known effective practices.
For all of our insights into student success, we still do not know exactly how individual people learn. We make very good guesses based on research from any number of disciplines, including biology, neuroscience, and education. We use longitudinal data about practices to make solid recommendations about which seem to have the best chance of leading to improved learning outcomes and degree completion, and which seem to be especially beneficial for historically un-derrepresented groups. With the expansion of at-scale learning, we have an unprecedented opportunity to use truly big data to test those learning practices for effectiveness at-scale. Although there is clear support for at-scale learning design practices in the student success and SOTL literature and vice versa, the relationship between the two could be strengthened to the benefit of institutional student success goals. Deliberately incorporating HIPs, high-impact teaching practices, and OER into at-scale learning and then testing their effectiveness for persistence and completion, even at the micro-level of a learning module, would provide data that can lead to increased financial, administrative, and even faculty support for continuous improvement of course design in all modalities and thereby to increased student success.
References Chuang I., Ho A. (2016) HarvardXand MITx: Four Years of Open Online Courses — Fall2012-Summer2016. Available at: https://ssrn.com/abstract=2889436 (accessed 12 October 2018). Feldstein A., Martin M., Hudson A., Warren K., Hilton III J., Wiley D. (2012) Open Textbooks and Increased Student Access and Outcomes. European Journal of Open, Distance and E-Learning, no 2. Available at: http://www.eurodl. org/?p=current&sp=full&article=533 (accessed 12 October 2018). Fink L. D. (2013) Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. San Francisco, CA: John Wiley & Sons.
Fink L. D. (2005) A Self-Directed Guide to Designing Courses for Significant Learning. Available at: https://www.deefinkandassociates.com/Guideto-CourseDesignAug05.pdf (accessed 12 October 2018). Fink L. D. (2016) Five High Impact Teaching Practices. Collected Essays on Learning and Teaching (CELT), vol. IX. Available at: https://files.eric.ed.gov/full-text/EJ1104478.pdf (accessed 12 October 2018). Grawe N. (2017) Demographics and the Demand for Higher Education. Baltimore
ML: Johns Hopkins University Press. Hilton J. (2016) Open Educational Resources and College Textbook Choices: A Review of Research on Efficacy and Perceptions. Educational Technology Research and Development, vol. 64, no 4, pp. 573-590. Kinzie J., Kuh G. (2017) What Really Makes a 'High-Impact' Practice High Impact? Inside Higher Ed, May, 1. Available at: https://www.insidehighered.com/ views/2018/05/01/kuh-and-kinzie-respond-essay-questioning-high-im-pact-practices-opinion (accessed 12 October 2018).
Kuh G. (2008) High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities.
Kuh G., O'Donnell K. (2013) Ensuring Quality & Taking High-Impact Practices to Scale. Washington, DC: Association of American Colleges and Universities.
National Student Clearinghouse Research Center (2017) Completing College: A National View of Student Completion Rates—Fall 2011 Cohort. Available at: https://nscresearchcenter.org/wp-content/uploads/SignatureReport14 Fi-nal.pdf (accessed 12 October 2018).
