I recently taught a Massive Online Open Class (MOOC) titled “Computational Investing, Part I” via coursera.org. 53,000 people “enrolled,” which is to say they clicked a “sign up” button. How many finished?
This is a re-blog of a previous post. It seemed timely.
Completion rates are low, but that statistic is misleading
Much of the criticism of MOOCs centers on supposedly low completion rates. And these rates do seem low when compared to completion rates of regular university courses. But the comparison isn’t apples to apples. Let’s dive in by considering what does it mean to start a course.
One of the 53,000 students in my class watches a lecture video.
What does it cost a student to enroll in a course?
The economics are significantly different for a student at a traditional university than for a student starting a MOOC.
At a regular university all of the students starting a course have paid tuition, they have moved to an apartment or dorm near the university, and they’ve set aside time to complete the course. They have changed their lives significantly in preparation for this course. Most MOOC students are simply adding the MOOC to their existing lifestyle. They’re not moving, they’re not paying a significant fee.
Also, at most universities, students may withdraw from a course early in the semester with no penalty. This enables, perhaps encourages, them to sample a few courses before settling on the set they’ll engage in for the semester. Completion rates for university courses do not include those students who enroll in a course, then withdraw early. On the other hand, completion rates for MOOCs may include these students.
Compare the costs above to the cost of enrolling in a course at one of the MOOC providers such as Coursera, Udacity, or EdX. The cost for a MOOC is zero. All a student need do is provide an email address, and click a button labeled “sign me up.”
- Enrolling in a course at a university is expensive monetarily and intellectually.
- Enrolling in a MOOC costs nothing.
What’s the cost of failure or withdrawal?
Failing a course at a university is costly in many ways for a student. Besides the time and funds lost, there’s the cost of that “F” on the transcript. There are no such costs associated with MOOCs
- Failing a course at a university is expensive both monetarily and intellectually.
- Failing a MOOC costs nothing.
What are the implications for completion rates?
There’s not a lot of data yet, but my hypothesis is that students are significantly more likely to enroll in an interesting MOOC than in a comparable course at a university. And importantly they are also substantially more likely to withdraw. This combination of factors points to a low completion rate for MOOCs.
That’s not necessarily a bad thing. More on that later.
Here are some numbers
For the course I recently wrapped up:
- Enrolled (clicked “sign me up”): 53,205
- Watched a video: 53% of those who enrolled
- Took a quiz: 26% of those who enrolled
- Submitted first homework: 12% of those who enrolled
- Completed the course:
- 4.8% of those who enrolled
- 18% of those who took a quiz.
- 39% of those who submitted the first project.
Note that the more investment we see from the students, the higher the completion rate. Overall I suggest that the definition of “enrolled” for a MOOC should probably not include those who simply click “sign me up.” Those who took a quiz are at least a bit invested, those who completed a project even more so.
Commentary: Drawing some conclusions and suggesting future directions
First of all, we need to recognize that completion rates for MOOCs really have a different meaning than those for regular university courses. This is mainly because of the differing level of investment the students make from the start. In other words “skin in the game” matters.
But MOOC completion rates aren’t really low in the context of Internet engagement. A click through rate of 5% for a google ad is considered a strong success. Convincing 5% to engage intellectually for 8 weeks is, I think, a big deal.
If we continue to keep the barrier to entry low, we’ll enable students to taste many many courses, and that may be a good thing for education.
On the other hand if we want to boost completion rates, and perhaps boost engagement in MOOCs we should consider forcing the students to invest. A small enrollment fee might make a big difference. I’d be very interested to see how this would affect engagement, learning and completion rates.
Related Articles
An article by Amy Bruckman about the consequences of costs for courses.
mike stepenaskie
July 24, 2013
I took the last augmented investment class. You mentioned investment, it would keep out those who click enroll but do not participate. I did invest my time, at my hourly rate at my job I invested $3000 of time. A nominal charge would be valid if you get compensated for your effort . I thought you did a great job.
The number of completions compared to the number of enrolled is not important.
Thanks
Tucker Balch
July 24, 2013
Thanks for the comment Mike.
Laura Gibbs
July 24, 2013
I’m curious if you have any insight as to why Coursera seems to be completely uninterested in gathering data from students who don’t finish the courses, but who do start them. Sure, instructors can send out surveys, but what about Coursera itself? When you un-enroll from a course, Coursera asks nothing (I dropped a course just now – and it is still the case that Coursera makes no effort to gather feedback at that crucial moment). It seems to me there are very different reasons why students stop participating in these courses and actively un-enroll… why is Coursera so uninterested in finding out about those reasons?
Tucker Balch
July 25, 2013
Good point. I’ll forward that question to them.
Laura Gibbs
July 25, 2013
Super, I’ve asked that of various people and it never went anywhere (although I enroll in courses periodically just because I am curious about changes to Coursera software interface) – one of these days when I un-enroll, I’ll be happily surprised I’m sure to find a survey waiting. Thanks!
Ken Brantferger
July 30, 2013
I’m surprised this would be a real issue. According to your stats 4.8% of 53,205 completed the course – or 2554 students. For large physical classes of 50 students, you would need to teach this course for 25 years with two sessions (fall and spring) to reach the same number of students – that is what is important.
Personally, I’ve signed up for several classes, only to be sidetracked by work or family. I don’t think my failure to complete should have any bearing on the measure of quality of those courses. Because, as you point out, they cost me nothing – except the knowledge I lost by not completing the course :)
James peters
August 15, 2013
Putting on my analytics hat, would be interesting to map class forum activity and participation against who stuck it out and completed, and predict based on forum usage, the likelihood of completion. bTW, I was a completer..
W. Prins
September 6, 2013
For what it’s worth I completely agree with your take on things and it accurately reflects my own experience/approach. There’s been several Coursera courses that I’ve started but had to abandon due to lack of time and/or being a bit optimistic with what I took on.
This is facilitated by the cost barrier to entry being so low (e.g. free). Does it matter or reflect negatively on the MOOC in question? No IMHO. People doing statistical analysis should bear this difference in mind otherwise incorrect conclusions follow.
Speaking for myself, usually when I’ve had to abandon a course I’ve done so with the intent to resit it on the next iteration, so the fact that I’ve not completed in such a case is not any reflection on the course per se, and entirely a reflection of the personal aspects of decisionmaking surrounding MOOC participation. One might say, it’s more related to personal/individual student “Alpha” than “Beta” ;)
Miljan Martic
September 18, 2013
I am taking the class at the moment and I have to say it is one of the best Coursera courses I have taken in a while. Also, since I am in the process of researching about a suitable CS master’s degree, it has prompted me to think strongly about applying to Georgia Tech next year. So in this way, there are multiple benefits for both students, instructors and the university as well. Anyway, just wanted to say, great job and thanks for the awesome lectures!
Ray Marcontel
September 23, 2013
When I was in college, there were also several courses which I audited. The cost was minimal, and in a few cases where I already knew the professor, it was free. Likewise, with Coursera in general, there are classes where I want to get a good ‘survey’ of the material, but do not have time to invest in completing the projects. For me, it’s still a successful use of time and I got exactly what I needed.
Byron Mehl
September 24, 2013
I just finished previewing the course without enrolling (and thus may not have been counted). I found the course to be useful and interesting and I believe I learned quite a bit. You may wish to find a way to account for those of us who took this third option for experiencing the course.
I chose not to enroll due to a fear of having to commit to improving my Python skills on a tight schedule, but I might consider taking the course for score if it’s offered again.
Paul
September 28, 2013
Many people, like me, just want to watch the video lectures, since they do not have time to do homework. For instance, your course uses Python, and I am ignorant of it and want to keep so (I do not have time to learn Python). However, I do know quite well R, and I can program in R all that is taught in the course. So, my goal is to learn the the main ideas of the course, and I do not need any certificate of completion of the course. (Note that the videos were watched by about half of the enrolled students.)
I personally watch all the videos lectures of several different courses, and I have learned a lot. But I do not do the homework, and I do not want to obtain any certificate — I only want to learn
If there is no additional cost of having 50000 students enrolled instead of 3000, why to bother about that?
Harmeet Singh
October 6, 2013
I have a comment relating to setting a fee to join a course. I think many (including myself) will not even try (in your words “taste”) a course because of that. I joined your “Computational Investing Part 1” course not because I wanted to, but because I was curious. I had also enrolled in the “Calculus One” course by Bart Snapp. The Calculus course I really wanted to do. But, when the time pressure came on me, I dropped out of the Calculus course!
If there was a price to “get-in”, I would pay for the calculus course. I would never even discover this (Computational Investing) field!
Toan Tran
October 17, 2013
$5 sign-up fee. $0 penalty for A, $20 for B, $40 for C, $60 for D, and $80 penalty for F or incomplete. I BET number completed will go through the roof, with much higher quality to boot. If you get an A, you get a useless piece of crap mailed to you (like a bumper sticker or something).
rpsx
October 22, 2013
i actually signed up for your class twice. i made it to the first assignment, and almost through it… twice. and, i have since “dropped out” of the class… twice. i loved every minute engaged with it, though.
i think the problem with coursera’s style of MOOC is the forced schedule. had there been a “learn at your own pace” track (no quiz penalties, class and forum always accessible, post/tweet on completion as reward), i am sure i would have signed up only once, and have completed it by now. in fact, i would imagine that the completion rate for such an track would probably be extremely high for everyone using that option.
personally, i have now decided to give up on coursera’s forced pacing completely, and just “audit” all classes (including yours). if i am rushing through the class, just to get assignments in on time, then i am not learning. if i am not learning, it’s pointless.
i think the data is being read as: because student completes X, it *causes* them to go on to complete the course. i think it is simply student has time to complete X, and so they are likely to also have time to complete the course as well.
Steven
November 8, 2013
You discuss the cost of not completing the class, but there is also a difference in the benefit of completing a MOOC course vs a university course. Presumably the university course leads to a degree which has a direct impact on an individual’s employability and/or compensation. Most employers do not factor in completion of online courses when making hiring or compensation decisions.
There are of course many motivations for enrolling in online courses, so I don’t want to imply that online courses are only useful for employment purposes, but I would feel safe assuming that this is a strong motivation for some.
Tucker Balch
November 9, 2013
I agree with your point, and this is another reason that (non-credit) MOOCs should not be compared with on campus courses. However, there are massive online courses that that will offer credit.
Michal Sourek
January 21, 2014
As having many colleagues from previous courses here,
and
as having Coursera rather seldom, if at all,
respond to questions about the course statistics
— with a bright and excellent exception of prof. Tucker Balch (!)
;o)
let me share
a link to a fresh MIT / Harvard published Quantitative Data about MOOC
>>> http://gigaom.com/2014/01/21/harvard-and-mit-make-a-compelling-case-for-moocs/
Hope you enjoy how soft “information” is embedded inside the Fig. 13
The full paper is >>> http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263
Thanks go to Hubert Dulay, for raising a flag on this subject [+1]
Mark Ravitz
April 12, 2014
I think it’s a mistake to measure the success of a MOOC by counting the students who completed it. Everyone who started the course was curious. Everyone continued the course until the time/energy they were investing outweighed the educational returns.
It’s also a mistake to consider a MOOC in isolation. I never signed up for your course. I’m working my way through it, as well as taking side excursions into downloaded macroeconomics textbooks, Kahn Academy matrix algebra courses, articles in Wikipedia and Investopedia, python courseware …
Having the course free and online is of immeasureable valuable. Think of some kid online in backwoods Brazil. The course is a gift to humanity.
Howard Kaplan
October 18, 2014
I have started a number of online MOOC’s. I also have a college degree in business from Penn State, and a law degree from Widener. I have only completed one course. For all of the other courses I signed up for (ie, clicked a button for), including calculus, computational investing, learning python, learning R, data science, etc… I watched most of the videos, and attempted those homework projects and quizzes that I had any understanding for. My problem in completing a course was when I was unable to complete a homework assignment or project (usually somewhere in the middle of a course), and the rest of the course relied on that homework or project. Sometimes, answers were given, and I was able to review the answer, and move on, most often, no answer other than, take an introductory course was given, and I had no way of completing future assignments that relied upon past assignments. Of course, the courses I am taking, and have taken in the past, are simply things I was interested in learning, and not for any type of job or career advancement. I for one, appreciate that these courses are given online, and for the most part, do not have an external cost for attendance, and intend to keep trying them on subjects I am interested in, but have no expertise for. I most especially appreciate all the work that the instructors put into creating and teaching these very interesting subjects.
For instance, two courses I am taking involve using programming in investing and econometrics. While, I understand the business, economics, and investment principles involved, I have great difficulty with the programming, even though I have taken many programming courses online (including R, Python, Java, Javascript, C, C++), and have programmed in quick basic, and the macro languages in Lotus 123 and Wordperfect. One thing that I noticed that aids my learning (and, I noticed this from an early age when taking junior high and high geometry, trigonometry, and calculus), are solutions to problems that I cannot otherwise solve.
Tucker Balch
October 30, 2014
I think we (MOOC developers) should offer MOOCs that can offer multiple levels of accessibility. For instance, 1) you should be able to learn just by watching the videos 2) You should be able to take quizzes on the lectures, 3) You can add attempting projects, and finally 4) comprehensive grading and personalized feedback.