Teaching a MOOC: Lessons Learned & Best Balch Practices

I just completed teaching a MOOC on Computational Investing via coursera.org. I did some things right and a lot of things wrong. Here are my lessons learned from the first round. I’m very excited now about the second go at this, and the ability to make the course even better. Some of these items are coursera specific, but many apply more broadly.

Some people will be upset: Be prepared

I’m not talking about the students.

Your course will be closely scrutinized. Some folks at your institution may be critical of the idea. Many outside your institution will be critical as well.

One critique MOOCs are susceptible to is an accusation of “dumbing down” or “oversimplification.” Sebastian Thrun’s statistics course was attacked by others in this way.  He responded appropriately, which is to say his course is aimed at a particular educational goal (namely as preparation for later robotics courses), and also that he appreciated the feedback and he would use it to improve the course.

Many of these attacks arise from a belief that MOOCs are promoted as “identical” to college course content; Or that the course is “just as rigorous” as graduate course CS XXXX at Georgia Tech. I personally never intended my MOOC to serve that role. My goal is to use MOOCs as a way to connect to willing learners on topics I’m very excited about.

That being said, I do believe that we can produce and deliver “rigorous” content via MOOCs, and many are working on that. But not all MOOCs have that purpose.

Set expectations for the students

In your course description and your video introduction be sure to make it very clear who your intended audience is. I would also remind students of this frequently in the early modules of the course.

You probably want to discourage folks who don’t have the tools to succeed (for instance in my course, programming experience). If they take the course they will not be happy, they will complain on the forums.  They will also contact your TA and consume his or her valuable time.

You might offer these folks an alternative: Invite them to watch the course videos without the assignments.  Coursera has a “preview” mode for this.

You also want to avoid having folks in the course who are “over qualified.”  They’ll get bored and complain about the simplicity of the content.  Their comments on the forums and blogs may tend to undermine the value of the course to others.

Get good help and delegate: Focus on the content

Ask your support team to handle as much as possible. This will enable you to focus on creating great content.

I received great support from Instructional Designer Fatimah Wirth, Ph.D., and from TA Sourabh Bajaj. They handled many details that would have overwhelmed me otherwise. (If you’re reading this Fatimah and Sourabh, thanks!)

Why slide decks are important

MOOCs offer a significant number of alternative ways for creating and delivering content. There isn’t a strict need for a powerpoint or keynote deck for every module.

When I teach “in person” I usually use the whiteboard a lot. I walk into the class with a few ideas about what I will cover, and because I know a lot about the topic (usually) I am able to provide an informative lecture.  I found that I can’t do this very easily via MOOC.

Slide decks can help. I leverage slide decks in my MOOC workflow and my larger plan as follows.

First, I rely on the deck as a sort of “script” for each video. I’m always a bit “hopped up” on camera and it is nice to have the slides there to keep me on track.

Next, the deck enables me to easily recreate the video if necessary later. In fact I’m anticipating to redo a lot of my videos in response to student suggestions.  If you don’t keep the slide deck “script” you might forget some of the content. In particular the deck helps you recall how that lecture fits in between two others.

Finally, the students often like to be able to view the slide decks for review.

Creating content will take a lot more time than you think it will

For me about 1 hour for each 15 minutes of video.

When creating content, figure out a workflow and stick to it

If you follow my approach of slide decks as “scripts” for your videos, you’ll be creating a lot of slide decks.  I settled on the following work flow:

1. Don’t worry about drawings or diagrams at first.
2. Create the slide deck, use placeholder slides (just titles) for drawings you will add later.
3. Create the drawings. I use penultimate on an iPad. It’s a lot faster than creating a “serious” diagram using a more formal tool. I feel that in this application the ability to churn out diagrams and revisions quickly outweighs potential concerns about quality. The image at the top of this post was created in this way.
4. Add the drawings in to your slide deck.
5. Create your video using the slide deck as a script.

Keep videos short: 5 to 12 minutes

Keep in mind that your students will be watching your content on mobile devices at lunch or during breaks, perhaps even on the subway. Studies have shown that 5 to 12 minutes is a digestible nugget’s-worth of knowledge.

When planning my videos I aim for about 10 slides. Even then I often find that the content I thought would take only 5 minutes takes 15 minutes.

Here endeth the lesson: keep track of time while taping

If you find yourself running long, just close the lesson and move the rest of that content to the next slide deck / video.  As a rule of thumb, once I hit 8 minutes, I’m looking for the next logical end point.

This enables you to keep the video at the right length and keeps you from having to reshoot it. Just say something like “That’s all we have time for now. Next: We’ll learn more about X and Y.”

If you do that, remember to correct your slide deck accordingly after the video.

Continuity: Avoid mentions of dates, times and content order

If you say something like “in the next module we will learn X” then you can never move the present video to a different order in your course.

Now sometimes you DO want to do this, and it’s OK; just be sure about it. My personal approach is to organize my course as modules of 3 or 4 videos, and to allow myself to talk about content order within a module, but not across modules.

Interact on the forums a lot, and personally

One of the leading criticisms of MOOCs, and a valid one I think, is the lack of face-to-face one-on-one interaction between the educator and the student. The forums offered by MOOC providers are a first answer to this. Certainly the students of the MOOC interact there, but the teacher and the TAs should be there too.

I did this a lot at the beginning of the course, but as my workload increased (due to other non-MOOC responsibilities) I wasn’t able to do this as much.  My impression is that the excitement for the students about the course dimmed a bit towards the end because of this. I can say that the students really value having the lead instructor participate him or herself.  I also suggest to have your TA or TAs spend as much time as possible interacting with students on the forums.

Send email to the students for important announcements

I initially just posted significant announcements on the course announcement page. But many students didn’t know they should check there and were frustrated that they missed information about deadlines and so on.

So my suggestion is to start by posting all announcements by email, and to always email the major announcements.  You can, later, reduce the number of announcements that you also send by email, but I would suggest to warn the students first, e.g., “In the future, homework details will be posted here: http…”

Ideas for interaction

Here’s something I tried that was very popular:

Invite the students to post questions for you on the forums. Then suggest that other students “vote up” the questions they most want the instructor to respond to. Devote a video or two to answering the questions.

These were my most popular videos.

Other possibilities include the use of a google hangout or live webinar

Think carefully about grading, especially peer review

Coursera offers a method for grading assignments in which students grade each other. Each student is supposed to assess 5 other students’ submissions.  It worked OK, but there were a number of downsides:

First, it takes effort on your part to produce a detailed rubric.  In order to have consistent grading across a wide spectrum of students you need very clear cut rules.  Here’s the rubric for the peer reviewed assignment from my course.

Next, it takes time. All of the deadlines have to be stretched out to allow time for grading. It also takes the time of the students, which they may not be happy about. On the other hand, depending on the subject, this may be time well spent.

Finally, some students do not execute the peer review well. This leads to difficulties that may be hard to resolve, especially for a large class.

Grading can be easier for computer science-based courses

If the project is to develop software that can do X, then you can construct a grading method whereby the student is posed several questions that should be run through the program. In the case of my course, which is investing based, an example might go like this: Suppose you invested $100 in IBM on Sept 7, 2006, what would the value be on Sept 7, 2012?

More to come

If more lessons learned occur to me, I’ll post them here. Send me questions if you like.