Shallow Thoughts : : education

The Stanford online Machine Learning class

In case you haven't been following it, Stanford's computer science department began a grand experiment in online learning early this month: free, upper division college courses, given online and open to the whole world. There are three classes offered: Artificial Intelligence, Machine Learning and Introduction to Databases.

They've sparked an incredible response: exact numbers don't seem to be available, but rumor is that AI had about 130,000 enrolees, while ML had about 70,000. (Nobody seems to have published numbers for DB.) Update, a day later: @seemsArtless tweets that ML currently has 87,000 registered users.

Why so much interest? Surely there are lots of places to get free information (like wikipedia) and even course lectures (like MIT). And there are plenty of places to take classes for relatively low cost, like local junior colleges or ed2go.

What's different about the Stanford classes is that they cover advanced material, in far more depth than you'd find at a junior college or typical online site. They offer graded homework so you can see how you're doing, and there are other students taking the class at the same time, so if you get stuck, there are all sorts of discussion groups you can turn to. It's one thing to read a textbook or watch a video by yourself; I find a class much more helpful, and judging by the response to the Stanford classes, I'm not alone in that.

I agonized over whether to take AI or Machine Learning. They both sounded so interesting! Since I couldn't decide, I initially signed up for both, figuring I'd drop one if the load was too great. By the end of the second week, I'd settled on Machine Learning. I was starting to dread the AI class flash quizzes -- which didn't always work right, but made it hard to proceed until you'd answered the question right even if you couldn't see the question -- and to feel frustrated about the lectures, which clearly were meant as a jumping off point for students to go do their own outside reading.

On the other hand, I was really enjoying the Machine Learning lectures, and looking forward to each new one. And the real kicker: Machine Learning includes programming assignments, so students can implement the algorithms Professor Ng talks about in the lectures.

What's great about Machine Learning

Andrew Ng's video lectures are wonderfully clear, well paced and full of interesting content.

He uses a lot of graphs to help students visualize what's going on geometrically, rather than just relying on the equations. (Better yet, in the programming exercises he shows us how to create those graphs for ourselves.)

And he's great about flagging certain portions as possibly review (you can skip this lecture if you already know linear algebra) or advanced (this is some extra background for people who know calculus, but you can skip it and still do fine in the course).

The technology is simpler than that used in the AI course. If you have a slow net connection or travel a lot, you can download the lectures as mp4 files and watch them offline. You can download lecture slides as a PDF or PPT. Review questions (graded) are handled with simple HTML forms. All very simple, well-tested technology, and it works great. I've had no problems accessing the servers, submitting homework or anything else -- very impressive!

But the heart of the course is the programming exercises. ML is taught in GNU octave, a framework and language for numerical computing and matrix operations. Students aren't absolutely required to use octave, but it's highly recommended: Professor Ng says he's found that students learn much faster that way. Sounds good to me, and octave looks like a useful skill, well worth acquiring. I'm having fun learning it.

The programming exercises come with a lot of scaffold code plus a few files with "Your code goes here". The actual amount of coding isn't large. But I'm finding that it does the job: it forces me to make sure I understand the matrix operations discussed in the lectures. And at the end, you come out with something that's actually useful! From the first few weeks, I have linear and logistic regression code that I could use to analyze and visualize all sorts of datasets. Now, at the end of week 4, we're halfway through writing a neural network to recognize handwritten numerals from image data. How cool is that?

Suggestions for improvement

The class is a huge success. Who would have thought that you could teach something this advanced on such a huge scale, so effectively?

I have only a couple of small suggestions -- ways the class could be even better next time.

• An errata page. In week 3, there was an error in the lecture and notes, a - instead of a +, that made one part of programming ex. 2 quite a bit trickier than it would otherwise have been. If I hadn't noticed that the slides used + in some places and - in others, I might never have gotten that part of the assignment working. Lots of other people found that too, and there were discussions in the Q&A forum ... but you wouldn't find it without coming up with clever search terms.
• The Q&A forum would be so much more useful if it was organized by topic, and/or by week. There are some great discussions there, but the only way of getting to them is by searching for the right terms. There's no way to browse discussions, see how people are doing on assignment 3, or look for errata and similar warnings. It would help make the class more of a community, more like a real in-person class.

Hope for future expansion

I mentioned my suggestions because I fervently hope there is a "next time". These classes are a great service, and I hope the huge response isn't putting too much burden on the instructors.

"Common wisdom" among providers of online classes seems to be that there's no demand outside of enrolled university students for hard courses, courses with prerequisites, and especially courses that involve (shudder) math. Just look at the offerings from any online courseware or adult ed program -- they're long on art appreciation and "Introduction to MS Word", short on physics and econometrics. Even the for-pay online degree mills concentrate on humanities and business, not technical subjects.

Stanford's experiment has proven that "common wisdom" is wrong -- that tens of thousands of students will jump at the chance to take highly technical, mathematical courses. I'd love to see the model expanded to other subjects, such as statistics, economics, physics, geology and climate science.

And, yes, there is money to be made here. If this many people will take a free class, wouldn't quite a few of them be willing to pay? Most couldn't afford $1000 like UC Extension classes -- but how about $100, comparable to other online education classes? Would people pay more if you offered college credit?

Online education providers, take note! There's a large, underserved market for scientific and technical classes out here in the long tail.

Teaching programming with "program a person"

A few weeks ago, at the annual GetSET engineering summer camp for high school girls, I taught my usual one-day workshop on beginning programming in Javascript.

The big question every year is always how to make the class more interactive. The girls who come to GetSET are great -- smart and motivated -- but after six hours of lectures and working through exercises, anyone, of any age, is going to glaze over. Especially when it's their first introduction to programming and they only have a day to learn it. People learn better when they're asking questions, thinking and solving problems, not just listening or following instructions.

For years I've heard vague references to "programming a person" as an exercise for teaching the basic idea of programming. The idea is to get the students to come up with step-by-step instructions for someone to do something -- say, walk across the room and pick up a water bottle -- so they realize how specific you have to be. It also solves another problem: giving everyone a break from sitting still and focusing on a computer screen.

But how do you really do it? What kind of problems work best in practice? How much time should you allow? If you have a volunteer carrying out the instructions, how do you keep them from skipping steps? Surprisingly, I couldn't find anything written up to help an inexperienced would-be teacher of programming.

What I needed was a chance to try out some ideas, or watch someone with more of a clue on this sort of teaching. This year, I found opportunities for both.

First try: Toastmasters

One of the reasons I love Toastmasters, especially with a small and friendly club like Coherent Communicators, is that it offers a safe place to try new presentation techniques and get good feedback about what does and doesn't work. So I made my first try at a Toastmasters meeting a few weeks before the GetSET workshop.

I allowed 15-20 minutes for the exercise. I explained to the audience that I wanted them to get me to turn left, walk over to the easel at the side of the room, touch it, turn around, walk back to the lectern, pick up the gavel and pound it on the lectern. I would solicit a command from them, write it on the whiteboard, then carry out the command and ask for the next command.

The day's audience was a fairly even mix of techies and non. I had wondered whether the audience would be widely mixed in how specific their instructions were, but they were fairly uniform -- mostly along the lines of "Turn 90 degrees left." "Take 5 steps." "Take 2 more steps". Of course, there were a few joking suggestions from the techies, like "send an electrical impulse from your brain to your left quadriceps", that you wouldn't expect with a high school group, but mostly everyone was on the same page.

When I got near the easel, we hit "Raise your right arm". (Oops, not close enough yet.) "Um ... lean forward about a foot?" A good illustration of being specific ... just the sort of thing I was hoping for.

They got me back to the lectern, got me to pick up the gavel (I was letting them skip a few steps by this point) ... and improvised a little, getting me to knock my head rather than the lectern. That was fun, and got some laughs ... it worked well.

I had hoped to do a second run where I guided them into understanding a while loop ("while (not yet to the easel), take another step"). But seeing a yellow light from the timer, I opted for a quick explanation of how a loop would work rather than guiding the audience into it. I found out later that the timer had hit the wrong button and only given me 8 minutes rather than my requested 15-20 ... so 20 minutes actually would have been plenty of time to cover loops as well as basic instructions. Disappointing ... but I was surprised we'd gotten so much done in so little time.

Lessons learned:

• Draft a volunteer to write the instructions on the board. It was distracting and time-wasting to run back from the side of the room to write each new instruction.
• You can teach the basic concepts in less than ten minutes.

Try 2: "Program a blind robot"

For the real workshop, I had help in the form of Esther Heller, an experienced girl scout leader as well as many year GetSET veteran. Esther had done exercises like this before and was willing to take the lead; I was looking forward to learning from her. We had discussed two different variants, and decided to try both of them at different times during the day.

For the first variant, we waited until mid-morning when the class was bogging down a bit and looked like they needed a break. Esther called for two volunteers: one programmer and one robot. The girl playing the robot was blindfolded with a bandanna and escorted to the door of the room, while Esther whispered the task to the other girl. The task was something like walking over to a water bottle, picking it up, walking over to another girl and handing it to her -- though the rest of us didn't know that until it was completed.

The instructions suggested by the girls were quite similar to the ones I'd heard in Toastmasters. There was lots of "Take 5 steps" ... "take two more steps", guessing at how many steps it would take to get from one place to another. No one came up with anything like a loop or conditional. I'd wondered if anyone would try remote control -- "walk" then wait until the right moment to yell "STOP!" -- but no one did.

The blindfolding worked really well. I'd worried that with a volunteer chosen to be the robot, she might skip steps she hadn't been given. But if the "robot" is blindfolded and doesn't know the task, she can't skip steps; she can only do what she's programmed to. The only problem was that a blindfolded person told to walk straight ahead does not necessarily hold to a straight line, much to the consternation of the girl playing the programmer.

There was a lot of "turn right" ... "no, not that much, turn back left again" ... "now turn JUST A LITTLE to the right" that helped stress the need for specificity -- exactly what we were after. I had wondered beforehand whether anyone would ever suggest anything like "turn right by 30 degrees", but no one, either in Toastmasters or GetSET, ever did.

The exercise was successful and everybody seemed to have fun, so it broke up the morning well. We didn't get to loops or conditionals, though. I didn't record how long we spent, but it was probably in the neighborhood of 20 minutes.

Lessons:

• Blindfolding and choosing a volunteer definitely helps this exercise: it solves the problem of a volunteer who might skip steps.
• I wished the whole room knew what the task was ... but I'm not sure how to accomplish that. Either you have to escort the "robot" far enough away that she can't hear you explain it, or write it on the board after she's blindfolded. Extra time either way.

Try 3, in groups: "The muffin is ready"

At the end of the day, we tried Esther's favorite variant. You're watching TV, and you want to go to the kitchen, get an English muffin, toast it, put butter/jam/peanut butter/whatever on it, take it back to your seat and eat it. What are the steps?

Esther divided the girls into groups of 4-5 and passed out post-its on which to write the steps. There was some inertia getting started ... it was late in the day and everybody was tired. (That's not unique to this exercise -- it's always a challenge to come up with something that will hold the girls' interest for the last hour. It's a long day for everyone.)

Eventually they got rolling and got into it -- I saw some very long stacks of post-its from various groups. With ten minutes left to go in the session, Esther picked two volunteers from one group: one to read the instructions, one to execute them. She pointed out places where they skipped steps -- "Hey, wait, how can she get the muffins out of the cupboard without opening the cupboard first?" After a minute or two, Esther called on a new pair from a different group to continue where the first pair had left off.

As she worked through all the groups, you could see each group becoming more cognizant of steps they had skipped, and improvising them on the spot. Despite the end-of-day crankiness, you could see they were learning from the exercise.

Lessons:

• Splitting into groups allows for more discussion among the girls, and comparing various groups' answers is fun.
• Splitting into groups takes a lot of time, and you have to monitor to make sure all the groups are actually working on the problem and not just chatting.

So which is better? The muffin exercise was definitely more time consuming than the previous "robot" exercise, due to overhead of splitting into groups and bringing up volunteers from each group. On the other hand, I could see there was benefit in having them work in small groups, and in the touch of competition in comparing their group's answers with the ones from other groups.

It was hard to compare the two exercises directly to see which one worked better, because of end-of-day crankiness. But they both worked well -- I'm going to keep using some variant of this in future workshops, ideally with loops and conditionals added. Thanks, Esther, for your expertise ... and to the students and the rest of the volunteers for making it a successful class!

Beginning Python: Sorting lists of objects

The Beginning Python class has pretty much died down -- although there are still a couple of interested students posting really great homework solutions, I think most people have fallen behind, and it's time to wrap up the course.

So today, I didn't post a formal lesson. But I did have something to share about how I used Python's object-oriented capabilities to solve a problem I had copying new podcast files onto my MP3 player. I used Python's built-in list sort() function, along with the easy way it lets me define operators like < and > for any object I define.

You can read all about it in my post to the Courses list describing how I sorted my list of podcast objects. Or just go straight to the final program, pods.

Beginning Python, Lesson 9: More extras

Lesson 9 in my online Python course is up: Lesson 9: Extras (requested topics), including string operations, web development and GUI toolkits.

The web development and GUI toolkits are topics which were requested by students, while the string ops are things that just seemed too useful not to include.

Beginning Python, Lesson 8: Extras

Lesson 8 in my online Python course is up: Lesson 8: Extras, including exception handling, optional arguments, and running system commands. A motley collection of fun and useful topics that didn't quite fit anywhere in the earlier formal lessons, but you'll find a lot of use for them in writing real-world Python scripts. In the homework, I have some examples of some of my scripts using these techniques; I'm sure the students will have lots of interesting problems of their own.

Beginning Python, Lesson 7: Object-oriented programming

Lesson 7 in my online Python course is up: Lesson 7: Object-oriented programming.

This is the last formal lesson in the Beginning Python class. But I will be posting a few more "tips and tricks" lessons, little things that didn't fit in other lessons plus suggestions for useful Python packages students may want to check out as they continue their Python hacking.

Beginning Python, Lesson 6: Functions and Dictionaries

Lesson 6 in my online Python course is up: Lesson 6: Functions and Dictionaries.

We're getting near the end of the course -- partly because I think students may be saturated, though I may post one more lesson. I'll post on the list and see what the students think about it.

This afternoon, though, is pretty much booked up trying to get my mother's new Nook Touch e-book reader working with Linux. Would be easy ... except that she wants to be able to check out books from her local public library, which of course uses proprietary software from Adobe and other companies to do DRM. It remains to be seen if this will be possible ... of course, I'll post the results once we know.

Beginning Python, Lesson 5

Lesson 5 in my online Python course is up: Infinite loops, modulo, and random numbers.

It's a motley mix of topics, mostly because I wanted to have a fun homework project that actually did something interesting. I hope everyone enjoys it!