Monday, October 03, 2005

How big is big?

Finally time to finish this up... sorry it took so long. The only way to get this is actually add stuff up, and I've been busy.

Last time, I talked about the awful article in the Collegiate Times humping the administration over how great the Math Emporium was. At the end, we had the claim that, really, the Emporium didn't target lower level courses because the Math department didn't care about them. Really, it was just about class size, and that's all. You just naturally put the biggest classes online and leave the rest in the classroom.

Let's check a few things here.

Let's check out the timetable to see how large classes are, and look at whether they're online or really meet. I'm leaving out courses with less than a couple of hundred enrollment, so we'll just look at large enrollment sections. I'll also leave out the Honors sections--those won't go online. Honors for 1114 meet in the classroom now, even though everyone else has to take it online. I'm also leaving out enrollment in three special classroom sections of 1015--yes, you heard right: there are classroom sections of 1015. I don't know what kind of a note from God it requires to get into one of these, but they exist.

I'll start the list in order by course number:

1015: 1178 (online)
1016: 789 (online)
1114: 1461 (online)
1205: 1124 (classroom)
1206: 705 (classroom)
1224: 733 (classroom)
1525: 844 (online)
1526: 272 (classroom?)
1535: 348 (classroom)
2224: 856 (classroom)
2015: 427 (classroom)
2214: 651 (classroom)

1526 follows 1525, so I guess it'll be online soon, but at the moment it seems to meet in a classroom.

Hmm... now obviously, none of the 2000 level courses have been put online, so there does tend to be a trend toward lower level courses being put online. But we're assured it's based on enrollment, so let's sort these by enrollment levels. Then we'll just have to draw the dividing line to find out which courses are "big enough" to put at the Emporium:

1114: 1461 (online)
1015: 1178 (online)
1205: 1124 (classroom)
2224: 856 (classroom)
1525: 844 (online)
1016: 789 (online)
1224: 733 (classroom)
1206: 705 (classroom)
2214: 651 (classroom)
2015: 427 (classroom)
1535: 348 (classroom)
1526: 272 (classroom?)

Oops, that doesn't really work. 1205 and 2224 both have higher enrollments than 1525, and they decided to put 1525 online this semester. And everything's bigger than 1526, which is probably next in line.

Maybe we could try a different system... I know, how about this:
For Math Majors:
1114: 1461 (online)
1205: 1124 (classroom)
1206: 705 (classroom)
1224: 733 (classroom)
2214: 651 (classroom)
2224: 856 (classroom)

For Everybody Else:
1015: 1178 (online)
1016: 789 (online)
1525: 844 (online)
1526: 272 (classroom?)
1535: 348 (classroom)
2015: 427 (classroom)
Oooh, now that makes a neater dividing line. If we look at courses math majors take, these courses get offered in the classroom (with the exception of 1114). Of course, if you're in the honors program, you take 1114 in the classroom too. I guess they must really hate the honors students to force them into that awful, passive classroom environment, where they lost and tuned out after 10 minutes.

Everybody else in a big course has to take it at the Emporium, except 2015 and 1535. (Which are pretty small for "big".)

So I guess the next course to go online would have to be 1205, the first course math majors take, since it seems to have been unfairly passed up this time. Wanna make a bet on whether it's 1205 or one of the last courses for non-majors that goes online next?

Isn't it strange how this much better method of teaching is chivalrously denied to the department's own majors, and restricted only for those poor souls who are from some other field? You'd think they'd be jealously restricting it to courses for their own students, since it's clearly so much better.

Saturday, October 01, 2005

What a difference seven years makes.

In 1998, the Chronicle of Higher Education published an article critical of the Math Emporium, which was a year old at the time. The university's PR machine responded quickly with a letter to the Chronicle, which the Chronicle was diplomatic enough to publish. That letter appears here in its entirety.

To the Editor:

The headline of your February 20 article about the Virginia Tech Math Emporium, "Students Dislike Va. Tech Math Classes in Which Computers Do Much of the Teaching," does not accurately depict the comprehensive nature of the emporium and its potential as a learning environment. Although the voiced discontent of some students is a part of the Math Emporium experience, your article, and particularly its headline, fail to draw a true picture of the situation.

The Math Emporium offers much more than computer tutorials and on-line mathematics courses. It is designed to serve students who have a wide variety of learning styles. For example, students can watch and listen to lectures recorded on CD-ROM ... . Students' learning is further supported by a myriad of professors and teaching assistants available in the emporium 74 hours per week. These teachers are ready to help in a variety of ways, from a quick suggestion on how to work a problem to a full, individualized lecture on a topic.

2005: The lectures on CD-ROM are no longer offered. The "myriad of professors" is currently two, who are scheduled in overlapping morning and early afternoon time slots when students are most likely to be in other classes. The number of hours that staffing is available has shrunk from 74 to 58. Floor staff are forbidden from offering a "full, individualized lecture on a topic". In other words, every point mentioned in this paragraph (and those following) has been eliminated or curtailed.

In the linear-algebra course, students have the choice of going to actual, live lectures sprinkled throughout the day to accommodate their schedules -- but, as even your article pointed out, only one-third of the students, at most, have chosen to attend those lectures. Our experience with pre-calculus students is that only about 5 per cent attend the optional lectures that are made available to them. Consequently, our students are voting for the new methods over the old with their feet.

2005: The "actual, live lectures" are a thing of the past for the linear algebra course. In the other courses -- 1015, 1016, and 1525 -- a 30 minute "help session" is offered in only one timeslot per week, and if you are in another class then, you're out of luck.

A few students voiced concerns to your reporter about the Math Emporium approach, but studies in progress indicate that students learn better under this method. We are well aware of the fact that many students are accustomed to passive learning and, at first, find active learning to be uncomfortable. Therefore, we go to great lengths to help students make the necessary adjustments. After the initial adjustment period, most pre-calculus students say they like many things about the emporium-based course. The two that stand out are the self-paced nature of the course and the fact that they can take as many as three versions of each test, with coaching offered to support their learning between attempts.

2005: "Studies in progress" are conducted internally. To my knowledge, no review of the Math Emporium by a body external to the university has ever been published. "Self-paced" is a misnomer; these courses have weekly deadlines that are electronically enforced. Statements like "Deadlines for Graded Quizzes 2-14 will not be extended for any reason" appear in the course contracts for all of our online courses. As well, students may now take each test only once in 1114, no more than twice in 1015, 1016 and 1525. The "Coach's Corner" has been long, long gone.

While we are concerned with our students' comfort level, the design of the courses offered in the emporium focuses on student success. We taught several experimental courses in smaller computer labs before moving to the full-scale Math Emporium courses. We used the results of extensive assessment of the experimental courses to design the current courses. Of course, the process of assessment and development is an ongoing one. ...

The ellipsis here was part of the originally published letter; I don't know if this was an omission by the Chronicle or part of the original text. 2005: Ongoing development means moving more courses online; this year, 1525 has been replaced with an online course, and 1526 will be moved online in the spring. Except for an early transition from commercial material to in-house material, there is little apparent ongoing development of already-existing courses.

The rest of the letter is general fluff, and not a specific rebuttal to the Chronicle article; I'm posting it without further comment. I'll mention that Robert Bates and Robert Olin have both left the university, and Sally Harris has transferred internally to another position.

The statistics clearly show that students are understanding core concepts better after taking mathematics in the emporium. We are seeing grades rising and failure rates dropping. We expect that our subsequent evaluations of student learning will also show the value of this educational experience. Furthermore, given that it is believed that most college graduates will have to retrain themselves several times during their careers, and given that active learners are in a far better position to use the World-Wide Web and other resources to retrain themselves, we are convinced that the emporium experience will add great value to our students' education. The importance of these objectives for our students far outweighs the negatives associated with the degree of discomfort that our students are experiencing and, we feel, should have been the subject of the headline of your story and should have played a more prominent role in the article itself.

At a time when universities are faced with decreasing revenues and increasing numbers of students, it is imperative that new ways of teaching be explored. With the Math Emporium, we are not only offering a new way of teaching but are conscientiously evaluating the results of the new approach in an attempt to find the best methods for teaching in a new age that will have vastly different expectations of graduates. ... The many universities that are using technology to improve the quality of the education their students receive would be well served if prestigious publications such as The Chronicle would promote informative discussion of the pedagogical issues involved in such enterprises as the Math Emporium so that we all could learn from each other.

Robert C. Bates
College of Arts and Sciences

Robert Olin
Department of Mathematics

Sally L. Harris
Public-Relations Coordinator
College of Arts and Sciences
Virginia Tech
Blacksburg, Va.

Wednesday, September 28, 2005

Welcome to LJP

We've added an author; LJP will be joining us. He's working on some good stuff right now.

Class size stuff is still coming, but everything's busy now.

Sunday, September 25, 2005

And now the CT

Unbelievable. One day after I go online with this and start to talk about all the garbage the University likes to spew about how the Math Emporium spins straw into gold and lets students walk on water, we get one of the most insane fluff pieces in the Collegiate Times itself. This is really shameful. I had wanted to take a longer look at the article I took the quote in my last post next, but I can see I'm going to have to rip this one apart first.

The beginning of the article ("Math Emporium Frees up Classroom Space") talks about (reasonably enough) the budget crisis and its effect on classes. Not directly of course. It talks about how the increase in enrollment has lead to a shortage of classroom space. But now why would the University be trying to squeeze more students in than can fit in the classrooms? Oh right--to try and get more tuition.

Well here comes the Math Emporium to the rescue! No need for a classroom if you just take your classes online, so we can just squeeze, and squeeze, and squeeze more people out there until the whole thing pops. (Leave me my fantasies, OK?) And if you've ever been out there near a quiz deadline, you know squeeze is the right word.

But from there, the article quickly turns into a discussion of how much "better" the Math Emporium is than those nasty old classroom-based classes:

"Back in the early 1990s, there was a lot of emphasis on using technology to improve things which is why we decided to convert in 1997 from the traditional classroom instruction to the Math Emporium-based technique," said Michael Williams, associate professor and director of the Emporium. ... Williams said the state supported the program to use technology for classroom instruction and decided to begin with the math department.

"The philosophy was to turn students into independent learners," Williams said.

Independent part achieved; students aren't getting any support from anyone anymore. We're not so sure about the learning part. And here's my major question about the whole "making students more independent" thing: Could they name one specific thing happening in courses at the Math Emporium that is designed to help students become more independent, other than providing absolutely no support to students? Throwing someone into a lake is not really a philosophy of teaching swimming.

So let's see what we learn about the Math Emporium, once again as compared to those nasty, nasty, actual classes:
"We were wasting time with the lecture classes because students were only paying attention for around 10 minutes or would lose a step in a math process and couldn't recover."
OK, quick show of hands: Anyone ever felt like the Math Emporium wasted your time?

This is also where the article enters the surreal. Let's see, traditional classroom: I miss a step in the beginning of a problem, and say for instance, forget to square something or lose a negative sign. A real live human being looks over my work and lets me know: "Your reasoning was right, but you forgot to square this term at the beginning... 9.5/10".

Now let's try the same scenario at the beloved Math Emporium: Look at that, my answer isn't one of the four approved answers I can click on. I can't figure out what went wrong--looks like the same way I've studied these problems. Let's try the closest answer--oops, that was wrong, I just got a 75 on my quiz, with no explanation of why. I'm no better equipped to understand what went wrong. I don't know what step I missed, I don't know how wrong or right I was, and the student next to me who had no idea how to even start the problem has just gotten that question right because their guess was luckier.

In fact, in a lecture class, a lot of steps get "recovered" everyday. And when I look for help, I have a teacher I know and who knows me. A teacher who may have some idea what my background is, what homework problems I'm working on, and what may be giving me trouble. Either in class or during office hours, I can get some help with this. At the Emporium, I can keep putting up a cup until I find someone who will attempt to explain it to me. At the end of course, any minor error on my part results in an instant 75 on my quiz, no matter how much I understand or don't understand the material.

Oh yeah, and notice that the amount of time it takes for students to stop paying attention is down from 20 minutes (see the quote in my last post) to 10 minutes? That's halved in less than a year. Amazing. (And yes, it's the same guy quoted both times. He's obviously gdesperatesparate--his anecdotal evidence is getting more ridiculous.)

The article goes on to quote a student who "doesn't mind" the Math Emporium (what a ringing endorsement--must be the best they could get). Then we get the following gem:
"When we started out, we tested out by having parallel sessions with traditional lectures and the technology-driven emporium class and then measured the future success in calculus courses," Williams said. "The emporium-based class was found to bring more prominent success in the future for math students, but we haven't done this test since then because it is hard to do a statistical analysis of this due to the difficulty in pinpointing what is being measured."
So what's he saying here? Let's break it down.

First, he talks about how much more successful students once were who took math at the Emporium at some point in the past. Then he explains that they stopped doing the analyses because nobody had any frigging idea what the hell they were measuring, if anything. So on some nebulous, undefined, something, people who studied at the Math Emporium were much better at it, whatever it was.

Second, I have to wonder what the "later math courses" were. Let's see, 1015 goes on to 1016, and that's about it. I see there's also a 2015 and 2016 in the catalog, but I don't think most students go past 1016. (The enrollment numbers are a lot smaller anyway.) Now 1016 wasn't always online, so I'm assuming they mean performance in this class for benchmarking 1015.

Of course, this is really measuring people who could pass 1015 at the Math Emporium in the first place, as opposed to those who (a) failed; (b) late dropped; (c) avoided taking 1015 and decided to take it somewhere else and transfer it instead. I have no doubt that there are math brainiacs taking 1015 who don't care that there is no teacher and they just have to teach themselves. And I have no doubt that these students usually do better in later courses than students who actually need someone to teach them. Whaddaya know: If you can pass 1015 at the Math Emporium, you can do anything. I think we're back to the "throw 'em in the lake" philosophy of teaching swimming again.

Let's continue in the article and see what they include as dissent to the new orthodoxy of the Emporium:
Some students feel that the traditional lecture-based classes would yield a larger learning environment, but determining an alternative to the emporium is nearly impossible.
I'm not sure what this person means by a "larger learning environment". I don't think we have any classrooms larger than the Emporium. Maybe they meant to say "better" or something? Who knows. Maybe this person learned to write at the "Journalism Emporium."
"Even though I feel like I didn't learn anything by taking math at the emporium, I don't see how else they could get the information out to such a large number of people, said (name), junior biology and psychology major.
This is it? This is the full extent of opposition to the Math Emporium? A quote which starts off as semi-opposed, but which ends up supporting the Math Emporium, even though he "didn't learn anything by taking math at the emporium." If he didn't learn anything, what was the point of doing it at all? If the Emporium doesn't teach anything, why not just eliminate the classes altogether?

And how could they get this information to so many people without the Emporium? How did they do it before we built this monstrosity? Oh right, there were actual classes, with an actual teacher. Some classes are still taught that way, even with large enrollments. But I digress.

Williams said that the Emporium has been financially advantageous because it costs less to run the classes in that way and Virginia Tech doesn't have enough building space in order to hold the classes in an on-campus building.

Here's something I buy. The Math Emporium is cheap. (Remember my comments about "outsourcing" professors jobs last post?) To be honest, I'd be less annoyed if the administration just admitted, "We can't afford to teach real classes, and we're doing what we can, and that's why we have the Emporium." Not that I'd let 'em off the hook, but I'd give 'em some credit for honesty. And it might be a good place to allow some changes that might make the damn place better.

Notice what happened here? The article started off as "news" about there not being enough classroom space, and suddenly became a big puff piece on how great the Emporium is, and how, golly, there just isn't any other way to do things anyway. But we're fortunate it's so great. This is front page news?

I almost find it hard to believe that the CT felt it necessary to just quote all the talking points of the Math Department about how great this all is, and didn't bother to ask any questions about this. I particularly like the "factoid" box at the bottom of the article, which includes under its first bullet point the following:
The Emporium is equipped with software that provides a better learning tool for students.
They actually published this on the front page. The Emporium just is a "better learning tool for students." This isn't (supposedly) an editorial. This is just a statement of "fact" on the front page of the paper. Not: "The Math Department claims", or even "Scores on such-and-such are higher since..." but just "The Emporium ... provides a better learning tool...." Not even an answer to the question "Better than what?"

It seems maybe the Math Emporium wasn't popular, and the university decided they'd better do another PR piece on how students should just love it. And the CT knows to ask "how high?" when the administration says "jump." If someone in power says it's better, well, they'll run an article saying it's better.

Finally, at the end of the article, we're promised that it has nothing to do with the on-line courses being low-level courses the math department doesn't care about; really it's just about the size of the enrollment:
"We aren't targeting freshman classes in particular, just the largely enrolled classes because that is what makes the system work the most efficiently," Williams said.

I'd like to look into this claim a little more carefully next time. What do we actually see, courtesy of Hokie Spa. Stay tuned...

Monday, September 19, 2005

"One-to-one" help

You can find plenty of criticism of the Math Emporium, mostly from students who have to use it. (You can also find plenty of fluffing of the Math Emporium. Tech puts on a great PR face and gives it big wet sloppy kisses constantly.)

Some of the criticism doesn't come across so well. Sometimes it just sounds like "it's hard," and the fluffers like to chuckle about how the students would just like to "sit back and let it come to them," or about "sitting in the back of a class of 500 and doing your e-mail." (Comments taken from the first two pages of Google hits for "Math Emporium.")

Yeah, we get it. Students are lazy, and if we weren't, we'd just love, love, love the Emporium. (This is for your own good, don't you know...) In fact, we're assured that, gosh, at the Math Emporium, students get lots of individualized attention. It's not like sitting in a classroom with 40 students and one teacher. Why, this is all one-on-one, personalized and individualized instruction.

Like in this article, linked off the Math Emporium's homepage. (Check out the visitors links for some real fluffing.) We hear the following:

"What is (traditional) teaching?" asked Mike Williams. "It's 40 to 80 students in a room. A broadcast, not unlike watching TV. A very passive act. The majority are zoned within 20 minutes. My view is that the lecture is not worthwhile. But there are those who are very prideful about their material, being the 'sage on the stage.'"

One instructor with many students is highly inefficient, Williams added. "What we have now is one-to-one. We train our helpers to be good listeners, not to solve the problem for the student but to figure out the right question to make the light go on. The work of discovery changes a person's brain. We try to understand exactly what it is the student doesn't understand."

How fantastic! Students aren't getting that nasty, passive, classroom teaching, but careful one-on-one attention from people specially trained to piece together just exactly what isn't understood. Bravo, bravo... And isn't it amazing that this brilliant work can be done in under five minutes per case from somebody who's only a junior in college and getting paid by the hour to work at the Math Emporium?

Of course, there's a real professional teacher (one), who is in charge of each of these courses of thousands. So at least there's someplace students can get that carefully crafted, one-to-one teaching. Let's hear from the students, from, about how a 1015/1016 teacher handles student questions. (I've deleted references the teacher's name, because this isn't about a teacher. This is about what I think are problems with the Empo itself.)

  • She did not help much, usually said to just look at the Lesson Pages. The help at the emporium is a joke.
  • This lady does not like giving help much.
  • if you ask her for help she asks if you read the lesson and then pretty much asks you to figure it out yourself. there is no real help for this class.
  • she gives you no help grade-wise or with problems.
  • This is an online course and she is completely worthless when you ask for help....all she does is ask if you've read the lesson(which doesnt help) and then gives a smart remark. You are on your own in this class.

There is a positive review of this person teaching this class, but no comments with it to say what they liked.

Hmm... how about (This one includes courses other than 1015 and 1016, so I'll restrict to comments about these two.) This one is a little more mixed, but do we begin to see a pattern in some of the responses?

  • Dont ask HER for help. She will just refer you to her lessons. She thinks everyone should be able to learn off the computer.
  • She can either be really nice or really cruel when you don't understand something. And she never really explains anything either.
  • [She] does not help her "students" at all. If you put your cup up and get her and you tell her you don't understand the material, she will say "well did you read the lesson pages." Even if you did read the lesson pages she will say "did you work out every example, you need to go back and work out every example." She does not help to explain the material. And IF she does help, she seems to have an attitude.
  • If you ever manage to track down [the teacher] at the emporium, be prepared to not be helped. Try and get someone else over there to help you with any problems. I found that her knowledge does not extend beyond the lesson pages on the computer screen (which are not helpful for the quizzes and tests). I finally was able to get her for help once and all she did was read me the computer screen, and I can read too so that did nothing for me.
  • she seemed nice enough, and when you could get her specific attention, then she could explanin material to you.
  • she was helpful if you asked questions and she responded to emails promptly.
  • She was very helpful and considerate of students, and does a good job explaining something to you if you donĀ“t understand.

Now the last three (and oldest) reviews speak fairly positively to the help that this teacher can provide, so credit where credit is due. There are lots of both positive and negative reviews, but these are everything I could find that relates to how this teacher deals with questions at the Empo.

Personally, I've seen the same thing out there ("just read the web pages again!"), so a lot of people are just putting up a cup and hoping to find somebody else who will explain it to them. (And if the first one doesn't work, try, try again...) And who else do we have? Student workers paid hourly, or professors from completely different classes who have probably never taught 1015 or 1016. I can't imagine why the help varies so much from person to person.

So we've outsourced jobs for professors to a bunch of students on hourly wage, and the one remaining professor that essentially just runs from pod to pod saying the equivalent of "read the textbook and figure it out on your own." Of course, I'm not surprised she doesn't have time to individually tutor each student. I'm also not surprised if trying to individually tutor each student gets frustrating for her (maybe why some people above had some harsh experiences), but then maybe having one professor for a couple thousand students wasn't such a hot idea in the first place.

Here's a thought: let's take it one step further and really outsource this. Hire tutors in Bangladesh or somewhere who will work for twenty cents an hour. We can set up video conferences on the computer to handle the student questions. Heck, at that price, maybe we can afford to have a tutor available 24/7 for every 1015 student, and finally get that "one-to-one" tutoring we've been hearing so much about. And best of all, nobody would have to every have human contact with anybody ever again. (On second, thought, don't mention this. They'll try it.)

So I don't think this is a problem with somebody teaching a course. I don't think it's a problem with the students taking the course. I don't think it's the fault of students working for hourly wages at the Empo. (Hey, I've known some, and they actually work their butts off out there.) I think it's the result of having one professor for a couple thousand students, plus a bunch of other people (including other students working hourly) who don't really know what's going on in the course and have to try to answer each question as fast as possible so they can move on to the next cup. That's systemic. It's the result of decisions made at the top, and anything to fix it would be a pretty radical change.

And that's what this blog's mostly for, to talk about some real, legitimate criticism of the Empo that I don't see get talked about much. I'm sure plenty of other people have lots to say too, and maybe we can get some good discussions going. Maybe even some "self-help" among other students still taking these courses.

Some suggestions about how to really improve the Empo would be great too. (Even better if any of these find their way into happening! I think we're stuck with the place for a long time anyway.) And if we actually run across anything good about the Empo, well that's fine too.