Shortlink

Rating Professors

survey.jpgOnly 2 percent of respondents in a Harris poll changed physicians based on online ratings. I have yet to find anyone who made a course choice based on sites like RateMyProfessors.com. Word of mouth, which often occurs through social networking, is much more frequently used than these rating sites for locating professors, doctors, and other professionals.
For many professions, there is no centralized feedback mechanism other than rating sites, however faculty at most schools are rated in end-of-course evaluations. They are well-designed surveys administered in a rigorous manner, typically anonymous, although I’m sure I’m not the only one who guessed who authored a comment or gave me a rating.
Clark Quinn told me that when he was a student at UCSD, they used Course And Professor Evaluations (CAPE), which was student-run but mentored. It was used by students and promotion committees alike (though the latter wasn’t official). In contrast, sites like RateMyProfessors do not have systematic data collection; while they get quantitative data, it’s not likely to be thorough and consequently the validity is questionable.
Some further arguments against professor evaluation websites include that students are “more likely to post if they have either a very positive or very negative experience” even to the point of using it for “revenge for a bad grade” and that the people posting are not necessarily students. The arguments for the sites include that they empower students and that they hold professors more accountable.
Curious about the rating sites, I looked to see if any of my friends were listed. A few were and most of feedback was extreme. This is typical of many review sites, where the impetus to contribute is having a fantastic or a horrible experience. I was surprised to find a category on RateMyProfessors.com for how “hot” a professor is. While it did not seem to be widely used, I have trouble understanding the relevance to course selection.
I wonder if the students who use these find the traditional feedback mechanisms schools offer inadequate in some way. And I am curious if these sites, which do not seem to be heavily used given the number of reviews most schools have, will die out or will evolve. Recommendations for professionals happen fluidly between people when there is an opportunity for questioning and clarification, not just a few lines of text. eBay has added additional categories for rating each transaction to refine and make feedback more useful. If faculty rating sites similarly evolve, I can think of new categories to add such as how well the rater did in the course to determine if the review is based on payback.

Shortlink

Is Your Course Hot or Not?

You just developed an online course and everyone loves it, right? While you might believe that your students never doze off or take a break to play a game, do you really know? Most people I asked who develop online materials, myself included, never see detailed usage metrics.
Suppose that instead of developing a course you create videos – not that these are mutually exclusive, of course. Hot Spots provides usage metrics for YouTube videos, measuring the popularity of different parts of a video. “By comparing your video’s abandonment rate at that moment to other videos on YouTube of the same length, and incorporating data about rewinds and fast-forwards,” the “hot” and, most interestingly, the”cold” spots are located.
The most recent course I developed was for the NIH and was about Reasonable Accommodations. It used a case study to illustrate both the complexity of the topic and that a resolution is only effective until circumstances change. The case study, developed by the lawyer I worked on the course with, was gripping – to me. But I was not one of the target students. With usage metrics to show that abandonment was high at certain points, the lawyer and I could modify the story to increase engagement – and hopefully learning as well.
In contrast to how the more recreational YouTube is used, students often need to complete a course. But even with a captive audience, there is great value in learning what the reactions of students are, especially since those who abandon a course are the ones least likely to provide feedback. Having a “hot” course is important, not to win a popularity contest, but to know what keeps students engaged.

Shortlink

Learning Workplace Survival Skills

the-office-nbc.jpgWhile “workplace survival” conjures up images of moose hunting in Alaska (sorry, I’ve listened to too much election coverage), in fact, many of the skills that are needed to succeed in the workplace are never taught. I thought about this yesterday when listening to a talk by Paul Rosenthal, Director for Consumer Segmentation and Engagement at Health Dialog. Paul, an alumni of the joint Emerson-Tufts Health Communication Program, told students and faculty about how he got to and what he does in his current role. Much of what he described was how he developed – and is still learning – workplace survival skills.
A quick search on workplace survival brought up results ranging from wilderness training to a Dale Carnegie course on How to Communicate with Diplomacy and Tact (I love the course title!) None of the courses I saw are offered in the Health Communication Program except Professional Communication. When I teach I convey some information to my students about the “real world”. Paul, who is also an adjunct professor, undoubtedly does too. And then, of course, there is what viewers learn, for better or for worse, from The Office and other television shows.
Ultimately, workplace survival may be mostly learned by experience in the trenches, so to speak. But I believe that more explicit descriptions, like Paul provided in his talk, of dealing with difficult situations successfully, helps. Furthermore, social networking affords opportunities for graduates to stay in touch and benefit, not just from sharing job openings, but from each others’ experiences and lessons learned in the workplace.

Shortlink

What Employees Really Do Online Revisited

“Workers are increasingly using the Internet to do, well, whatever they want,” was a key finding from a study reported in today’s Wall Street Journal. Accessing video is one of the most common online activities of employees, not surprisingly. While many undoubtedly watched the Olympics or the last season of House, others are engaged in learning. Just-in-time learning, in fact.
Suppose the snores from your last talk drowned out your voice – even with a microphone. You are giving a presentation tomorrow and need it to go a lot better. Search on “how to be a great public speaker” on YouTube and you get 564 results (many with little relevance, admittedly) including videos from both self-proclaimed and award-winning world champions. Suppose you have a different problem: you name it, the instructional video is there. And the great thing is that if you want to plan an evening out to celebrate your success on stage, you can check out movie trailers right there.

Shortlink

Predictors of Success: Toward Accurate Screening that Can’t Be Gamed

I ate dinner tonight at an amazing restaurant in Koreatown in New York that has a city-block long buffet. The price for dinner was based on height, with one price for those over 4 1/2 feet and lower ones for 3 1/2 to 4 1/2 feet and 2 1/2 to 3 1/2 feet. I wondered how accurate that was as a predictor of food consumption – and if the waitstaff eyeballs or actually measures clientele. Coincidentally, today’s New York Times had an article about how “some of the country’s most influential college admissions officials” recommend a shift from SAT and ACT scores toward an admission exam that is a better predictor of college grades. Online programs have struggled with the same question of predictors of success with the focus being more on self-motivation and distractibility.
I have seen online universities that offer a test to potential applicants to help applicants decide if they will be successful online learners. I have no idea how successful they are as a screening device although I can think of a number of schools that I imagine use ability to pay as one of their primary screening techniques. In the corporate world, there is rarely screening because there is rarely a choice. Everyone must succeed.
I have heard stories over the years from people who cheat or game the system in some way – I’ll even admit to taking the exam at the end of a course without going through it in hopes that some combination of common sense, good guesses, and luck would help me pass without actually taking the course. The college officials’ recommendation to find more accurate predictors was partially based on the growing role of “the billion-dollar test-prep industry that encourages students to try to game the tests”. At least one thing is certain: I can’t walk in to the restaurant on my knees hoping to pass for under 4 1/2 feet tall.