We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

The Performance Principle

Russell Rothstein

Harvard Study on Expert Reviews and User Reviews

Vote 0 Votes

I came across a fascinating research report from Harvard Business School that found that "user reviews tended to reflect fairly accurately the judgments made by professional reviewers." While the study focused on online book reviews, I believe that the findings are relevant also for reviews of enterprise technology products and services.

What's clear is that there is a revolution happening online with user reviews and IT Central Station is leading that revolution for enterprise IT. Write a review of a product you use and make your voice heard!

Here is an extract from the story:

If you're trying to decide whether or not to read a book, where are you more likely to find useful answers? In the New York Times or London Book Review, or among the short, amateur reviews on Amazon.com and sites like GoodReads? It may not help to know that one may be as good as the other. A new study from Harvard Business School shows that, when considered as a whole rather than individually, the hundreds of reviews they analyzed tended to reflect fairly accurately the judgments made by professional reviewers.

There are differences, certainly: pro critics are harder on first-time writers and easier on award-winners. That sounds a bit stereotypical of the critic (snobbishness and joining others in acclaim), but it may also just be the result of having read more books.

And the general public is no less biased, if one can be biased in giving one's opinion: novice reviewers prefer uplifting stories and likable protagonists. So unpleasantly real and flawed characters -- award-winning author Cynthia Ozick notes that this list includes Hamlet, King Lear and Middlemarch among others -- and bleak circumstances lead to more negative reviews.

The question seems to be whether you are looking for a recommendation or a professional judgment. You'd turn to your friends and acquaintances to learn about a good sushi restaurant, but most of us will admit that Roger Ebert's opinion on cinema should be weighted higher than our neighbor's. And yet if you took the average of a hundred people's opinions, it would resemble the pro opinion more than you might expect.

Regardless of whether "the democratization of reviewing is synonymous with the decay of reviewing" as author and City University English Professor Morris Dickstein put it to The Daily Beast, there seems still to be room for both word-of-mouth recommendations and tastemakers like review magazines.

The Harvard study, by Loretti I. Dobrescu, Michael Luca, and Alberto Motta, can be read in its entirety here on the Harvard Business School website. The story appeared on the msnbc.com website under the title "Are Amazon reviews usurping the role of professional critics?" It was written by Devin Coldewey, a contributing writer for msnbc.com. His personal website is coldewey.cc. Will user reviews "democratize" the role that professional consultants and analysts have in recommending enterprise technology? What do you think? Post your comment below or send me an email.

Russell Rothstein blogs about cloud computing, performance management, business service management and related topics, examining how new technologies and business models impact the dynamic IT service management market.

Russell Rothstein

Russell Rothstein has spent his 20+ year career in the enterprise technology industry at the crossroads between technology and business. He has spoken at industry events including Interop, CloudConnect, CMG, Red Herring, and TeleManagement World. Russell is currently Founder and CEO of IT Central Station, a B2B social networking site that provides user reviews and ratings of enterprise software, hardware and services. Previously, Russell was Vice President of Product Marketing at OpTier, a vendor of application performance management (APM) solutions. Before joining OpTier, Russell was AVP Product Marketing at OPNET Technologies (Nasdaq: OPNT) where he helped lead the company’s focus into APM. He was co-founder and CEO of Zettapoint, a venture-backed enterprise software startup that was acquired by EMC, and ran marketing for Open Sesame, a Web 1.0 startup that was acquired by Bowne/RR Donnelley (NYSE:BNE). Russell began his career at Oracle, deploying Oracle Applications for Fortune 1000 companies. Russell received a BA in Computer Science from Harvard University, an MS in Technology and Policy from MIT and an MS in Management from the MIT Sloan School of Management. Follow Russell on twitter at @RussRothsteinIT .

Recently Commented On

Recent Webinars

    Monthly Archives