A long running debate at TEC, is it a good idea or bad idea to enable public visitor comments on our research? I’m not referring to blogs, which by their very nature are intended to enable commentary. I’m thinking in the context of analyst firm research. I think there is a lot of room here to create an interesting and valuable research methodology (I’m sure I’m not the first to say so). Here’s some background on my query.
TEC has published articles and other research on the IT/enterprise software front since the early 90s. For the majority of that time we haven’t asked our visitors to pay for much of this research. I often compare what we offer (rightly and wrongly) to things available from other analyst companies like Gartner. Gartner, for example, has just about everything locked behind its e-walls. It’s almost all for sale over there. If you go to Gartner for a report or some other research, you won’t see commentary posted under the report by regular visitors debating/debasing that report. Should you? Haven’t we all seen that some of the most significant cultural, business, political, and other developments are based on the new communication and collaboration means enabled by Internet technologies?
Back to TEC, I pushed for a while to have a simple comment system on our site. Something that our visitors could use to post thoughts about our articles, podcasts, reports, etc. It was implemented and people began using it. There were a mixture of comments. As you’d expect some were nice, some were not, some were well-thought out, others not so well. C’est la vie.
We didn’t implement an community moderation system like, say, Slashdot does. This then is where potential problems enter. I happen to be opposed to any electronic forum censorship (note: I don’t view a community moderation system as censorship, rather it’s a peer reviewed ranking device). Wading through online censorship experiences first-hand (dating all the way back to the days of BBSs) I’ve seen how censoring comments tends to destroy online communities or at least ultimately drives their quality down (I’d make an exception for things like spam, which aren’t comments in the first place). But that’s another debate.
A portion of TEC vehemently opposed displaying negative or poorly written comments, and with well-intentioned reasons. “Imagine if all analyst firms allowed such comments, they figured” (I’m paraphrasing the ideas). “Would they still be able to sell their research?” I think it’s a good question. Will people see commentary by other visitors and lose trust in what you have to publish? Does it detract from the professional image of the site? After all, sites like my Slashdot example, never portray themselves as analyst firms–they aim for a different impression entirely. Can an analyst firm, often sought out as subject matter experts, survive while fostering its own public criticism?
I think it could. A well-considered approach could enable that firm to take the reins and harness that criticism to improve. I think if you really are a subject matter expert, or even if you’re not an expert (I’m more of a generalist) but practice well-refined analysis and synthesis skills, you have nothing to hide and would welcome the opportunity to discuss your research publicly.
I would like to see greater online visitor participation. I think there is a lot of potential in getting all the different people related to aspects of the IT industry involved in voicing their activities, concerns, ideas, etc. around a specific body of research. It would probably make that research more valuable rather than detract from it. It could even give the firm totally new ideas for improving their products/services, just the way participation in FOSS development can.
Of course right now, we can all do this to some degree using blogs, but then aren’t we all just circling around the research, rather than assaulting it directly, in its home, where everyone else gets a chance to form some perspective. Maybe it’d be in an analyst firm’s interest to maintain that home? I’ve seen several peer reviewed journals on the Web, like First Monday. A few sites, such as ITerating, seem to be making some sort of effort to approach certain forms of IT research from this angle. RedMonk is interesting in that they espouse a similar idea through blogging. As I mentioned at the start however, the potential for reader participation is inherent to blogging; it’s not the same as offering a particular piece of research or report (or the methodology of developing it) to be ripped to shreads, lauded, or critically enhanced by its community of software users, consultants, vendors, developers, etc.
Whether or not it can be purchased is relevant to the business model, but not so much to the greater issue of what’s more useful–what can be done better? If you could derive a certain edge from opening up all your analyst research to public commentary, I think you might discover some very interesting competitive advantages. I’ve got ideas–but that’s for another time. I don’t know that I made a strong enough argument for the value of uncensored commentary and had to ask the dev team to remove the comment capability altogether. Maybe the implementation was too basic. Perhaps down the road we’ll find a way to make it more productive by implementing it differently. In any case, in the meantime, I’m happy to say that although our comment system died today, we simultaneously launched an official TEC blog. And that will be the subject of my next post.