Newsblur as an Intelligent Agent Used for CI

Newsblur ( is a Web-based RSS feed reading service. This is a review of how I found the service useful while working on some competitive intelligence (CI). It’s convenient and conducive for tracking issues, trends, commentary, and news.

Newsblur has a set of features that I find make it worth paying for an annual subscription (it also offers a no-cost option). I’ve long used RSS readers, initially preferring desktop readers like Akregator or RSSOwl. The quantity of feeds that I follow is so large now that it can take desktop readers a long time to update, whereas Web services can use their resources more efficiently to deliver the feed content. I currently use Newsblur to follow about 1300 feeds but I’ll just cover a few examples in the context of how it might be used in CI work.

Newsblur sets itself apart from many other feed readers with its “Intelligence Trainer” functionality. The Intelligence Trainer enables the system to proactively filter new entries by marking key words, tags, authors, and sites that should be raised as a priority for reading because they match user-specified characteristics. The Intelligence Trainer also allows a user to decrease the priority of entries based on the same criteria.

A user can go through a step-by-step process of training the service for every feed—but that can be time consuming so users can also train it on an ad hoc basis, whenever there is a good opportunity. The Intelligence Trainer proactively filters results that would otherwise require manual searching.

Examples with the Intelligence Trainer

In order to research some issues related to the subject of open innovation (OI), I created a folder in Newsblur to store feeds from any site with potential to publish information on the subject. I also created subfolders for feeds from organizations that might offer information even though their domain was not my primary interest. This allowed me to distinguish and pay better attention to the context of my sources.

Intelligence Trainer example
Newsblur Intelligence Trainer example

I trained Newsblur to identify key words or phrases, examples I’m showing for this post include “open innovation”, “r&d”, “management practices”, “patent”, etc. These varied depending on the particular feed. Some feeds, for example PubMed, can be pre-filtered for the open innovation topic so the Intelligence Trainer could be focused more on specific issues within that subject. This technique can produce useful results.

Although it’s possible, I wouldn’t recommend manually re-entering those terms to search the feeds for relevant information.

Newsblur "focus" screen
Newsblur “focus” screen shows entries the Intelligence Trainer filtered.

Instead I’d click Newsblur’s “focus” button (after setting up the training) and scan the results that it provides me—these are essentially automated search results based on the intelligence training. I would then use the search function if the recall was too high and I wanted to narrow the list of results. For example, I trained my feed for PubMed OI on following terms:

  • partners
  • collaborative
  • shared governance
  • open source
  • patent
  • management practices

This resulted in about 10 entries from a list of over 100. From within that pool, it’s possible to search on specific terms but that can produce overly narrow results. A sampling from scanning the results on their own (without the deeper search term precision) produced a good number of entry titles, which suggested useful intelligence. When I reviewed results this way, they strongly matched my need.

How it Works and Other Notable Functionality

title text intelligence training
Newsblur Intelligence Trainer example 2, with title text.

The Intelligence Trainer (as far as I understand) picks up words based on the entry title, body text, the feed publisher’s tags associated with entries, authors’ names, and the feed name itself. It allows the user to specify different terms based on the fields you want to use. I discovered a “cheat” to train the system for words that haven’t yet been included in any entry’s text, by typing the word into the editable title field. That’s useful to prepare for potential future interests.

Newsblur has other functionality that can be useful for CI work.

Newsblur example showing how it displays edits to an entry
Newsblur example showing how it displays edits to an entry trained on the term “pharmaceutical”

If a user wants to see changes to entries over time, it is possible to turn on a change tracking feature. For example, a feed of press releases might seem static: the release is issued, the reader picks it up, and the user reads it. However it’s possible that a PR person made an error and later corrected the release. In such a case, Newsblur can be set to show a red strike-through of the original text, and have the new text in green beside it.

Newsblur also provides a “save” feature and a Blurblog. Both of these features enable different ways of capturing feed entries for later review or collaborating with other people. In both cases you can add notes to the entry and review other users’ notes.

Previously, Newsblur’s search was somewhat confined but now it supports searching across all of a user’s feeds. It would be nice if it had advanced search functionality that made it possible to designate which fields to search (title, body, etc.).


Although the intelligence trainer is a very nice feature, it requires some amount of effort to use it well. Newsblur would be more useful if it offered a management tool that pooled all the terms and tags you’ve identified, to prioritize or deprioritize. It currently requires that you manage them feed-by-feed or story-by-story. It should also enable users to organize terms by folder groups, to apply to all the feeds within, or across folders.

Generally users need to paste the URL of an RSS feed into Newsblur. The system makes a valiant try at discovering feeds on a Web site if the user doesn’t know the exact URL. If the site has no available feeds however, Newsblur cannot track its content and this of course is one of the limits in using RSS for competitive intelligence.

A risk with any third-party, Web-based service is the lack of control: it can be shutdown (as Google demonstrated with its Reader). While Newsblur has proven reliable and convenient with regular refinements to features for over a year (in my experience), it is a small organization so there is little guarantee of its longevity (of course, Google is big and that didn’t guarantee Reader’s longevity). It is easy however, to export your list of organized feeds into an OPML file to transfer to another reader if the site were ever to shut down.

Overall, I find Newsblur a very efficient way to track a large quantity of updates and news related to a CI subject but I would not recommend relying on any RSS reader as the sole means for finding and tracking information. Applicable intelligence results depend on the utility of the feeds that you’re able to find, as well as your resourcefulness and precision in refining the Intelligence Trainer.

CASAA Birthing – New Decision and Knowledge Engines

I’ve been talking about computer-assisted shallow atom assembly (CASAA) in my posts thinking about how we acquire knowledge in life with the pervasive Internet. Yesterday I read about Microsoft’s new search engine, Bing, which they’re actually calling a “decision engine.” From what I’ve read they’re making a clear effort to push search in the CASAA direction. Look how Balmer describes it: Continue reading “CASAA Birthing – New Decision and Knowledge Engines”

The Nervous System’s Emerging Stream

In a recent post, Nova Spivack considers “the stream” as the Internet’s next evolutionary stage. I think he makes a lot of compelling points and I’m clearly partial to stream terminology (like it says above, I’m trying to mind the current). It builds on McLuhan’s notion of the nervous system, which is neat. Spivack’s conceptualization of recent Web innovations are something akin to a stream of consciousness, or more specifically streams of thought and conversation. But I end up wondering how fluid this stream really is. Continue reading “The Nervous System’s Emerging Stream”

Acquiring Knowledge: Computer-Assisted Shallow Atom Assembly (2)

In a previous post, I said that search engines essentially accomplished their jobs but created a big problem.

Search engines initially answered our question of “How or where can I find the information I want?” but in indexing the content of the Internet and providing access, they created a much more troubling problem. That question tends to overshadow another question, which is equally if not more important, “How do I assemble knowledge from the information I find?” That question will be solved by computer-assisted shallow atom assembly, which I think may be a new significant stage of Internet-related development. Continue reading “Acquiring Knowledge: Computer-Assisted Shallow Atom Assembly (2)”

Acquiring Knowledge: A Great Shallow Breadth Over Depth (1)

Has our approach to acquiring knowledge moved from the deep end of a continuum to the broad but shallow end? The Internet medium and associated technologies used to develop, contribute, and distribute knowledge with it, call out for knowledge acquisition through breadth. I think, in general, we’re using it to acquire knowledge via a great shallow breadth of sources over acquiring it via single deep sources. We’re developing an acceptance that acquiring knowledge via a great shallow breadth delivers an equivalent fulfillment of knowledge and in most cases, we may even be developing a preference for this method of knowledge acquisition. Continue reading “Acquiring Knowledge: A Great Shallow Breadth Over Depth (1)”

Personal Wikiesque Note Taking Mind Mappish Killer KDE App: BasKet

I’ve found one of my favourite applications ever. It’s called BasKet Note Pads. Here’s my problem, I’m always typing up little notes to myself and saving them as text files, all over my desktop, all over my hard drive. Sometimes I try to organize them, sometimes I send myself reminder e-mails, or I create calendar entries, or use a wiki. The wiki is good for certain things, especially in a collaborative environment, but for personal work it’s not quite right. And if my scattering of electronic notes is doing anything, it’s certainly not helping how I treat physical notes. The free KDE application, BasKet, is perfect for those jobs and more. Continue reading “Personal Wikiesque Note Taking Mind Mappish Killer KDE App: BasKet”

Corporate Wiki, a TWiki Announcement

After a lengthy post yesterday about TEC’s internal use of a corporate wiki, I read an announcement today from TWiki about the launch of its enterprise wiki service TWIKI.NET. TWiki is a venerable open source wiki system, with a huge quantity of interesting and useful plugin functionality. The company’s press release says

“TWIKI.NET will provide premium support to a tested, reliable and secure version of TWiki. “We’re adding a professional company to a proven software platform so Fortune 500 companies and organizations of all sizes can feel safe, supported and secure while also accessing the innovation and flexibility of the TWiki solution,” added Beckström.”

Looks like they’re taking one of the common open source business models in hand, providing services to ensure dependability, upgrades, security, features, etc. A few years ago wikis seemed to be the little booth in the corner at trade shows, without a huge amount of people paying attention to why these would be useful in an enterprise context. Persistence seems to be paying off as these wikis continue to mature and gain acceptance, and most seem to be growing from their open source seeds. The list includes SocialText, Atlassian’s Confluence, XWiki, DekiWiki, and a lot of others.

One other thing of note, TWIKI.NET has a page with brief reasons why companies use an enterprise wiki–lots of interesting reasons.

Wiki While You Work

The Globe and Mail published an article about using wiki applications in the workplace. While not a new notion, this is the first time I’ve seen it in a regular newspaper and not an IT business rag. A point the article touches on is the wiki’s security. I think wiki security may be one of the more misunderstood issues about using a wiki for work and an important differentiating factor in determining when to use an enterprise content or document management system (CMS/DMS) and when to use a wiki. In fact, I think it’s hard to beat a wiki if you need an application to capture and disseminate employee knowledge.

“One drawback is security. Much of the hype around wikis concerns their ability to place everyone from the receptionists to clients to chief executive officers on the same virtual playing field.”

The key phrase above is that it puts people “on the same virtual playing field.” Useful things take place when people are uniformly able to document their activities, collaborative or otherwise. Simplicity is a defining aspect of wiki applications–they make it incredibly simple to collaborate on developing, publishing, or otherwise contributing to company information, documents, in some cases products, etc. I’ll talk about an internal wiki only, as I realize that one open to clients as well may present a slightly different set of issues. Still, I’d argue that in most cases the somewhat loose security issue is more of a benefit than a drawback. Let me illustrate this with how the company I work for, uses one.

Some time ago, frustrated with the problems of repeatedly sending mass e-mails to everyone in our company, I set up an internal corporate wiki. A wiki is excellent for work that is in constant flux or must be accessible by everyone in the company.

  • communicate important news or announcements
  • inform about policies that must be adhered to
  • distribute documents
  • collaborate on work issues
  • capture and disseminate the day-to-day knowledge that employees develop

I think these things fail through e-mail but work with a wiki. I think most of these things are usually (though not always) too encumbered with hierarchy structures, metadata entry, and access controls to be the most effective for the types of things I mentioned above. Even when people save e-mail messages, they must make repeated archaeological expeditions through their e-mail histories. If announcements need to be referred to in the future, there’s no guarantee people will be able to find them in an inbox. Policies and problems that have been solved are likely to be forgotten if they’re not easily present and visible, as they are in a wiki. Ensuring that people always use the most up-to-date versions of documents means making them easily accessible and that is so nicely accomplished with a wiki. Using e-mail to collaborate on projects can become a nightmare of criss-crossing information, which often leaves people out of the loop. If people are in the habit of working with a wiki on all sorts of general day-to-day tasks, it becomes an automatic, company-wide storehouse of employee knowledge.

Using a wiki facilitates these activities. For example, at TEC, internally we use the fantastic, open source Wikka Wiki application. It’s simple enough that people can be productive with it after about five/ten minutes of instruction. It doesn’t confuse with over-sparkly and burdensome features. It’s fast–takes fractions of a second to access and edit in a web browser. It doesn’t require manipulating difficult access permissions. These are all important features because they make it at least on par, if not sometimes easier than sending an e-mail or accessing a DMS. If you want to change peoples’ work habits from constant e-mail use, then I think the alternative ought to be at least as easy and efficient or else offer something so incredibly good as to compel its use.

Before the wiki, people would forget what an important policy might be after six months. Now, even if forgotten, it can be easily found for reference. Before the wiki, frequently used documents were sometimes difficult to disseminate in their most up-to-date form. Now they’re updated, in short order, on their corresponding wiki page.

Before the wiki, information about projects that different groups in the company had to collaborate on, was spread across different people’s e-mails. There was the risk that someone wouldn’t get all the information s/he needed. Now it gets collaboratively updated on pages that anyone within the company can see, which has the added benefit that sometimes people without an obvious, direct connection to the project can discover it and contribute or use it in positive ways that nobody would have imagined previously.

I don’t think a wiki replaces a DMS or vice versa. A DMS might sound like it is designed to capture and better enable such collaboration but I don’t believe that is necessarily its strongest point. I think a DMS is probably better-suited to developing documents that require tight version control, traditional hierarchy structures, and cannot necessarily be developed as content within web pages. A DMS might be more useful for archival purposes or for documents that are sensitive and absolutely must have special access controls. But a DMS tends to be more cumbersome in the security and access area, and thus loses utility in the area of capturing and disseminating employee knowledge.

Spreading the wiki. In the past, people sometimes would tell me about some sort of project they needed to work on or information they wanted to store in an easily usable way. I’d recommend they try the wiki to facilitate it. So they’d ask of course, “what’s that?” and I’d spend five/ten minutes explaining it. The interesting thing is that then they go off and explain it to other people on their teams, then the different teams work on things with the wiki, word-of-mouth makes its use spread. I’m sure this isn’t a 100% effective way to promote its use but I was pleasantly surprised that after implementing the wiki and announcing it, people started pushing its use of their own accord.

A system that requires a lot of security, perhaps needing more of a top-down approach, wouldn’t permit this type of usage to happen. Setting up access controls, accounts, and maybe designing structures for how a company uses its systems of collaboration and knowledge sharing may be time-consuming and ultimately not do the job for which they’re intended. On the other hand, a wiki method allows this to self-organize. The chaos of knowledge that frequently gets developed and lost throughout a work place gains a facility in which to reside and that attracts use.