Yes, a new study of lost productivity is out. No, it’s not about the number of people who spend time on Facebook or Woot.com at work. Thomson Reuters quantifies the problem as $1.5 trillion in lost productivity, with some interesting numbers around how many professionals are getting 50+ emails per day (I think the answer is “everyone I know”.)  Cleverly enough, I’ve seen more than one legacy commercial search technology company come up with the answer: Search!

Well. I think that a big part of that wasted $1.5 trillion is time wasted searching. With all the different things coming at us, what we need is a search box to go right in the middle of them? More time spent finding, less time spent searching, I’d warrant, is part of the answer. Probably not just a matter of promoting a commercial proprietary black-box search engine as the way to save time and money (or get a piece of that $1.5 trillion, too).

Here’s a different way to think about the problem. Say you’re a salesperson, and you want to optimize the information you have access to, by drawing together your structured and unstructured data, a 360 degree view of information. Where’s that information? More often than not, in your Salesforce.com CRM system. Got a proposal for a new prospect that resembles some deals you once did? You want to look at your notes from other contacts and customers. And so the folks at Salesforce.com are working on making that easier: they build search into the application, using Lucene/Solr open source search. (A word from our sponsor: come hear Salesforce.com’s Bill Press talk at Lucene Revolution about their use of Lucene/Solr search).

If you have important information that you need your end users and customers to find more quickly, chances are that you want search to function as part of work they are already doing. You want to integrate search into the applications that run your business, as an integrated, streamlined portion of the work flow. The better they integrate, the more time your end users and customers spend _not_ searching. The great virtue of Solr/Lucene open source search is how well suited it is to building search applications to fit your business.  Charlie Hull, also speaking at Lucene Revolution, makes the case nicely in his  interview on Steve Arnold’s blog:

“The key tasks for any search solution are indexing the original data, providing search results and providing management tools. All of these will require custom development work in most cases, even with a closed source technology. So why pay license fees on top? The other thing to remember is anything could happen to the closed source technology – it could be bought up by another company, stuck on a shelf and you could be forced to ‘upgrade’ to something else, or a vital feature or supported platform could be discontinued…there’s too much risk. With open source you get the code, forever, to do what you want with.”

Open is good, search that helps your customers do less searching is better. How does Solr/Lucene Open Source manage to do both? Tune into the second interview I did with Steve Arnold on Beyond Search — the podcast is here.