Here’s the steps I took:

1. Download and “install” (aka unzip) Apache Solr 3.3.0
2. Launch Solr (cd example; java -jar start.jar)
3. Index files

That’s it.  Here’s the indexing script I used:

require 'net/http'
 
@dir = Dir.new("/Users/erikhatcher/apache-solr-3.3.0/docs")
 
@url = URI.parse("http://localhost:8983/solr")
@connection = Net::HTTP.new(@url.host, @url.port)
 
def index(filename)
@connection.get(@url.path + "/update/extract?stream.file=#{filename}&literal.id=#{filename}")
end
 
def commit
@connection.get(@url.path + "/update?commit=true")
end
 
@dir.each {|name|
  f = "#{@dir.path}/#{name}"
  if File.file?(f)
    puts "Indexing #{f}..."
    index(f)
  end
}
 
puts "Committing..."
commit
 
puts "Done!"

To make it look prettier, only a little dabbling with the templates is needed – add your company logo, customize the colors. And a change to the example (/browse handler) configuration to facet on content_type will allow you to easily search just within documents of specific types through the included UI.  The example code above indexed the docs that ship with Apache Solr 3.3.0; just change the path to a directory of yours to index your own content.

Before I dig in too much, though, a special thanks to all our customers at Lucid Imagination as well as to my coworkers.  I’m coming up on 15 years out in the “real world” and I can honestly say I’ve never enjoyed what I do as much as I do here and that even accounts for the normal rough patches one goes through in any job.  As an engineer, there are few things as cool as getting to work with customers who are not only using, but pushing your work/project/product on a daily basis to do new and interesting things (I think this is a direct result of the project being Open Source, which I believe has an inherently lower cost of experimentation).  I’ve been fortunate enough to meet and talk with many people doing all kinds of things with Lucene and Solr ranging from the “mundane” of basic keyword search to those building next generation search capabilities at incredible scale.  Through it all, I’m constantly amazed at the flexibility and efficiency of Lucene and Solr.  For instance, I’ve been working with one customer now whose Solr-based solution (for the exact same content) will use ~50% less hardware and will have an index that is 1/6 the size of their FAST index all while saving them major dinero.

Speaking of Lucid, one of the highlights of the year for us that relates directly to Lucene and Solr is the launch of our enterprise version: LucidWorks Enterprise.   I like to think of it as Apache Solr with a whole lot of Lucid expertise on how to use Solr baked in and topped off with other features and functionality to make building search applications easier.

OK, time to move on to the open source projects…

1. Without a doubt, the biggest news of the year is the merging of the Lucene and Solr code base as well as the “graduation” of several subprojects to Apache Soft. Foundation Top Level Projects (TLP).  The graduating projects are Tika, Nutch, and Mahout.  We also spun Lucy (a C port) to the Incubator, where it is working on it’s own community.  These moves were primarily done to focus the project management on single code base, but they also demonstrate the project has reached a level of maturity at the ASF.  The move also has the side benefit of bringing each project higher visibility.
2. I’m particularly excited about the addition of OpenNLP to the Apache umbrella.  OpenNLP is a nice open source Java project for natural language processing that has lived at Source Forge for quite some time.  I would expect development to grow quite a bit under the ASF community based model.  Also, integrating OpenNLP with Solr and Lucene is pretty easy to do.  I would be remiss if I didn’t also give a nod to the addition of the ManifoldCF project to the ASF.  ManifoldCF will help unlock content in Sharepoint, Documentum and other repositories for users of Lucene and Solr.
3. Lucene’s trunk code base now implements our “Flex APIs”, which should allow users to have near total control over what goes in the index as well as alternate compression techniques, different scoring models, etc.  See Michael McCandless’ excellent talk at Lucene Revolution for more details.
4. With all the location aware devices and capabilities on the market, geo-spatial search is a hot topic and Lucene and Solr have been adding quite a bit of capabilities in this regard with the ability to filter, boost and sort results based on location information in documents.  See Solr’s Spatial Search Wiki page for more info as well as several of my past blog posts.
5. Of course, everyone was a buzz about the cloud this year.  For Solr, this translates into greater efforts to make Solr easier to scale to very large installations (100s to 1000s of nodes and billions and billions of documents) via the Solr Cloud project that Yonik Seeley and Mark Miller have been spearheading.
6. On the user side, one of the biggest pieces of buzz this year related to Lucene was the migration of Twitter search to Lucene.  At 1 billion queries per day and 50 million posts per day (all indexed and searchable in near real time), Twitter’s search system certainly has it’s work cut out for itself.  However, as Michael Busch outlined at Lucene Revolution, Apache Lucene was up to the task!  Naturally, there were lots of other companies that migrated to Solr and Lucene as well.  Have you shared your use case?

Well, I’ve no doubt missed a bunch of other things, but those items, to me, are some of the bigger highlights.  Looking forward, there are some other exciting things coming to Lucene and Solr.  In particular, I’m working on adding language identification, related searches and point in polygon filtering to Solr.  I would also expect we will release Lucene/Solr 3.1 fairly soon, too, but you can’t pin me down on a date just yet.

Here’s hoping you all have a Happy Holidays and a Happy New Year!

Tika provides a uniform Java API for all of the supported document formats to make life easier for the user.  Additionally, Tika provides functionality for detecting document type and content language.

In my earlier article, Content Extraction with Tika, I looked into using Tika either with Solr or in standalone mode. In this post I will go though some of the aspects involved when implementing support for new document formats. I will also provide a couple of example parsers and a full maven project to get you up to speed quickly.

Extension mechanisms

The basic principle of adding support for more document formats in Tika is very simple. All you need to do is write a Java class that implements the Tika Parser interface and let Tika know about your extension. If you implemented a parser for one of the >900 file formats Tika knows by filename extension or one of the ~300 formats that Tika can recognize from the number of file content bytes this is all you need to do.

There are at least three different ways to let Tika know about the new parser. The first way (and the most flexible one) is to wire it up with java code. If you wanted to use Tika AutodetectParser you’d call setParsers method on AutodetectParser with the parser of your choice. By wiring up things with java code you could also customize the detection logic easily too just by calling the setDetector method.

  public static void registerParser(AutoDetectParser autodetectParser,
      Parser parser) {
    for (MediaType type : parser.getSupportedTypes(new ParseContext())) {
      autodetectParser.getParsers().put(type.toString(), parser);
    }
  }

The next way to customize the set of available parsers is to use an external XML file and construct a TikaConfig object with that configuration file.

The last of the three methods explained here is to use the new mechanism that was added in version 0.7 of Tika: the standard Java ServiceProvider API. In practice this means that as a provider of a new parsing functionality you’d need to list your parser implementation classes in the file META-INF/services/org.apache.tika.Parser inside the .jar that you provide to the implementation.


com.lucid.tika.MyTXTParser
com.lucid.tika.VCardParser


Extending the capabilities of Tika by using the ServiceProvider API is very straightforward and simple. There are however a couple of details you should pay attention to when using this mechanism.

If you need to replace a Tika provided parser implementation with your custom implementation you need to make sure that your .jar file is loaded after the tika-parsers.jar file. This is because in the current implementation the last parser registered for certain mime type is used to parse content for that mime type.

To support completely new document types (that Tika knows nothing about) you need to customize the detection process of AutoDetectParser manually. This is because there is no similar mechanism to extend the detection step as there is for adding new parsers. One way to do this is to use CompositeDetector to add your “overlay” detections to be done and trust for the default Detector for detecting the other types.

  public static void setOverlayDetector(Detector overlay, AutoDetectParser parser) {
    ArrayList detectors = new ArrayList();
    detectors.add(parser.getDetector());
    detectors.add(overlay);
    parser.setDetector(new CompositeDetector(detectors));
  }

In this blog post I have demonstrated some ways to extend Tika parsing and detection capabilities if needed in your custom environment. Process-wise, the best possible way to add new capabilities to Tika is to contribute your new parser integrations and enhancements back to the Tika project. This way the community as whole will benefit from the results.

Running the provided example

• Download project
• Compile project
  mvn clean install
• Copy dependencies to directory target/dependencies
  mvn dependency:copy-dependencies
• Execute the default TikaGUI with our additions (“enhanced” .txt parser, vCard parser)
  
java -cp target/dependency/tika-app-0.7.jar:target/extending-tika-post-1.0-SNAPSHOT.jar:target/dependency/ical4j-vcard-0.9.2.jar:target/dependency/ical4j-1.0-rc2.jar:target/dependency/commons-codec-1.3.jar:target/dependency/commons-lang-2.4.jar org.apache.tika.gui.TikaGUI


As for the talks, I gave the Keynote on using open source tools like Apache Solr and Mahout to deliver intelligent applications (slides — really should be a PPT so you can see the animations) on Monday first thing in the morning and I felt it went pretty well, but I’ll let others be the judge (videos should be online soon).  The rest of the day, I spent going in and out of the various tracks.  The Lucene track was very well done, with good talks by: Uwe Schindler and Simon Willnauer on the State of Lucene, Robert Muir on Finite State queries in Lucene; Michael Busch on Real Time Search at Twitter, Jukka Zitting on Tika and Andrzej Bialecki on Nutch. See Berlinbuzzwords: Links To Slides for all the slides (not all are available just yet).

I also went to a variety of the Hadoop and NoSQL talks.  Lots of people in the NoSQL talks making pitches on why their approach is best, which is very helpful in determining what tool to use at the appropriate time.  I still, however, can’t shake the feeling that one could take the new Solr Cloud stuff, a dead simple schema (id and one or two simple fields), and have a large scale distributed key-value storage that overcomes almost all of the limitations of many of the NoSQL technologies (ad-hoc queries, range queries, search within the values, extendability) with minimal overhead of indexing (which can be greatly reduced by using either literals or very simple analysis).  Not only that, Lucene/Solr already is “document-centric” and I’ve seen it scale to billions of documents with high availability and high QPS and that was using “real” documents (i.e. articles, etc.), not simple key-value pairs, so I can’t help but feel like simple key-value pairs would be even faster and more scalable.  In other words, Lucene isn’t just for text search.  Naturally, this is just a thought at this point, I haven’t tried testing it just yet. Also, once the new real time stuff is in Lucene, I think it will be even faster.

At any rate, the best thing about the conference was the fact that it shows the eagerness for new solutions to large scale solutions that cost less money than the sturdy old database.

Again, congrats to Isabel and team for a well executed conference in a great city and at a great venue.  If you are interested in more on the Lucene portion of the conference, make sure you come visit us in Boston for Lucene Revolution!

One of the things I really like about this agenda is it is a great mix of basics, use cases from all over the search map (CMS, news, social media, advertising), business decisions (see last list and next list) and advanced topics (NLP, collab filtering, machine learning, advanced visualization, multilingual).  Moreover, the content, even though it is centered in Lucene, goes well beyond just being about Lucene and is really about search, in all of it’s power and glory.  It’s about real users with real needs getting real problems solved using the Lucene ecosystem.  Oh, and by the way, those users are doing it at scale!  Big scale.

That’s powerful stuff, because, in case you hadn’t noticed (shh, it’s our little secret) there is a revolution going on in search.  (Funny how that line coincides with Lucid’s frontman, Eric Gries, giving a talk titled “The Search Revolution”)

Are you a part of the revolution?  See you in Prague in mid-May.

In the end, though, what does all this mean for the users of the Lucene ecosystem?  On one hand, some of the move is just shuffling around of domain names, mailing lists and SVN source trees, but on the other hand, the moves are symbolic and represent a project reaching a level of maturity and self determination, not to mention critical mass and brand awareness.  Thus, in my mind, all of these moves are good things for Lucene as well as the associated projects that are spinning out.  As far as the actual code, I think users will still see the same high quality contributions and products coming out of Apache (aside: Lucid Imagination will still be business as usual in regards to these moves) as well as much more focus within the Project Management Committee (PMC) on the specific project.

Which brings me to a bit more on my view of the merge of Lucene and Solr.  I think we are already seeing the fruits of the merge for both Lucene and Solr (I know my open source life is easier already).  For instance, much of the analyzer code is going to being combined from Solr and Lucene to provide a single coherent analyzer library.  This is great news for people who have been using Lucene and pulling in Solr analyzers and is good for Solr users because it now has many more people keeping an eye on Solr’s analyzers as well as new Lucene analyzers showing up sooner (things like the WordDelimiterFilter, etc.)  Another example is the spatial work that we’ve been working pretty heavily on (see SOLR-773, SOLR-1568 and LUCENE-2350).  With the combination of the two development projects, it is now much easier for us to make sure there is a single, integrated way of delivering spatial search across both the Java API and the Solr REST-like API.

Moreover, in the short run, existing Lucene and Solr users should notice no difference in terms of the products, user communities and the like.  In the long run, it should make for less repeated code, faster integration, more test coverage and a larger, cohesive development team as well as more of Solr’s capabilities available in pure library form as well as many of Lucene’s cutting edge capabilities appearing sooner in Solr (flexible indexing and scoring, etc.)

Wrapping up, congrats to Lucene and all of the new top level projects!

The Apache Lucene Connector Framework project has officially entered incubation.  LCF, for short, is going to be a framework for connecting to content repositories like Sharepoint, Documentum, etc. and will make it easy to hook into Lucene, Solr, Nutch, Mahout, Tika, while, of course, remaining agnostic of the final destination of the data.  See the Connectors website and the original proposal for more info.  Help wanted!

Long Version

Background

A while back, MetaCarta, a spatial search company, approached us about open sourcing their internally developed Connector Framework at the Apache Software Foundation.  After several discussions and a whole bunch of legwork getting a proposal together, the LCF is now officially launched in the Apache Incubator!  We’ve already got a great roster of committers lined up and are working to incorporate the software grant from MetaCarta, from which we can build out a first release, so stay tuned!  Lucid Imagination, of course, is a big supporter of this project and we look forward to it’s success!

What is a Connector Framework?

To quote the proposal:

[The Lucene] Connector Framework is an extendible [sic] incremental crawler, which uses a database to manage configuration and crawl history, and provides reasonably high performance in accessing content in multiple repositories for the main purpose of search engine indexing. Connector Framework also establishes a repository-specific security model which can be used to limit search user access to repository content based on a user’s identity. Connector Framework also includes existing connectors and authorities for:

• File system
• Windows shares
• JDBC-supported databases
• RSS feeds
• General websites
• LiveLink [from OpenText]
• Documentum [from EMC]
• SharePoint [from Microsoft]
• Meridio [from Meridio]
• Memex [from Memex]
• FileNet [from IBM]

There are two pieces in particular to highlight in the quote.  First of all, it’s an extensible framework, meaning new connectors can be added without the need for application developers writing “one-off” code just for that connector.  For anyone who’s lived that pain, you know first hand what I mean.  In fact, I’ve already heard from others who are thinking of contributing their connectors for other data stores as well!  Second, the framework accounts for repository specific security.  In corporate environments, this is vital to making sure that the right people, and only the right people, have access to the right information at the right time.

Why is this important?

Many, many search engines, not too mention many other applications, have either rolled their own connectors or bought a company that provides them.  Connectors, in some situations, are the cost of entry into  certain markets, but are rarely the feature that seals the deal.  By making these open source, we can all share the cost of maintaining it while increasing the quality of a piece of software well beyond what any one company can achieve.  Beyond that, we hope the repository companies will also step up and contribute (some are already quite open), as making it easier to access these repositories will no doubt lead to more applications, which of course should mean more sales for said companies.

How can you contribute?

For starters, subscribe to the mailing lists.  Then check out the How To Contribute page on the Wiki.  Beyond that, chip in with your connector use cases on the mailing lists and be a part of the community.

What’s next?

First off, the community will have to process the software grant from MetaCarta and then commit the code to LCF’s Subversion repository.  From there, we’ll do just like any Apache project does and look to build out not only the code, but also the community, all on the path to graduating from the Incubator and taking our place as a full-fledged Lucene subproject.  Keep your eyes here and on the mailing lists and websites for more information in the future!

First and foremost, it should be obvious to even the most casual observer that the Apache Lucene communities are thriving.  Not only is it a great time to be involved in open source, it’s a great time to be involved in Lucene.  Both as a committer and as an employee of Lucid Imagination, I’m continuously amazed at the vibe produced by the people using the Lucene suite of libraries, tools and applications.  People are routinely solving both large scale and really hard problems using the Lucene ecosystem and they are doing it on time and on budget.  For instance, this year alone, I’ve seen companies and individuals using Lucene and Solr to provide search in production environments with document counts ranging from the few tens of thousands all the way up to 5-10 billion plus and query rates that barely register a blip to 1000+ QPS.  I’ve also seen many people using Lucene to power recommendation engines, content management systems, machine learning/NLP applications and log analysis tools.

Much to my initial surprise, the number one reason I hear for why they chose Lucene: flexibility. (I thought it would be the fact that they are free to use, but that is just icing on the proverbial cake, I guess)  Namely, Lucene gives them the flexibility to build what they want or simply to use it out of the box.  It gives them the flexibility to bring in other tools from other open source projects or other commercial vendors, all without compromising speed or scale.

With that in mind, I thought I would give some highlights of both the top level project (TLP — http://lucene.apache.org — the ASF project that “houses” all of the Lucene related subprojects) that is Lucene as well as the individual projects.  (I’m not involved in all them, so please correct me if I’m wrong!)

Lucene TLP

Whew!  It’s been a busy year for the Lucene TLP.  We started the Open Relevance Project (ORP), added PyLucene (a Python port of Lucene) and successfully graduated a .NET version of Lucene from ASF incubation, not to mention the fact that the Lucene PMC is responsible for overseeing the release of all the various bits and bytes for each and every subproject (which is a lot of releases!)  We also, for the first time ever, organized two days of Lucene related talks at ApacheCon US plus two days of training and meetups. (In the past, organization was always handled by the ASF Conference Committee).

In looking ahead for the TLP, I see a continued focus on providing quality software across all the projects.  Additionally, keep your ears open, as there is a new sub project brewing that I think will really make it even easier for people to deploy Lucene based solutions.  Finally, just as Lucene gave birth to Apache Hadoop and is happy to see it doing so well, there is growing talk that Lucene will look to see Apache Mahout off as it’s own TLP.  Of course, none of that is in stone yet!

For those looking for more on the big picture that is Lucene, see my talk at ApacheCon US for more details on the ecosystem.  Not sure why the slides aren’t there, so I put them here.

Lucene Java

Lucene Java (i.e. what everyone knows as “Lucene”) continues to not only provide a rock solid indexing and search API, it continues to push forward with new capabilities.  In 2009, Lucene did 4 releases (2.4.1, 2.9.0, 2.9.1 and 3.0.0).  2.9.0 was probably the most interesting, as it significantly improved performance in a number of areas, while 3.0.0 removed all of the deprecated APIs and finally, officially, dropped support for Java JDK 1.4.  I’ll leave it to the reader to go look up all the features and changes as they are numerous.

Looking ahead, the phrase of the year appears to be “flexible indexing”.  Flex Indexing looks to make it even easier for people to custom craft what is in their index, whether that is rich token attributes (aka “typed” payloads), alternative scoring models (like Okapi BM25) or a bare bones index designed for blazing fast speed.

Solr

With Lucene as the engine, Solr has evolved into quite the car.   Building on all of the goodness that is Lucene, Solr, in 2009, released version 1.4 with a whole slew of new features, faster implementations and bug fixes.  Highlights for 1.4 include: improved filtering and faceting performance, support for clustering, rich document indexing via Apache Tika, multi-select faceting (see Lucid’s very own search.lucidimagination.com for a demo), many new Query capabilities and a whole bevy of new Components (Terms, Term Vectors, Auto-suggest, deduplication and Statistics on result sets) that truly make Solr an incredible search platform.

Looking ahead, Solr 1.5 (2.0?) is already in the works and looks to have even more functionality.  For instance, a lot of work is underway to integrate Apache ZooKeeper and other distributed capabilities, which will help make deploying Solr at scale even easier.  Meanwhile, many are hard at work adding “field collapsing” (search result grouping/deduplication) and spatial (local/geo) search.

Mahout

It’s been a very exciting year (in my completely biased opinion!) for Mahout, the scalable machine learning project under Lucene.  In 2009, Mahout shepherded through it’s very first release (0.1) built on the strength of a few dedicated volunteers working to add capabilities for clustering, categorization and collaborative filtering.  Next came 0.2 with many new features (frequent patternset mining, Latent Dirichlet Allocation, Random Decision Forests, new recommendation capabilities) API and performance improvements and a growing list of people who stopped lurking and stepped up to help out.  Towards the end of the year, Mahout is already reaching a list volume that I find difficult to keep up with if I miss a day or two.  For starters, we have taken on the task of integrating/transforming the Colt matrix library for our needs.  We are also working on adding truly large scale recommendation capabilities plus adding in a Support Vector Machine implementation and Logistic Regression.  Not only that, but the mahout-user@lucene.apache.org mailing list continues to be a valuable resources for people seeking practical advice on deploying machine learning in production environments regardless of the choice of Mahout or not.

In 2010, I suspect Mahout will become it’s own TLP, with several sub projects roughly divided as: core/utilities, recommendations (Taste) and NLP.  Of course, until it happens, this is just speculation.  I also think Mahout will look to finalize its APIs for a 1.0 release.

Nutch

In 2009, Apache Nutch released the long awaited version 1.0.  This release contained many new indexing and scoring capabilities, as well as integration with Solr.  The community continues to be focused on providing large scale crawling and search capabilities by leveraging Apache Hadoop and Lucene/Solr.  Currently, the community is actively looking at modularizing Nutch to allow it to more easily plug in other ecosystem components like Tika and Solr while focusing on the primary task of obtaining and managing content via crawling.

Tika

Apache Tika is a content extraction framework for “rich” documents like Adobe PDF and Microsoft Office.  In 2009, Tika released versions 0.3, 0.4 and 0.5, all with incremental improvements designed to make it more stable and easier to use.  Each release also seemed to carry with it a new list of supported file formats as more and more people join the project to lend a hand.

Coming up, I suspect Tika will look to finalize a 1.0 release at some point in 2009 as well as focus in on standardizing, if such a thing is possible, on the metadata artifacts produced by Tika.

Open Relevance Project

The ORP is a project that has been in my brain for several years now and finally got off the ground in 2009.  The goal of ORP is to provide corpora, queries, judgments and other tools to help search and machine learning projects discuss relevance in a completely open way.  While the project is really young, it is slowly but surely building up steam by adding some basic tools and collections thanks to the hard work of several individuals.  In 2010, look for ORP to build out a more complete toolset while attracting more users and contributors.  It will also be vital for the ORP to create its own versioned corpora for download (free!) so that all experiments can be reliably reproduced.

Droids

Droids is a standalone crawler framework currently in incubation at the ASF.  Development was active in 2009, but has not yet had a release.  For now, it is a Spring based framework that allows one to quickly build out agents that can go and crawl and process content.

Lucene.NET

In 2009, Lucene.NET graduated (some infrastructure changes still need to happen) from ASF incubation and became a full-fledged member of the Lucene ecosystem.  While I’m not closely involved with Lucene.NET, the community continues to provide value to those looking for a solid search library in .NET.  Since the project is mostly autogenerated from the Java sources, the .NET version has tracked the Lucene Java releases fairly closely.

Looking forward, I expect the .NET version will strive to maintain a lockstep march with Lucene releases.

PyLucene

Similar to .NET, PyLucene produces a Python port of Lucene Java.  In 2009, PyLucene was formerly welcomed into the Lucene fold via a software donation by Andi Vajda.  It continues to produce releases of PyLucene in lockstep with Lucene Java.

Lucy

Lucy is a “loose” ‘C’ port of Lucene.  Lucy finally got off the ground in 2009 and is steadily working on building out a core search library that provides fast search capabilities for languages like Perl, C and Ruby.

For 2010, look for Lucy to continue to grow its community while adding capabilities.

Moving Forward

While the past is, of course, no prediction of the future, I think it’s safe to say Lucene is looking to continue to provide significant capabilities and value to both well established and new communities alike.  With open source, you never know where the next good idea is coming from, so make sure to stay tuned both here and on the mailing lists for more insight and more cool new capabilities.

Happy Holidays and here’s to an Open Source 2010!

Thanks to everyone who participated.

During the conference, there were several talks on Lucene, Solr,  Mahout and Droids.  Slides are available at:

• Introducing Mahout
• Lucene/Solr Case Studies
• Advanced Indexing (slides are missing, but should be up sometime soon)
• Apache Droids
• Best of Breed: HTTP Server, Forrest, Solr and Droids
• Apache Solr: A Case Study

Additionally, for the first time,  we had a Lucene Meetup (sponsored by Lucid).  I’d estimate there were around 60 people there and we had some good discussions on Tika, Lucene, Solr and Mahout.    Also, Uwe Schindler presented his new TrieRange Query capabilities.  Slides are available here.

Finally, my favorite part of the conference is always the individual conversations with people using the Lucene ecosystem to solve their problems.  Each year, it seems, people have more and more new ideas about how to use Lucene and Solr, many of which go beyond “traditional” search.  Over the coming months, I think you will see more and more of Lucid highlighting all Lucene ecosystem users through our Podcasts, Wiki Showcase and other features coming soon.  So, if you think you’ve got something cool using the Lucene ecosystem, add a comment below or drop us a line at feedback@lucidimagination.com

UPDATE: 4/9/09:  Uri has sent me his slides and they can be downloaded at: Apache Solr Case Study