Apache Lucene (the Lucene top level project, not Lucene the Java search API.  I know,  it’s confusing sometimes) has once again proved to be a fertile area for innovation (having already given birth to Apache Hadoop a few years back), as it once again has given birth, this time to three new Apache Top Level Projects (just approved by the Board at Apache): Apache Mahout, Apache Nutch and Apache Tika (never mind the URLs, they will be changing soon).  While none of these projects look alike, they all have a strong foundation built in the Lucene community.  Combine this with the recent merge of Lucene and Solr development lists (more on this later) and Lucene has been busy; and that doesn’t even mention all the really cool stuff baking in the source tree right now (spatial, flexible indexing/scoring, some new analyzers and a variety of other cool things — see Lucene’s CHANGES and Solr’s CHANGES).

In the end, though, what does all this mean for the users of the Lucene ecosystem?  On one hand, some of the move is just shuffling around of domain names, mailing lists and SVN source trees, but on the other hand, the moves are symbolic and represent a project reaching a level of maturity and self determination, not to mention critical mass and brand awareness.  Thus, in my mind, all of these moves are good things for Lucene as well as the associated projects that are spinning out.  As far as the actual code, I think users will still see the same high quality contributions and products coming out of Apache (aside: Lucid Imagination will still be business as usual in regards to these moves) as well as much more focus within the Project Management Committee (PMC) on the specific project.

Which brings me to a bit more on my view of the merge of Lucene and Solr.  I think we are already seeing the fruits of the merge for both Lucene and Solr (I know my open source life is easier already).  For instance, much of the analyzer code is going to being combined from Solr and Lucene to provide a single coherent analyzer library.  This is great news for people who have been using Lucene and pulling in Solr analyzers and is good for Solr users because it now has many more people keeping an eye on Solr’s analyzers as well as new Lucene analyzers showing up sooner (things like the WordDelimiterFilter, etc.)  Another example is the spatial work that we’ve been working pretty heavily on (see SOLR-773, SOLR-1568 and LUCENE-2350).  With the combination of the two development projects, it is now much easier for us to make sure there is a single, integrated way of delivering spatial search across both the Java API and the Solr REST-like API.

Moreover, in the short run, existing Lucene and Solr users should notice no difference in terms of the products, user communities and the like.  In the long run, it should make for less repeated code, faster integration, more test coverage and a larger, cohesive development team as well as more of Solr’s capabilities available in pure library form as well as many of Lucene’s cutting edge capabilities appearing sooner in Solr (flexible indexing and scoring, etc.)

Wrapping up, congrats to Lucene and all of the new top level projects!