• Products
    • Overview
    • LucidWorks Search Platform
      • Features and Benefits
      • Technical Overview
      • Only with LucidWorks
      • LucidWorks and Solr
      • White Papers
      • LucidWorks Enterprise
      • LucidWorks Cloud
    • Certified Distributions
      • Certified Solr
      • Certified Lucene
    • Apache Releases
      • Apache Solr
      • Apache Lucene
  • Support & Services
    • Overview
    • Support
    • Training
    • Solr/Lucene Certification
    • ExpertLink Advisory
    • Consulting
    • Partners
    • Subscriptions
  • Why Lucid?
    • Why Lucid?
    • Technology
    • Technical Leadership
    • Who uses Lucene/Solr?
      • What customers are saying
    • Case Studies
    • Whitepapers
    • Demos
    • Webinars
  • Blog
  • DevZone
    • DevZone Overview
    • Forums (LWE)
    • Videos & Podcasts
      • How To's
      • Screencasts
      • Podcasts
      • Conference Videos
    • Technical Articles
      • Whitepapers
    • Reference Materials
      • Documentation
      • Solr Reference Guide
      • Solr & LucidWorks Matrix
      • Tutorials
    • Events
      • Conferences
      • Meet Ups
    • Code & Test
  • Downloads
  • About Us
    • Management
    • Careers
    • News
      • Media Coverage
      • Press Releases
    • Contact Us
Sign Up or Log In
Home . Blog

July 20, 2010

[UPDATE] Spatial Search in Apache Lucene and Solr

Posted by Grant Ingersoll

One of the most frequent things I get asked is “what is the state of spatial in Lucene and Solr?”  So here is my answer as of today:

  1. I just committed SOLR-1568 the other day, which adds automatic filter generation to the various point based Field Types in Solr.  It also has some small refactoring in the underlying Lucene code.  Furthermore, it adds a new LatLonType which can be used to represent latitude/longitude pairs seamlessly.  See http://wiki.apache.org/solr/SpatialSearch for the full details on Solr spatial.  Note, this is only available on trunk.  Volunteers to backport to 3.x would be most welcome.
  2. As part of SOLR-1568, it became increasingly clear to me that the Cartesian Tier stuff in Lucene spatial simply does not work as intended for many, many things.  In my review and attempt at fixing the code, it became more than apparent that it only really works for the Western Hemisphere above the equator, i.e. the United States.  It may also work in the Eastern Hemisphere above the equator, too.  The reason it only really works above the equator is due to a miscalculation in the SinusoidalProjector.  See LUCENE-2519.  It also does not handle edge cases well at all, such as at the poles or the Prime/Anit Meridians, so if you have that case, then don’t bother.  I didn’t fix the SinusoidalProjector because it turned into a very tangled web of broken unit tests.  In discussions with other developers, we decided the whole tier system (and much of Lucene’s spatial should be deprecated/replaced).

I believe trunk is now in pretty decent shape for spatial search for applications that need:

  1. Sorting by distance
  2. Boosting by distance
  3. Range-query (using Numeric Fields) based bounding box calculations, which should be sufficient for most people
  4. Geohash based calculations

Trunk does not yet have the ability to:

  1. add “pseudo” fields to the result set, so it is not possible to include the distance in the result set just like other stored fields
  2. A tier/tile/grid based approach to filtering.  These approaches are especially helpful in highly dense areas as they can significantly reduce the number of terms that need to be enumerated
  3. Faceting by functions, which can be useful for putting distances into buckets, as in something like: walking, biking, driving

For a list of all the related Solr/Lucene spatial issues, see SOLR-773.  Again, see http://wiki.apache.org/solr/SpatialSearch for a full accounting of what is in Solr and how to use it.

In summary, I think trunk is in pretty decent shape for spatial, as far as Solr is concerned.  Pure Lucene users will seem some upheaval in the coming months, but it is for the better.  Testers are needed and patches are welcome.  And, while the tier stuff feels like a step backward, I think it is clear to me that we have several committers along with many contributors who are very interested in seeing spatial support live and prosper.

  • Share this:
  • Email
  • Facebook
  • Digg
  • Share
  • Print
  • Reddit
  • StumbleUpon

Category: Lucene, Solr, spatial search

4 Responses to “[UPDATE] Spatial Search in Apache Lucene and Solr”

  1. [...] search in Lucene and Solr, as things appear to be looking up for this trunk.  According to a recent Lucid Imagination post on the subject, the leader in Lucene/Solr technologies claimed this trunk adds automatic filter generation to the [...]

    July 22, 2010 18:45 — Solr Filtering Steps Up : Lucene Blog

  2. Is there support in the trunk to allow storage of polygons and/or querying for lat/lon indexed documents based on polygons?

    August 10, 2010 12:16 — Jonathan Suchland

  3. Hi Jonathan,

    Unfortunately, at the moment, there is not. Patches welcome, though!

    August 11, 2010 05:42 — Grant Ingersoll

  4. [...] Status of Solr and spatial searches Before reading on, you should be aware of some limitations with this plugin, More here: http://ianibbo.me/?p=127 http://www.lucidimagination.com/blog/2010/07/20/update-spatial-search-in-apache-lucene-and-solr/ [...]

    October 27, 2010 02:25 — Geo search with spatial-solr-plugin — Jayway Team Blog

Leave a Reply

Go to Blog Front Page

  • Recent Posts

    • Lucene Revolution 2012 – Call for Participation now open!
    • SolrCloud is Coming (and looking to mix in even more ‘NoSQL’)
    • Our Solr Reference Guide updated for v3.5
    • Enhancing Discovery with Solr and Mahout – session slides now available!
    • Solr and LucidWorks feature matrix available
    • LucidWorks Enterprise latest version 2.0.1 released!
    • Why Not AND, OR, And NOT?
    • Options to tune document’s relevance in Solr
    • Dallas JavaMUG December 14th 2011
    • Apache Mahout user meeting – session slides and videos are now available!
  • Archives

    • January 2012
    • December 2011
    • November 2011
    • October 2011
    • September 2011
    • August 2011
  • Tags

    acts_as_solr apache Apache Mahout best practices chump code4lib dismax drupal enterprise search Erik Hatcher field collapsing function query Grant Ingersoll hoss image isfdb local params Lucene lucene revolution LucidGaze lucid imagination Mahout Marc Krellenstein Mark Miller nested queries nutch Open Source Open Source Search qparser query parser queryparser Rails release result grouping Richmond Ruby schema design sint Solr solr 3.1 solr 4.0 solr cloud sortable Tika VA
  • Contact Us
  • About Lucid Imagination
  • Help & Support
  • Training
  • Privacy Policy
  • Legal Terms of Use
  • Copyrights and Disclaimers
  • Log in

Apache Solr, Solr, Apache Lucene, Lucene and their logos are trademarks of the Apache Software Foundation.

© 2011 Lucid Imagination. All Right reserved.

loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.