• Products
    • Overview
    • LucidWorks Search Platform
      • Features and Benefits
      • Technical Overview
      • Only with LucidWorks
      • LucidWorks and Solr
      • White Papers
      • LucidWorks Enterprise
      • LucidWorks Cloud
    • Certified Distributions
      • Certified Solr
      • Certified Lucene
    • Apache Releases
      • Apache Solr
      • Apache Lucene
  • Support & Services
    • Overview
    • Support
    • Training
    • Solr/Lucene Certification
    • ExpertLink Advisory
    • Consulting
    • Partners
    • Subscriptions
  • Why Lucid?
    • Why Lucid?
    • Technology
    • Who uses Lucene/Solr?
      • What customers are saying
    • Case Studies
    • Whitepapers
    • Demos
    • Webinars
  • Blog
  • DevZone
    • DevZone Overview
    • Forums (LWE)
    • Videos & Podcasts
      • How To's
      • Screencasts
      • Podcasts
      • Conference Videos
    • Technical Articles
      • Whitepapers
    • Reference Materials
      • Documentation
      • Solr Reference Guide
      • Solr & LucidWorks Matrix
      • Tutorials
    • Events
      • Conferences
      • Meet Ups
    • Code & Test
  • Downloads
  • About Us
    • Management
    • Apache Lucene/Solr Committers
    • Careers
    • News
      • Media Coverage
      • Press Releases
    • Contact Us
Sign Up or Log In
Home . Blog

July 6, 2009

Ranges over Functions in Solr 1.4

Posted by yonik

Solr 1.4 contains a new feature that allows range queries or range filters over arbitrary functions. It’s implemented as a standard Solr QParser plugin, and thus easily available for use any place that accepts the standard Solr Query Syntax by specifying the frange query type. Here’s an example of a filter specifying the lower and upper bounds for a function:

fq={!frange l=0 u=2.2}log(sum(user_ranking,editor_ranking))

The other interesting use for frange is to trade off memory for speed when doing range queries on any type of single-valued field. For example, one can use frange on a string field provided that there is only one value per field, and that numeric functions are avoided.

For example, here is a filter that only allows authors between martin and rowling, specified using a standard range query:
fq=author_last_name:[martin TO rowling]

And the same filter using a function range query (frange):
fq={!frange l=martin u=rowling}author_last_name

This can lead to significant performance improvements for range queries with many terms between the endpoints, at the cost of memory to hold the un-inverted form of the field in memory (i.e. a FieldCache entry – same as would be used for sorting). If the field in question is already being used for sorting or other function queries, there won’t be any additional memory overhead.

The following chart shows the results of a test of frange queries vs standard range queries on a string field with 200,000 unique values. For example, frange was 14 times faster when executing a range query / range filter that covered 20% of the terms in the field. For narrower ranges that matched less than 5% of the values, the traditional range query performed better.

Percent of terms covered Fastest implementation Speedup (how many times faster)
100% frange 43.32
20% frange 14.25
10% frange 8.07
5% frange 1.337
1% normal range query 3.59

Of course, Solr 1.4 also contains the new TrieRange functionality that will generally have the best time/space profile for range queries over numeric fields.

  • Share this:
  • Email
  • Facebook
  • Digg
  • Share
  • Print
  • Reddit
  • StumbleUpon

Category: Lucene, Solr

Tags: frange, function query, Lucene, qparser, range filter, range query, Solr

5 Responses to “Ranges over Functions in Solr 1.4”

  1. [...] and (b) innovative ways of delivering on those needs. Open source search innovation gave them numerical analysis, geo search, faceted navigation, and more. Yelp’s not the only one who’s used [...]

    December 18, 2009 17:49 — Lucid Imagination » Solr makes Google Yelp

  2. How do do this with geodist() ?

    fq=score_lat_lon:[39.7,-104 TO 38,-104.5]

    fq={!frange l=39.7 u=38}author_last_name

    Where to put the other point?

    ??

    December 22, 2010 15:03 — Bill Bell

  3. Bill,

    If I’m understanding your question, the answer would be to put the geodist() function bound to the frange, as in fq={!frange l=0 u=5}geodist(…)

    December 27, 2010 06:47 — Grant Ingersoll

  4. So, if I understood correctly, for numeric fields it should typically be better to use TrieRange functionality (which of course, I understand that frange does not take profit of it).
    Is it better only in speed/memory tradeoff or also if you only care about speed? (In other words, is a TrieRangeQuery typically faster or slower than frange over a numeric field?)
    Do date fields count as numeric fields (I think they are indexed as a long, but not sure…)? Is also TrieRange better for dates than frange?
    If you already have a field in the fieldcache because you are using it for sorting, is then typically better to use frange, even for numeric fields, as there’s no additional memory overhead?
    Thanks,

    Juan

    May 12, 2011 07:05 — Juan

  5. How do you execute a query with multiple !frange in them. I don’t seem to get any result when I have two frange,

    {!frange l=33787806 u=33787918}id
    {!frange l=40817415}id
    −

    createdOnGMTDate:[2011-07-01T14:30:00Z TO 2011-07-21T14:30:00Z]

    Regards,
    Rohit

    July 20, 2011 23:53 — Rohit

Leave a Reply

Go to Blog Front Page

  • Recent Posts

    • Lucene Revolution 2012 – Call for Participation now open!
    • SolrCloud is Coming (and looking to mix in even more ‘NoSQL’)
    • Our Solr Reference Guide updated for v3.5
    • Enhancing Discovery with Solr and Mahout – session slides now available!
    • Solr and LucidWorks feature matrix available
    • LucidWorks Enterprise latest version 2.0.1 released!
    • Why Not AND, OR, And NOT?
    • Options to tune document’s relevance in Solr
    • Dallas JavaMUG December 14th 2011
    • Apache Mahout user meeting – session slides and videos are now available!
  • Archives

    • January 2012
    • December 2011
    • November 2011
    • October 2011
    • September 2011
    • August 2011
  • Tags

    acts_as_solr apache Apache Mahout best practices chump code4lib dismax drupal enterprise search Erik Hatcher field collapsing function query Grant Ingersoll hoss image isfdb local params Lucene lucene revolution LucidGaze lucid imagination Mahout Marc Krellenstein Mark Miller nested queries nutch Open Source Open Source Search qparser query parser queryparser Rails release result grouping Richmond Ruby schema design sint Solr solr 3.1 solr 4.0 solr cloud sortable Tika VA
  • Contact Us
  • About Lucid Imagination
  • Help & Support
  • Training
  • Privacy Policy
  • Legal Terms of Use
  • Copyrights and Disclaimers
  • Log in

Apache Solr, Solr, Apache Lucene, Lucene and their logos are trademarks of the Apache Software Foundation.

© 2011 Lucid Imagination. All Right reserved.

loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.