<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Lucid Imagination &#187; function query</title>
	<atom:link href="http://www.lucidimagination.com/blog/tag/function-query/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.lucidimagination.com/blog</link>
	<description>Exclusively dedicated to Apache Lucene/Solr open source search technology</description>
	<lastBuildDate>Sat, 04 Feb 2012 01:12:03 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3</generator>
		<item>
		<title>Solr Result Grouping / Field Collapsing</title>
		<link>http://www.lucidimagination.com/blog/2010/09/16/2446/</link>
		<comments>http://www.lucidimagination.com/blog/2010/09/16/2446/#comments</comments>
		<pubDate>Fri, 17 Sep 2010 01:52:26 +0000</pubDate>
		<dc:creator>yonik</dc:creator>
				<category><![CDATA[Enterprise Search]]></category>
		<category><![CDATA[functions]]></category>
		<category><![CDATA[Solr]]></category>
		<category><![CDATA[spatial search]]></category>
		<category><![CDATA[field collapsing]]></category>
		<category><![CDATA[function query]]></category>
		<category><![CDATA[geo search]]></category>
		<category><![CDATA[result grouping]]></category>
		<category><![CDATA[solr 4.0]]></category>

		<guid isPermaLink="false">http://www.lucidimagination.com/blog/?p=2446</guid>
		<description><![CDATA[<p><strong>Result Grouping</strong>, also called <strong>Field Collapsing</strong>, has been committed to Solr!<br />
This functionality limits the number of documents for each &#8220;group&#8221;, usually defined by the unique values in a field (just like field faceting).</p>
<p>You can think of it like faceted search, except instead of just getting a count, you get the top documents for that constraint or category.  There are tons of potential use cases:</p>
<ul>
<li>For web search, only show 1 or </li>&#8230;</ul>]]></description>
			<content:encoded><![CDATA[<p><strong>Result Grouping</strong>, also called <strong>Field Collapsing</strong>, has been committed to Solr!<br />
This functionality limits the number of documents for each &#8220;group&#8221;, usually defined by the unique values in a field (just like field faceting).</p>
<p>You can think of it like faceted search, except instead of just getting a count, you get the top documents for that constraint or category.  There are tons of potential use cases:</p>
<ul>
<li>For web search, only show 1 or 2 results for a given website by collapsing on a site field.</li>
<li>For email search, only show 1 or 2 results for a given email thread</li>
<li>For e-commerce, show the top 3 products for each store category (i.e. &#8220;electronics&#8221;, &#8220;housewares&#8221;)</li>
<li>Hiding duplicate documents at query time.</li>
</ul>
<p>In addition to being able to group by the values of a field, you can also group by the values of a function query.  Given that geo search works as a function query, this also opens up possibilities for showing top query matches within 1 mile, between 1 and 2 miles, etc.</p>
<p>Just like faceting, we&#8217;ll be adding new functionality and making continual improvements.<br />
Result Grouping is documented on the <a href="http://wiki.apache.org/solr/FieldCollapsing">Solr Wiki</a>, and you will need a recent<br />
<a href="http://wiki.apache.org/solr/FrontPage#solr_development">nightly build</a> of Solr 4.0-dev to try it out (just make sure it&#8217;s dated after this post).</p>
]]></content:encoded>
			<wfw:commentRss>http://www.lucidimagination.com/blog/2010/09/16/2446/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Ranges over Functions in Solr 1.4</title>
		<link>http://www.lucidimagination.com/blog/2009/07/06/ranges-over-functions-in-solr-14/</link>
		<comments>http://www.lucidimagination.com/blog/2009/07/06/ranges-over-functions-in-solr-14/#comments</comments>
		<pubDate>Tue, 07 Jul 2009 01:43:02 +0000</pubDate>
		<dc:creator>yonik</dc:creator>
				<category><![CDATA[Lucene]]></category>
		<category><![CDATA[Solr]]></category>
		<category><![CDATA[frange]]></category>
		<category><![CDATA[function query]]></category>
		<category><![CDATA[qparser]]></category>
		<category><![CDATA[range filter]]></category>
		<category><![CDATA[range query]]></category>

		<guid isPermaLink="false">http://www.lucidimagination.com/blog/?p=778</guid>
		<description><![CDATA[<p>Solr 1.4 contains a new feature that allows range queries or range filters over arbitrary functions.  It&#8217;s implemented as a standard <a href="http://lucene.apache.org/solr/api/org/apache/solr/search/FunctionRangeQParserPlugin.html">Solr QParser plugin</a>, and thus easily available for use any place that accepts the standard <a href="http://wiki.apache.org/solr/SolrQuerySyntax">Solr Query Syntax</a> by specifying the <strong>frange </strong>query type.  Here&#8217;s an example of a filter specifying the lower and upper bounds for a function:</p>
<p><code>fq={!frange l=0 u=2.2}log(sum(user_ranking,editor_ranking))</code></p>
<p>The other interesting use for frange is to trade off memory &#8230;</p>]]></description>
			<content:encoded><![CDATA[<p>Solr 1.4 contains a new feature that allows range queries or range filters over arbitrary functions.  It&#8217;s implemented as a standard <a href="http://lucene.apache.org/solr/api/org/apache/solr/search/FunctionRangeQParserPlugin.html">Solr QParser plugin</a>, and thus easily available for use any place that accepts the standard <a href="http://wiki.apache.org/solr/SolrQuerySyntax">Solr Query Syntax</a> by specifying the <strong>frange </strong>query type.  Here&#8217;s an example of a filter specifying the lower and upper bounds for a function:</p>
<p><code>fq={!frange l=0 u=2.2}log(sum(user_ranking,editor_ranking))</code></p>
<p>The other interesting use for frange is to trade off memory for speed when doing range queries on any type of single-valued field.  For example, one can use <strong>frange </strong>on a string field provided that there is only one value per field, and that numeric functions are avoided.</p>
<p>For example, here is a filter that only allows authors between martin and rowling, specified using a standard range query:<br />
<code>fq=author_last_name:[martin TO rowling]</code></p>
<p>And the same filter using a function range query (<strong>frange</strong>):<br />
<code>fq={!frange l=martin u=rowling}author_last_name</code></p>
<p>This can lead to significant performance improvements for range queries with many terms between the endpoints, at the cost of memory to hold the un-inverted form of the field in memory (i.e. a FieldCache entry &#8211; same as would be used for sorting).  If the field in question is already being used for sorting or other function queries, there won&#8217;t be any additional memory overhead.</p>
<p>The following chart shows the results of a test of frange queries vs standard range queries on a string field with 200,000 unique values.  For example, frange was 14 times faster when executing a range query / range filter that covered 20% of the terms in the field.  For narrower ranges that matched less than 5% of the values, the traditional range query performed better.</p>
<table border="1">
<tbody>
<tr>
<th>Percent of terms covered</th>
<th>Fastest implementation</th>
<th>Speedup (how many times faster)</th>
</tr>
<tr>
<td>100%</td>
<td>frange</td>
<td>43.32</td>
</tr>
<tr>
<td>20%</td>
<td>frange</td>
<td>14.25</td>
</tr>
<tr>
<td>10%</td>
<td>frange</td>
<td>8.07</td>
</tr>
<tr>
<td>5%</td>
<td>frange</td>
<td>1.337</td>
</tr>
<tr>
<td>1%</td>
<td>normal range query</td>
<td>3.59</td>
</tr>
</tbody>
</table>
<p>Of course, Solr 1.4 also contains the new <a href="http://www.lucidimagination.com/blog/2009/05/13/exploring-lucene-and-solrs-trierange-capabilities/">TrieRange </a>functionality that will generally have the best time/space profile for range queries over numeric fields.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.lucidimagination.com/blog/2009/07/06/ranges-over-functions-in-solr-14/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Nested Queries in Solr</title>
		<link>http://www.lucidimagination.com/blog/2009/03/31/nested-queries-in-solr/</link>
		<comments>http://www.lucidimagination.com/blog/2009/03/31/nested-queries-in-solr/#comments</comments>
		<pubDate>Tue, 31 Mar 2009 21:36:55 +0000</pubDate>
		<dc:creator>yonik</dc:creator>
				<category><![CDATA[Lucene]]></category>
		<category><![CDATA[Solr]]></category>
		<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[function query]]></category>
		<category><![CDATA[local params]]></category>
		<category><![CDATA[nested queries]]></category>
		<category><![CDATA[query parser]]></category>
		<category><![CDATA[query parser plugin]]></category>
		<category><![CDATA[query syntax]]></category>

		<guid isPermaLink="false">http://www.lucidimagination.com/blog/?p=586</guid>
		<description><![CDATA[<table class="plain" border="0" width="100%" bordercolor="#ffccff">
<tbody>
<tr valign="top">
<td width="60%">The ability to nest an arbitrary query type inside another query type is a useful feature that was quietly added to Solr some time ago, along with the support for query parser plugins to support different query types.I finally got around to <a href="http://issues.apache.org/jira/browse/SOLR-1046" target="_blank">fixing</a> nested queries for the function query parser, and figured it was high time I documented nested queries, along with the <a href="http://wiki.apache.org/solr/LocalParams" target="_blank">LocalParams</a> syntax that allows one to add metadata to a query parameter, </td></tr></tbody>&#8230;</table>]]></description>
			<content:encoded><![CDATA[<table class="plain" border="0" width="100%" bordercolor="#ffccff">
<tbody>
<tr valign="top">
<td width="60%">The ability to nest an arbitrary query type inside another query type is a useful feature that was quietly added to Solr some time ago, along with the support for query parser plugins to support different query types.I finally got around to <a href="http://issues.apache.org/jira/browse/SOLR-1046" target="_blank">fixing</a> nested queries for the function query parser, and figured it was high time I documented nested queries, along with the <a href="http://wiki.apache.org/solr/LocalParams" target="_blank">LocalParams</a> syntax that allows one to add metadata to a query parameter, or even change the type of a query (i.e. which query parser is used to parse the query string.)</td>
<td width="5%"> </td>
<td width="35%"><strong><em>You might also be interested in:</em></strong></p>
<ul>
<li><a href="http://www.lucidimagination.com/Solutions/Webinars/Analyze-This-Tips-and-tricks-getting-LuceneSolr-Analyzer-index-and-search-your-content">Analyze This! Tips and tricks on getting the Lucene/Solr Analyzer to index and search your content right</a> &#8211; On-demand Webinar</li>
<li><a href="http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Optimizing-Findability-Lucene-and-Solr">Optimizing Findability</a> &#8211; Tech Article</li>
<li><a href="http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr">Solr 1.4 Download</a></li>
<li><a href="http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr/Reference-Guide">Solr 1.4 Reference Guide</a></li>
</ul>
</td>
</tr>
</tbody>
</table>
<h3>Nested Queries in Lucene Syntax</h3>
<p>To embed a query of another type in a Lucene/Solr query string, simply use the magic field name <strong>_query_</strong>.  The following example embeds a lucene query <strong>type:poems </strong>into another lucene query:</p>
<pre><strong>text:"roses are red" AND _query_:"type:poems"</strong></pre>
<p>Now of course this isn&#8217;t too useful on it&#8217;s own, but it becomes very powerful in conjunction with the query parser framework and local params which allows us to change the types of queries.  The following example embeds a <a href="http://wiki.apache.org/solr/DisMaxRequestHandler" target="_blank">DisMax query</a> in a normal lucene query:</p>
<pre><strong>text:hi  AND  _query_:"{!dismax qf=title pf=title}how now brown cow"</strong></pre>
<p>And we can further use parameter defererencing in the local params syntax to make it easier for the front-end to compose the request:</p>
<pre>&amp;<strong>q=text:hi  AND  _query_:"{!dismax qf=title pf=title v=$qq}</strong>
&amp;<strong>qq=how now brown cow</strong></pre>
<h3>Nested Queries in Function Query Syntax</h3>
<p>This is the part that was previously broken, and is only fixed/available in Solr 1.4.  You can use query() function to embed any other type of query in a function query, and do computations on the relevancy scores returned by that query.  Some examples from the Solr wiki are <a href="http://wiki.apache.org/solr/FunctionQuery#head-da96f90c1632609bae6fd86c853b9e13e514ce89" target="_blank">here</a>.</p>
<h3>Pure Nested Query</h3>
<p>There is also a nested query parser plugin that allows one to create pure nested queries.  Is a nested query without any containing query even useful? Surprisingly yes, as it allows further decomposition of query requests.</p>
<p>For example, the following allows an easy way for the client to specify that they want some sort of recency date boost added into the relevancy score, while leaving the exact query type up to the Solr server config (via search handler defaults in solrconfig.xml)</p>
<p>The client query would specify the boost query as $datefunc:</p>
<pre><strong>q=how now brown cow&amp;bq={!query v=$datefunc}</strong></pre>
<p>And the defaults for the handler in solrconfig.xml would contain the actual definition of datefunc as a function query:</p>
<pre>&lt;lst name="defaults"&gt;
   &lt;str name="datefunc"&gt;{!func}recip(rord(date),1,1000,1000)&lt;/str&gt;
   [...]</pre>
<p>The same idea could be used to allow a client to switch between complex filters, without having to specify what those filters are.</p>
<p>Without the nested query parser type, it would only be possible to specify the query value in a separate place (via local params v=$param) not the type also.</p>
<h3>The Future</h3>
<p>An XML Query Parser is on the way via <a href="https://issues.apache.org/jira/browse/SOLR-839">SOLR-839</a> that will allow expressing arbitrarily complex Lucene queries in XML.  As the number of query parsers grows, the importance of being able to mix, match, and nest them will become increasingly important.   One of the first extensions to the XML query parser should be to hook in nested queries of course!</p>
<p>The subclasses of QParserPlugin show <a href="http://lucene.apache.org/solr/api/org/apache/solr/search/QParserPlugin.html" target="_blank">all of the query parsers currently available to Solr</a>.  If you can&#8217;t find the query parser you&#8217;re looking for, you can create your own and register it via solrconfig.xml!</p>
<hr />
<p><strong><em>You might also be interested in:</em></strong></p>
<ul>
<li><a href="http://www.lucidimagination.com/Solutions/Webinars/Analyze-This-Tips-and-tricks-getting-LuceneSolr-Analyzer-index-and-search-your-content">Analyze This! Tips and tricks on getting the Lucene/Solr Analyzer to index and search your content right</a> &#8211; On-demand Webinar</li>
<li><a href="http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Optimizing-Findability-Lucene-and-Solr">Optimizing Findability</a> &#8211; Tech Article</li>
<li><a href="http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr">Solr 1.4 Download</a></li>
<li><a href="http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr/Reference-Guide">Solr 1.4 Reference Guide</a></li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://www.lucidimagination.com/blog/2009/03/31/nested-queries-in-solr/feed/</wfw:commentRss>
		<slash:comments>20</slash:comments>
		</item>
	</channel>
</rss>

