<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Lucid Imagination &#187; functions</title>
	<atom:link href="http://www.lucidimagination.com/blog/category/functions/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.lucidimagination.com/blog</link>
	<description>Exclusively dedicated to Apache Lucene/Solr open source search technology</description>
	<lastBuildDate>Sat, 04 Feb 2012 01:12:03 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3</generator>
		<item>
		<title>Solr Result Grouping / Field Collapsing</title>
		<link>http://www.lucidimagination.com/blog/2010/09/16/2446/</link>
		<comments>http://www.lucidimagination.com/blog/2010/09/16/2446/#comments</comments>
		<pubDate>Fri, 17 Sep 2010 01:52:26 +0000</pubDate>
		<dc:creator>yonik</dc:creator>
				<category><![CDATA[Enterprise Search]]></category>
		<category><![CDATA[functions]]></category>
		<category><![CDATA[Solr]]></category>
		<category><![CDATA[spatial search]]></category>
		<category><![CDATA[field collapsing]]></category>
		<category><![CDATA[function query]]></category>
		<category><![CDATA[geo search]]></category>
		<category><![CDATA[result grouping]]></category>
		<category><![CDATA[solr 4.0]]></category>

		<guid isPermaLink="false">http://www.lucidimagination.com/blog/?p=2446</guid>
		<description><![CDATA[<p><strong>Result Grouping</strong>, also called <strong>Field Collapsing</strong>, has been committed to Solr!<br />
This functionality limits the number of documents for each &#8220;group&#8221;, usually defined by the unique values in a field (just like field faceting).</p>
<p>You can think of it like faceted search, except instead of just getting a count, you get the top documents for that constraint or category.  There are tons of potential use cases:</p>
<ul>
<li>For web search, only show 1 or </li>&#8230;</ul>]]></description>
			<content:encoded><![CDATA[<p><strong>Result Grouping</strong>, also called <strong>Field Collapsing</strong>, has been committed to Solr!<br />
This functionality limits the number of documents for each &#8220;group&#8221;, usually defined by the unique values in a field (just like field faceting).</p>
<p>You can think of it like faceted search, except instead of just getting a count, you get the top documents for that constraint or category.  There are tons of potential use cases:</p>
<ul>
<li>For web search, only show 1 or 2 results for a given website by collapsing on a site field.</li>
<li>For email search, only show 1 or 2 results for a given email thread</li>
<li>For e-commerce, show the top 3 products for each store category (i.e. &#8220;electronics&#8221;, &#8220;housewares&#8221;)</li>
<li>Hiding duplicate documents at query time.</li>
</ul>
<p>In addition to being able to group by the values of a field, you can also group by the values of a function query.  Given that geo search works as a function query, this also opens up possibilities for showing top query matches within 1 mile, between 1 and 2 miles, etc.</p>
<p>Just like faceting, we&#8217;ll be adding new functionality and making continual improvements.<br />
Result Grouping is documented on the <a href="http://wiki.apache.org/solr/FieldCollapsing">Solr Wiki</a>, and you will need a recent<br />
<a href="http://wiki.apache.org/solr/FrontPage#solr_development">nightly build</a> of Solr 4.0-dev to try it out (just make sure it&#8217;s dated after this post).</p>
]]></content:encoded>
			<wfw:commentRss>http://www.lucidimagination.com/blog/2010/09/16/2446/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Apache Solr 1.5 on the move with more &#8220;functionality&#8221;</title>
		<link>http://www.lucidimagination.com/blog/2009/12/12/apache-solr-1-5-on-the-move-with-more-functionality/</link>
		<comments>http://www.lucidimagination.com/blog/2009/12/12/apache-solr-1-5-on-the-move-with-more-functionality/#comments</comments>
		<pubDate>Sat, 12 Dec 2009 23:45:26 +0000</pubDate>
		<dc:creator>Grant Ingersoll</dc:creator>
				<category><![CDATA[apache]]></category>
		<category><![CDATA[functions]]></category>
		<category><![CDATA[Lucene]]></category>
		<category><![CDATA[Open Source]]></category>
		<category><![CDATA[Payloads]]></category>
		<category><![CDATA[Solr]]></category>
		<category><![CDATA[spatial search]]></category>
		<category><![CDATA[ZooKeeper]]></category>

		<guid isPermaLink="false">http://www.lucidimagination.com/blog/?p=1404</guid>
		<description><![CDATA[<p>The paint is barely dry on <a href="http://lucene.apache.org/solr">Apache Solr</a> 1.4 and the community is already on the move for Solr 1.5 (which may actually be Solr 2.0, but for now let&#8217;s call it 1.5).</p>
<p>I&#8217;m particularly excited about a few things:</p>
<ol>
<li>Massive scalability capabilities via distributed search, indexing and shard management &#8211; Up until now, Solr scales pretty well on the search side (I&#8217;ve seen billion+ document instances and we&#8217;ve benchmarked it at that level too), </li>&#8230;</ol>]]></description>
			<content:encoded><![CDATA[<p>The paint is barely dry on <a href="http://lucene.apache.org/solr">Apache Solr</a> 1.4 and the community is already on the move for Solr 1.5 (which may actually be Solr 2.0, but for now let&#8217;s call it 1.5).</p>
<p>I&#8217;m particularly excited about a few things:</p>
<ol>
<li>Massive scalability capabilities via distributed search, indexing and shard management &#8211; Up until now, Solr scales pretty well on the search side (I&#8217;ve seen billion+ document instances and we&#8217;ve benchmarked it at that level too), but the work underway in Solr 1.5 will take it to a whole new level, thanks to the integration of Apache <a href="http://hadoop.apache.org/zookeeper/">ZooKeeper</a> and other distributed technologies.  For those interested, check out the &#8220;cloud&#8221; branch in <a href="https://svn.apache.org/repos/asf/lucene/solr/branches/cloud/">SVN</a>.</li>
<li>Functions, functions, functions!  We&#8217;ve already added a bunch of <a href="http://wiki.apache.org/solr/FunctionQuery">functions</a> (see my <a href="http://www.lucidimagination.com/blog/2009/11/20/fun-with-solr-functions/">earlier post</a>) and I see more on the horizon.  Additionally, I see great value in adding, for lack of a better phrase, aggregating functions to the mix (via <a href="https://issues.apache.org/jira/browse/SOLR-1622">SOLR-1622</a>).  This will allow application designers to do much more sophisticated math across a search result set than what is currently available via the StatsComponent.  In some ways, this can empower business intelligence applications on top of Solr (I realize it is just a small piece of the BI pie) as well as more sophisticated mathematical applications.</li>
<li>Spatial Search!  It&#8217;s funny, a lot of people want spatial search and Solr could have simply harnessed a really nice existing package (LocalSolr) just as many already do, but by stepping back and taking a look at spatial in the context of the bigger picture of things (see <a href="https://issues.apache.org/jira/browse/SOLR-773">SOLR-773</a>) that would be nice to have in Solr, the community will be able to not only implement spatial search (by leveraging key pieces of LocalSolr where appropriate), but will also get a whole bevy of other features, including:
<ol>
<li>Sort By Function &#8211; Instead of a one off that sorts solely by distance, why not enable Solr users to sort by any arbitrary function?  I just committed this tonight via <a href="https://issues.apache.org/jira/browse/SOLR-1297">SOLR-1297</a>.</li>
<li>&#8220;Poly&#8221; Field Types &#8211; Thanks to <a href="https://issues.apache.org/jira/browse/SOLR-1131">SOLR-1131</a>, Solr&#8217;s FieldType mechanism can be used to represent multiple underlying fields.  This is especially useful for representing things like points in an <em>n</em>-dimensional space, Cartesian Tiers (zoom levels) and other cool things.  Moreover, it shows the types of abstractions Solr can overlay on the already powerful Apache Lucene to provide even more functionality.</li>
<li>Facet By Function &#8211; Sure, it&#8217;s great to put your distances into buckets, but why not put the result of any function into buckets?  See <a href="https://issues.apache.org/jira/browse/SOLR-1581">SOLR-1581</a>.</li>
<li>Spatial Query Parsers &#8211; aka geocoding &#8211; Parse things like street addresses, etc. and get back appropriate Query instances. See <a href="https://issues.apache.org/jira/browse/SOLR-1568">SOLR-1568</a> and <a href="https://issues.apache.org/jira/browse/SOLR-1578">SOLR-1578.</a></li>
<li>Several different distance functions, including haversine (great circle), Manhattan, Euclidean (Solr actually now supports all p-norms as distance functions.)  See <a href="https://issues.apache.org/jira/browse/SOLR-1302">SOLR-1302</a>.  I even added in the ability to do String distance calculations using Levenstein (edit), Jaro-Winkler, n-gram (basically all of the Lucene spellchecker distance measures, as well as any user defined String Distance calculation.</li>
<li>&#8220;pseudo&#8221; fields &#8211; Instead of just hacking the ability to put a distance calculation into the result, why not allow the response to stream out &#8220;fields&#8221; based on things like functions or other user defined values?  See <a href="https://issues.apache.org/jira/browse/SOLR-1298">SOLR-1298</a>.</li>
</ol>
</li>
<li>Field Collapsing &#8211; I haven&#8217;t had time to work on it, but I suspect Field Collapsing will finally make it into 1.5.  Field Collapsing allows Solr to &#8220;roll-up&#8221; similar results much like you see on many Internet search sites that indent results from the same domain.</li>
<li>Payload and Span Query support &#8211; Solr&#8217;s been able to index payloads for some time now, but it still requires a user to hook in their own query parser support.  It would also be really great to see functions that can work on payloads, too. See <a href="https://issues.apache.org/jira/browse/SOLR-1337">SOLR-1337</a> and <a href="https://issues.apache.org/jira/browse/SOLR-1485">SOLR-1485</a>.</li>
</ol>
<p>Of course, as I always say, &#8220;in open source, you never know where the next good idea is going to come from&#8221;, so I have total faith that the Solr community will come up with a plethora of other great new features, as well as the usual bug fixes, etc.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.lucidimagination.com/blog/2009/12/12/apache-solr-1-5-on-the-move-with-more-functionality/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
	</channel>
</rss>

