1. senthilkumar.arumugam
   Member

   Hi,

   IS there a way to configure and control the MIME types as part of crawling in the current version of LWE? We are able to filter the specific MIME types at the time of querying. We would like to avoid crawling and indexing specific MIME types that are not our interest.  Can someone please let us know if there is a way to inform the crawler not to crawl specific MIME types for eg zip, .rar .

   Posted 3 months ago #

2. Lance
   Professional Services Engineer

   In the Data Source configuration page, you will see "Include paths" and "Exclude paths". You can use regular expressions to select or remove MIME types. To remove .zip files, add this to "Exclude paths":

   http://.*\.zip

   This will select every http link which is a zip file. About regular expressions:

   http://lucidworks.lucidimagination.com/display/help/Using+Regular+Expressions

   Posted 3 months ago #

3. senthilkumar.arumugam
   Member

   This helped and worked. Thank you for the quick reply.

   Posted 3 months ago #

RSS feed for this topic

Reply

You must log in to post.