Strategies for creating a meaningful Sitecore site search experience – Part 1

Today, data is considered as a strategic asset. Companies are going leaps and bounds to generate meaningful content from data.

Search in general has evolved as customers need sought out information fast and relevant.

As websites are built, providing a search feature has become the norm these days. Customers are focused, they want relevant information within a fraction of a second.

The most common example is Google search and you can read the basics of search, crawling, indexing here:

Let us move out of the generic search and focus on Sitecore as a Content Management System (CMS) and it’s search capabilities.

If you are using Sitecore, you have the option to use Lucene or SOLR as the indexing mechanism for search.

Sitecore comes with a default set of search indexes that along with your search and indexing provider help to improve the performance of your website search.

When you configure Sitecore servers in a scalable environment, you first decide whether you want to use Lucene or SOLR as your search and indexing provider. You then configure the indexes you need on each server.

At a high-level

  • If you use Lucene, it is file-based. The sharing of indexes is not supported. Each server must maintain its own Lucene indexes.
  • If you use SOLR, the index storage is centralized and can be shared across multiple servers.

The basic configuration of Lucene and SOLR can be obtained through sitecore knowledgebase below

Sitecore 8.1

Sitecore 8.2

Indexes required in a scalable sitecore environment


You can also integrate COVEO a search product by itself with sitecore. COVEO for Sitecore:


We are going to skip all this wonderful information assuming we know the basics of hooking SOLR and Lucene with Sitecore as we explore further.


The purpose of this two part blog is to look beyond the basics and list some Sitecore strategies to obtain meaningful search results using SOLR or Lucene.

Strategy 1: Crawlers and Crawler root – Configuration approach

You can customize the sitecore index configuration and update the crawlers to crawl individual sections of the site to generate relevant search results.

For the purpose of this blog we took the example of sitecore_web_index


File: Sitecore.ContentSearch.Solr.Index Web.config

<locations hint=”list:AddCrawler”>

<crawler type=”Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch”>





File: Sitecore.ContentSearch.Lucene.Index.Web.config

<locations hint=”list:AddCrawler”>

<crawler type=”Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch”>





You can also create multiple crawlers for the same index pointing to different locations in sitecore content tree for more meaningful content type to be indexed.

Example either SOLR or Lucene for sitecore_web_index:

<locations hint=”list:AddCrawler”>

<TreeCrawler type=”Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch”>




<MediaCrawler type=”Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch”>


<Root>/sitecore/Media Library</Root>



Following are the advantages of having multiple crawlers in the same index:

  1. Improve sitecore/application performance.
  2. Apply business rules directly in the query: such as pagination, page sorting.
  3. Easier to maintain.


Having one or multiple crawlers depends on business and technical requirements keeping performance and maintenance in mind.

Strategy 2: Selective indexing Strategy – Configuration approach

You can include/exclude templates and/or fields from getting indexed. The idea over here is to selectively index relevant items for search provider to filter results. This can be achieved by excluding templates and fields that you do not want to index.

Example: Sitecore.ContentSearch.Solr.Index.Web.config


<?xml version=”1.0″ encoding=”utf-8″ ?>

<configuration xmlns:patch=””>



<configuration type=”Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch”>

<indexes hint=”list:AddIndex”>

<configuration ref=”contentSearch/indexConfigurations/defaultSolrIndexConfiguration”>

<documentOptions type=”Sitecore.ContentSearch.SolrProvider.SolrDocumentBuilderOptions, Sitecore.ContentSearch.SolrProvider”>



               This setting allows you to exclude items that are based on specific templates from the index.


            <exclude hint=”list:AddExcludedTemplate”>




<Services_Page>{01D8C754-52ED-44EE-B36C-0E744A06E068}</ Services_Page>


<exclude hint=”list:AddExcludedField”>





<strategies hint=”list:AddStrategy”>…


<locations hint=”list:AddCrawler”>…

</locations>   …






You can also include and exclude templates/fields for lucene.

Example: Sitecore.ContentSearch.Lucene.Index.Web.config

Format for inclusion and exclusion is similar to SOLR file above.

Note: Use template name and template ID from sitecore or Field Name and Field ID from sitecore.

Leave a Reply