Overview
The following details the steps required to implement the Lucene search engine. Each of the steps up to 'Advanced Setup' are required for Lucene search to operate.
Further documentation on Lucene can be found at Product Search with the Lucene Search Engine
To Implement Lucene search:
Define the Index Fields
Set default settings
Create the Index
Select the Roles for Testing
Enable the Search Engine
Perform Some Tests
Go Live - Enable all roles
Advanced Setup
Step-by-step guides
1. Define the Index Fields
Lucene search must have a search index created to operate. The first step in creating an index is to define the index fields.
Setting up index fields is a required step in implementing Lucene. It determines what data will be stored in the search index, and therefore used is search queries by end users. Any fields in the Product or ProductCategory table can be included in the indexed data for a product. Index field maintenance also allows configuration of how the data is indexed - for example whether or not word stemming should be used, allowing searchers to find plurals and other word stems like "ed", "ing", etc.
To set-up an index field:
- Go to the 'Index fields' tab on the Lucene Settings Administration page.
- Add a new field.
- Enter the 'Field name'. This is the name of the field in the index, and does not need to be related to any of the product or category field. The field name can be used by searchers to limit searching to a specific field, for example "description:book" would show products that have the word "book" in the "description" index field.
- Select the 'Analyser' to be used. 'Standard' and 'Snowball' are supported.
- The Snowball analyser will apply stemming algorithms to the indexed data, allowing for plurals and other word stems to be searched.
- The standard analyser simply breaks the search field data in to words, without applying any stemming.
- It is recommended to use the Snowball analyser for index fields that contain descriptive text, and the standard analyser for other fields.
- Enter the Source of the index field data. You can enter tokens for any fields in the Product or ProductCategory fields, and multiple fields can be included in a single index field.
- For Product fields, use the token [Product.Fieldname], (e.g. [Product.Description]).
- For ProductCategory fields, use the token [ProductCategory.Fieldname], (e.g. [ProductCategory.Notes1]).
- To include more than one field, use the + symbol, e.g. [Product.Description]+[Product.Notes1]+[ProductCategory.Notes5]
- Enter a field Boost Value. Entering a value of greater than 1 will make the data in the field considered more relevant in search results. A value less than 1 (e.g 0.5) makes the data less relevant.The Boost Value must be greater than zero.
- Select if the Field Value should be included in the spell checker. This determines which data is used for Lucene's spell checking database. Generally, you would include descriptive fields in the spell check database, but you would not include fields that contain data such as product codes, as these will pollute the spell checking suggestions.
2. Set default settings
For a fast start, we recommend the following setting values which can be configured on the 'Index Settings' tab of the Lucene Settings Administration page:
Setting | Value |
|---|---|
Enabled | Leave off until the index is created (see below) |
Enable Product Boost | Off |
Enable Boost Words | On (will not have any impact until boost words are defined) |
All other flags are not relevant for a fast start (as they relate to product boost which is turned off)
3. Create the Index
Go to the 'Indexing' tab of the Lucene Settings page and Reindex the search database. It is not necessary to tick the 'Recreate Index' field, as the index will automatically be created if it does not exist.
A Progress Bar will display the progress of the search indexing. Depending on field setup, the system can generally index at a rate of approximately 500 products per second. So if your database contains 10,000 products, the index operation should take around 20 seconds.
Users can perform searches while the Index is being updated, however if the 'Recreate Index' option has been checked, searches during index building may not return all results.
Rebuilding the index without 'Recreate Index' ticked will update the index for existing products only. If an index has been built previously and a product is subsequently deleted, the product will NOT be removed from the index. Hence, searches can return the deleted product.
4. Select the Roles for Testing
Go to the 'Role Settings' page and enable some roles for testing purposes. Initially this can just be the 'Administrator' role so that testing can be done through the "tools" section of Lucene Administration.
5. Enable the Search Engine
Go to the 'Index Settings' tab and set the 'Enabled' flag. This will pass product search request for the enabled roles through the Lucene Search engine.
6. Perform Some Tests
Test some product searches using the 'Tools' tab in the Lucene Administration page, or through the product search pages of a role enabled for Lucene search. The Tools section allows you to analyse search result scores and raw Lucene search output.
7. Go Live - Enable all roles
Once you are happy with the search results, enable Lucene search for other roles if required.
8. Advanced Set-up
The following steps are not essential to using Lucene, but will give your customers a better search experience if you spend the time setting them up:
- Product Boost
- Boost Words
- Query and Index Translations
- Did You Mean search suggestions
- Adding Lucene tokens to your product search result templates
Related help
