Customizing Azure Search
Kentico provides a customization model that allows developers to extend or adjust how the system builds and maintains Azure Search indexes. Customization provides a way to set up Azure Search features that require advanced configuration, as well as adapt to changes in the Azure Search functionality and leverage new features.
For example, you can achieve the following scenarios by customizing the Azure Search:
- Add scoring profiles for indexes
- Add suggesters for indexes
- Set language analyzers for index fields
- Register custom data types for Azure Search
- Configure the retry policy and batch size used by indexing operations
To perform Azure Search customizations, you need to run code during the initialization of the application. Add the required code by creating a custom module class:
Open your Kentico solution in Visual Studio.
Create a new Class Library project in the Kentico solution (or reuse an existing custom project).
Add references to the required Kentico libraries (DLLs) for the new project:
Right-click the project and select Add -> Reference.
Switch to the Browse tab, click Browse, and navigate to the Lib folder of your Kentico web project.
Add references to the following libraries (and any others required for your code):
- CMS.Base.dll
- CMS.Core.dll
- CMS.DataEngine.dll
- CMS.Search.Azure.dll
Reference the custom project from the Kentico web project (CMSApp or CMS).
Edit the project’s AssemblyInfo.cs file (in the Properties folder).
Add the AssemblyDiscoverable assembly attribute:
using CMS; [assembly:AssemblyDiscoverable]Create the custom module class under your project.
If you are utilizing the MVC development model, also deploy the custom project to your separate MVC application (otherwise indexing may not work correctly for changes performed through the live site).
You can now add Azure Search customizations by overriding the module’s OnInit method. For most typical scenarios, you need to assign handler methods for Azure Search indexing events.
Azure Search SDK requirement
Most types of customizations additionally require usage of the Azure Search .NET SDK, which you can integrate by installing the Microsoft.Azure.Search NuGet package into your custom project.
Configuring the Azure Search retry policy and batch size
In addition to custom handling of Azure Search indexing events, the Kentico API allows developers to configure the retry policy and batch size that the system uses when performing Azure Search indexing operations.
Get an instance of the SearchEngineConfiguration class within the initialization code of your custom module class and set its properties (see the sections below).
Retry policy
Because Azure Search indexes are cloud-based and hosted outside of the Kentico application, requests that interact with indexes may fail due to network errors, service unavailability, etc. To deal with such issues, the system sends all indexing requests using a retry policy with exponential backoff. Requests that fail are automatically repeated several times with exponentially growing time intervals.
If necessary, you can adjust the parameters of the retry policy by setting the following properties of the SearchEngineConfiguration class:
- RetryCount – sets the maximum number of retry attempts that the system performs after an indexing operation fails. Setting the value to 0 disables the retry functionality (not recommended). The default value is 5.
- MaxBackoffTimeSeconds – sets the ceiling for the time interval between retry attempts (in seconds). The default value is 8.
For example, if you set RetryCount to 7 and MaxBackoffTimeSeconds to 16, the system retries failed indexing operations up to 7 times, with the following intervals between the attempts (in seconds): 0, 1, 2, 4, 8, 16, 16
Batch size
The DocumentsBatchSize property of the SearchEngineConfiguration class sets the maximum number of search documents processed by a single request to the Azure Search service. For example, when building a new index that covers 2500 pages in Kentico with a maximum batch size of 1000, the system first creates the Azure Search index with 1000 documents, and then updates the index twice by adding 1000 and 500 documents.
The default document batch size value is 1000. When setting a custom value, you need to respect the API Request limits of the Azure Search service. For example, you may need to decrease the batch size if the total request size exceeds the maximum limit of 16 MB (when indexing a large number of objects with long text values).
Azure Search document batch size vs. Index batch size
The maximum number of search documents processed per request is also limited by the Batch size setting of individual search indexes, which can be configured in the Kentico administration interface. The index batch size sets the maximum number of records that the system retrieves per query when loading data from the Kentico database, which effectively also limits the document batch size of Azure Search requests. By default, the index batch size is 500.
The SearchEngineConfiguration.DocumentsBatchSize property only has an effect for indexes whose batch size setting is greater than the property’s value.
Example
using CMS;
using CMS.DataEngine;
using CMS.Search.Azure;
// Registers the custom module into the system
[assembly: RegisterModule(typeof(CustomAzureSearchModule))]
public class CustomAzureSearchModule : Module
{
// Module class constructor, the system registers the module under the name "CustomAzureSearch"
public CustomAzureSearchModule()
: base("CustomAzureSearch")
{
}
// Contains initialization code that is executed when the application starts
protected override void OnInit()
{
base.OnInit();
var azureSearchConfiguration = SearchEngineConfiguration.Instance;
// Customizes the Azure Search retry policy
azureSearchConfiguration.RetryCount = 7;
azureSearchConfiguration.MaxBackoffTimeSeconds = 16;
// Sets the maximum number of search documents processed by each request to the Azure Search service
azureSearchConfiguration.DocumentsBatchSize = 500;
}
}
Changing the default domain suffix of Azure search services
Hotfix 12.0.97 required
The section applies only for instances running hotfix 12.0.97 or newer.
By default, the system assumes your Azure Search services are hosted on the search.windows.net domain (true for the majority of commercial subscriptions). Search requests for Azure indexes are generated using this suffix and the provided search service name.
However, certain Azure subscriptions or licenses host search services under a different domain. For example, Azure Government subscriptions use the search.azure.us domain. If your services are hosted on a different domain, you can set the CMSAzureSearchDnsSuffix configuration key to change the suffix used by the system when generating Azure search requests.
<appSettings>
...
<!-- Configures the system to generate requests in format: myazureservice.search.azure.us -->
<add key="CMSAzureSearchDnsSuffix" value="search.azure.us" />
</appSettings>
Reference - Azure Search events
This section provides an overview of system events that developers can handle to customize how the system builds and maintains Azure Search indexes.
For general information about Azure Search indexes and their structure, refer to the Create an Azure Search index article.
Indexes
Class: SearchServiceManager
|
Event |
Event types |
Description |
|
CreatingOrUpdatingIndex |
Execute |
Occurs when the system sends requests to create or update the definition of an index within an Azure Search service, for example when rebuilding Azure Search indexes in Kentico. The event is NOT triggered when adding, updating or removing documents within an existing index, unless an update of the index’s definition is required (e.g. when adding a document with new fields for the first time). Depending on the number of indexed pages or objects and the used batch size, the event may occur multiple times when building a single search index – separately for each batch of processed search documents that include a new field not yet contained by the index. When implementing handlers for the event, always consider cases where the related index already exists and does not yet contain all possible fields. Examples of use: Handler parameters: CreateOrUpdateIndexEventArgs
|
Documents
Class: DocumentCreator (access an Instance of the class to assign event handlers)
|
Event |
Event types |
Description |
|
CreatingDocument |
Before |
Occurs when the system creates individual documents within an Azure Search index. Triggered separately for every indexed object. Use the Before event if you wish to modify the processed source data for the given document (SearchDocument property of the handler’s CreateDocumentEventArgs parameter). Use the After event to adjust the properties of the resulting Document object (Document property of the handler’s CreateDocumentEventArgs parameter) or its fields (Fields property of the handler’s CreateDocumentEventArgs parameter). Handler parameters: CreateDocumentEventArgs
|
|
AddingDocumentValue |
Execute |
Occurs when the system sets the values of individual fields for documents within an Azure Search index. Triggered separately for each field of every indexed object. You can use the event to modify the AzureName and Value properties of the handler’s AddDocumentValueEventArgs parameter before the value is converted to an Azure Search data type and saved to the document. Handler parameters: AddDocumentValueEventArgs
|
Fields
Class: DocumentFieldCreator (access an Instance of the class to assign event handlers)
|
Event |
Event types |
Description |
|
CreatingField |
Before |
Occurs when the system creates individual fields for a document within an Azure Search index. Triggered separately for each field of every indexed object. Use the Before event if you wish to manually initialize the Field object (Microsoft.Azure.Search.Models.Field). Use the After event to adjust the properties of the resulting Field object (Field property of the handler’s CreateFieldEventArgs parameter). Example of use: Setting language analyzers for index fields Handler parameters: CreateFieldEventArgs
|
|
CreatingFields |
Before |
Occurs when the system creates the set of fields for a document within an Azure Search index. Triggered separately for every indexed object. Use the Before event if you wish to modify or extend the collection of processed source fields (SearchFields property of the handler’s CreateFieldsEventArgs parameter). Use the After event to modify the resulting list of Azure Search fields (Fields property of the handler’s CreateFieldsEventArgs parameter). Handler parameters: CreateFieldsEventArgs
|
Kentico system fields
Search indexes in Kentico contain special system fields that are required for general functionality and various other features. For example _content, _index, _idcolumnname. We do not recommend modifying or renaming the system fields within custom handlers of field or search document events.
In most cases, the names of system fields in Kentico start with an underscore. When creating matching fields within Azure Search indexes, the sys_ prefix is used instead (to comply with the Azure Search naming rules).