Troubleshooting local search indexes on Azure

Development model: MVC Portal Engine
✖
Copy to clipboard

This page describes how to troubleshoot issues related to locally stored smart search indexes when running the Kentico application on Azure. Please note that each section applies either to Web Apps, Cloud Services, or both.

Search indexes are not being rebuilt or there are hanging tasks in the system

If your local search indexes are not being built correctly or if many indexing tasks are stuck in the system, there is probably a conflict between the search index files located in the application’s file system and files located on the blob storage. When an application is mapped to the blob storage, files in the file system cannot be modified. Therefore, the system is not able to delete the old search indexes.

Please proceed to the section which corresponds to your Azure environment.

Cloud Services - removing search index files

Applies to: CLOUD SERVICES

Quick fix

The solution for this issue on Cloud Services is to delete all files in the ~/App_Data/CMSModules/SmartSearch folder. Connect to your service through remote desktop and delete the files remotely in the file system.

Prevention

Delete the files in the ~/App_Data/CMSModules/SmartSearch folder and redeploy the cloud service. Also delete the files every time before you deploy your application to Cloud Services.

Web Apps - removing search index files

Applies to: WEB APPS

Quick fix

The immediate solution on Web Apps is to delete all files in the ~/App_Data/CMSModules/SmartSearch folder. You can connect to your Web Apps service through FTP and delete the files in the file system.

Prevention

We recommend mapping only your media library folders to the blob storage and leaving all other files in the file system of the application.

To map only the media library folder to the blob storage, see Configuring Kentico to store only media files on Azure storage.

Deleting .lock files

Applies to: WEB APPS CLOUD SERVICES

Quick fix

If the search functionality is not working properly, you can try deleting files with the .lock extension on the blob storage. Connect to your blob storage (for example through Cloudberry for Azure) and manually delete them.

Explanation

A blob storage is shared between all instances of Kentico. If your Azure environment (Cloud Services or Web Apps) operates in multiple instances, it is important to prevent collisions. Therefore, to prevent multiple instances from manipulating the same index at the same time and ensure exclusive write operations, Kentico uses locking. Every time the smart search needs to manipulate a search index, it creates a special file with the .lock extension. This file indicates that a search indexer is working with the index. When the work is done, the system deletes the file from the storage.

There are two different lock files in the blob storage:

  • searchtaskindexer.lock – located in the ~/App_Data/CMSModules/SmartSearch directory.
  • write.lock – located in the directory of specific search indexes.

In some cases, these lock files may not get deleted. This may happen when the process working with the search index ends unexpectedly. When this happens, processing of indexing tasks for locally stored indexes becomes stuck. The solution is to manually delete the lock files.

Prevention

On Azure Web Apps, this problem can occur again if you store the search indexes on the blob storage. We recommend that you either:

SmartSearchWorker role troubleshooting on Cloud Services

Applies to: CLOUD SERVICES

The SmartSearchWorker role is part of the Kentico Azure project designed to be used on Azure Cloud Services. If you are not running the Cloud Services or not using the SmartSearchWorker role, this section does not apply to your environment.

When you are convinced that every setting and every configuration is correct and the worker role is running correctly, but still not processing the search tasks, you can look for an error in the Event log.

If you do not find anything related in the event log, try the following troubleshooting practice:

  1. Enable remote desktop for the worker role.
  2. Connect to the worker role through the remote desktop access.
    1. Switch to the Roles and Instances tab.
    2. Select the worker role instance.
    3. Click Connect.
  3. Look into the Smart search log in App_Data/CMSModules/SmartSearch/workerlog.txt.
  4. Look into the Windows event log using Event viewer. Try looking for errors from around the time the worker role was starting.

Next steps can differ based on the error you find. But the error should provide you with enough information to start with.