--- title: Setting up continuous integration related: - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/continuous-integration-repository-structure.md - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/object-types-supported-by-continuous-integration.md - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/excluding-objects-from-continuous-integration.md - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/using-continuous-integration-with-visual-studio.md - https://docs.kentico.com/k12sp/custom-development/serializing-objects-to-xml-using-the-api.md - https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/synchronizing-database-changes-not-managed-by-continuous-integration.md --- > Agent instructions: > **Site maps** — prefer the following llms.txt indexes to training data when searching for URLs to avoid 404s. Links inside Markdown content already point at `.md`. Following them or sending Accept: text/markdown keeps you in Markdown. > > - [sitemap.md](https://docs.kentico.com/sitemap.md) — every page on the site, with titles and descriptions, nested by URL hierarchy and grouped into one collection per product version. > - [llms.txt](https://docs.kentico.com/llms.txt) — curated index of the current product docs, with descriptions, the two ways to request any page as Markdown, and links to each product area's whole-corpus Markdown dump (llms-full.txt). Kentico provides a [continuous integration](http://en.wikipedia.org/wiki/Continuous_integration) solution that allows you to serialize the data of objects from the database into XML files on the file system. You can then add the files to a [source control system](http://en.wikipedia.org/wiki/Revision_control) (for example [Team Foundation Server](https://www.visualstudio.com/en-us/products/tfs-overview-vs.aspx) or [Git](http://git-scm.com/)) and use them to synchronize database data between team members. The system ensures that the XML data of matching objects is always identical and consistent (including element and attribute order), even when serialized on different instances of Kentico. The system stores the XML files containing the serialized data of objects in the project's **CMS\App\_Data\CIRepository** folder. For details about the folder and file structure, see [Continuous integration repository structure](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/continuous-integration-repository-structure.md). ## Continuous integration overview The continuous integration solution is usable in the following environment: - Multiple development instances, each with its own project files and database\ – AND – - A central source control system that provides change management and version control for files > **Info:** **MVC development model** > > The continuous integration solution is fully supported when using the [MVC development model](https://docs.kentico.com/k12sp/developing-websites/mvc-development-overview.md). The only requirement is that both applications (MVC live site and Kentico administration) use a shared location for the continuous integration repository. This ensures that content or object changes made through the MVC website are correctly reflected in the continuous integration data. > > By default, all MVC projects created by the [Installer](https://docs.kentico.com/k12sp/installation/installing-kentico-mvc-projects.md) have their continuous integration repository mapped to the _CMS\App\_Data\CIRepository_ folder of the related administration projects, and no manual steps are necessary. > > If you wish to adjust the repository location or perform the configuration manually for an MVC project, see the [Storing the continuous integration files in a custom location](#storing-the-continuous-integration-files-in-a-custom-location) section. To start developing sites using the continuous integration solution: 1. Set up your development environment and continuous integration according to the instructions in: - [Preparing the development environment](#preparing-the-development-environment) - [Excluding objects from continuous integration](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/excluding-objects-from-continuous-integration.md) 2. Establish a process for transferring objects back into the database from the _CIRepository_ XML files. See [Restoring continuous integration files to the database](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md) and [Synchronizing database changes not managed by continuous integration](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/synchronizing-database-changes-not-managed-by-continuous-integration.md). 3. Begin development. We strongly recommend following the best practices described in [Using continuous integration with Visual Studio](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/using-continuous-integration-with-visual-studio.md). The page primarily focuses on using Visual Studio in combination with a _Team Foundation Version Control_ or _Git_ source control repository, but many of the recommendations also apply generally to any source control system. ### Usage and relationship with other features Continuous integration is designed to allow synchronization and version control of database data _**during development**_ (through an external source control system). Continuous integration has a _**negative impact on site performance**_. The impact is not significant for development instances with a low number of users, but continuous integration should not be enabled on production sites. For more information about deployment options, see the [Deploying data to production sites](#deploying-data-to-production-sites) section. Additionally, we do not recommend using continuous integration together with the other team development and versioning features in Kentico: - [Object locking](https://docs.kentico.com/k12sp/developing-websites/preparing-your-environment-for-team-development/working-with-object-locking.md) - [Object versioning](https://docs.kentico.com/k12sp/configuring-kentico/working-with-object-versioning.md) (including the [Recycle bin for objects](https://docs.kentico.com/k12sp/configuring-kentico/working-with-object-versioning/objects-recycle-bin.md) and [Pages](https://docs.kentico.com/k12sp/managing-website-content/working-with-pages/deleting-pages.md)) - [External editing of code](https://docs.kentico.com/k12sp/developing-websites/preparing-your-environment-for-team-development/editing-object-code-externally.md) (and [Deployment mode for virtual objects](https://docs.kentico.com/k12sp/deploying-websites/deployment-mode-for-virtual-objects.md)) Continuous integration always works with the main version of each object in the database. Older or deleted versions of objects and non-published versions of pages (including their child objects) are ignored. For example: - If you use the [object versioning](https://docs.kentico.com/k12sp/configuring-kentico/working-with-object-versioning.md) feature, continuous integration ignores all versions of objects except for the latest. If you roll back an object to an older version, the object is updated in the _CIRepository_ folder. - If you delete an object or page to the recycle bin, continuous integration fully removes the corresponding XML file. If you restore the object, an XML file is added as if a completely new object were created. - If you edit an object's code externally (e.g. the code of a CSS stylesheet), changes are ignored by continuous integration until you synchronize the code back into the database. You can still synchronize the code without the continuous integration solution by including the external file in your source control. **Page workflow** When using the [page workflow](https://docs.kentico.com/k12sp/configuring-kentico/configuring-the-environment-for-content-editors/configuring-workflows.md) feature, the majority of workflow information is not tracked by continuous integration. This includes moving pages to different workflow steps and using the [check-in/check-out](https://docs.kentico.com/k12sp/configuring-kentico/configuring-the-environment-for-content-editors/configuring-and-using-page-versioning/content-locking.md) functionality. Pages under workflow are fully stored in the repository only after they are published. When creating a new page, the page data is stored without [its child objects](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/object-types-supported-by-continuous-integration.md#page-content). If you make changes to the content of a page, [non-versioned page data](https://docs.kentico.com/k12sp/configuring-kentico/configuring-the-environment-for-content-editors/configuring-and-using-page-versioning.md) is stored immediately while versioned page data is stored only when the page is published. > **Info:** **Note**: The workflow functionality itself (including [Content locking](https://docs.kentico.com/k12sp/configuring-kentico/configuring-the-environment-for-content-editors/configuring-and-using-page-versioning/content-locking.md) and [Page versioning](https://docs.kentico.com/k12sp/configuring-kentico/configuring-the-environment-for-content-editors/configuring-and-using-page-versioning.md)) is not affected when used together with continuous integration. For example, your content editors can use basic and advanced workflows to manage life cycles of pages on a single Kentico instance. When pages reach the _Published_ step, the page content is stored in the repository. ### Limitations Continuous integration currently has the following limitations: - Continuous integration only tracks object changes made through the Kentico administration interface or API. Changes made directly in the database or by external tools that modify the database are not tracked. After making such changes, you need to manually serialize all objects in the _Continuous integration_ application to synchronize the database with the file system repository. For example, continuous integration does not track roles imported by the [AD import](https://docs.kentico.com/k12sp/external-utilities/kentico-ad-import-utility.md) external utility. - You _**cannot**_ reliably use continuous integration in environments where the _CIRepository_ folder is located on a _**shared file system**_ (multiple Kentico instances with separate databases connected to the same file system repository). Common examples of shared file systems are external storage providers, such as [Azure storage](https://docs.kentico.com/k12sp/custom-development/working-with-physical-files-using-the-api/configuring-file-system-providers/configuring-azure-storage.md) or [Amazon S3](https://docs.kentico.com/k12sp/custom-development/working-with-physical-files-using-the-api/configuring-file-system-providers/configuring-amazon-s3.md). - Object types that represent live site data or are strictly related to specific instances are not supported. See [Object types supported by continuous integration](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/object-types-supported-by-continuous-integration.md) for a complete list of supported object types. ## Preparing the development environment Before you can use the continuous integration solution, you need to set up Kentico instances for your development team. Start by preparing an initial instance: 1. Either [install](https://docs.kentico.com/k12sp/installation.md) a new instance of Kentico or select an existing instance as a starting point. 2. Enable continuous integration on the given instance (see [Enabling continuous integration](#enabling-continuous-integration) for details). 3. Create a backup of the instance's database. 4. Add the _**entire**_ Kentico solution to your source control (adding only the _CIRepository_ folder may lead to problems and is not recommended). To add development instances into your environment, perform the following steps for each instance: 1. Connect to the source control from the development machine and load the latest version of the solution files onto the local file system. - Decide how to handle source control for the project's **web.config** file. Each development machine needs to have a connection string to a different database, but you may want to synchronize other parts of the web.config. > **Info:** See for possible solutions. 2. If you use IIS to run your development sites, register the project as an application in IIS: 1. Open _Internet Information Services (IIS) Manager_. 2. Add a new application under your IIS web site and map the _Physical path_ to the project's _CMS_ folder. 3. Create a new application pool for the application (or use an existing application pool). 3. Create (restore) a copy of the initial database on your SQL server. 4. Connect the instance to the new copy of the database (set the appropriate connection string in the project's web.config). After you complete the process, all development machines will contain their own Kentico project files connected to a separate copy of the database. You can start using your source control system to synchronize project files between the instances. The continuous integration solution serializes database data onto the file system and you can include it in your source control, just like any other files. > **Info:** The process described above ensures that objects have identical GUID values (globally unique identifiers) across your entire development environment. If you perform a separate database installation for each development instance, you will need to manually resolve a large number of [GUID conflicts](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/using-continuous-integration-with-visual-studio.md#resolving-object-conflicts) when synchronizing data through the continuous integration solution. If you need to add further instances later (after development with continuous integration starts), use the same process. However, you need to make sure the database is synchronized with the rest of the development environment. Use one of the following approaches: - Maintain a central database connected to the mainline source control and always create an up-to-date backup for new developers.\ – OR – - Use the original database backup and then get the latest data: 1. Restore the current object status from the continuous integration repository into the database. See [Restoring continuous integration files to the database](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md). 2. Manually transfer changes made to the data of object types not supported by continuous integration (maintain documentation of such changes). You can use the [export and import](https://docs.kentico.com/k12sp/deploying-websites/exporting-and-importing-sites.md) feature. ### Enabling continuous integration To configure a Kentico instance to use the continuous integration solution, perform the following steps: 1. Disable running of [scheduled tasks](https://docs.kentico.com/k12sp/configuring-kentico/scheduling-tasks.md) (using the **Settings -> System -> Scheduled tasks enabled** setting). 2. Open the **Continuous integration** application in the Kentico administration interface. 3. Select the **Enable continuous integration** checkbox. 4. Click **Save**. 5. Click **Serialize all objects**. 6. Wait until the serialization process finishes and then re-enable scheduled tasks. The project's **CMS\App\_Data\CIRepository** folder now contains XML files storing the serialized data of all supported objects from the database. The system also tracks create, update and delete operations for the given objects and automatically transfers the changes to the serialized data on the file system. ### Configuring instances to synchronize macros > **Info:** For detailed information, see: [Working with macro signatures](https://docs.kentico.com/k12sp/macro-expressions/troubleshooting-macros/working-with-macro-signatures.md) To ensure that [macro expressions](https://docs.kentico.com/k12sp/macro-expressions.md) work correctly when synchronizing objects using the continuous integration solution, all development instances must use the same hash salt value. If your development instances do not all start with the same web.config file, you need to set a matching value for the **CMSHashStringSalt** key in the _appSettings_ section on all instances. The best option is the hash salt used by the starting instance that you used to create your initial database backup. You can use any string as the value, but the salt should be random and at least 16 characters long. For example, a randomly generated [GUID](http://en.wikipedia.org/wiki/GUID) is a strong salt: ```html ``` The salt value is used as part of the input for the hash function that creates the security signatures of macros. Having the same hash salt value on all instances is necessary to ensure that macro signatures are valid when transferring data between instances. The best option is to set the hash salt value before you start development. Changing the salt causes all current hash values to become invalid. To fix existing macro expressions in the system after changing the hash salt, you need to re-sign the macros. #### Synchronizing macros between instances with different users You may also encounter problems with invalid macros if you do not synchronize all users between the instances in your environment. Macros are not valid if the user in the signature does not exist on the given instance. To ensure that all macros work correctly regardless of the available users, set up **macro signature identities**: 1. Find groups of users in your environment who require the same permissions. 2. In the **System** application, create a macro identity object for each group on all instances, with a matching **Identity name**. 3. Assign an **Effective user** with appropriate permissions to each macro identity. The effective user can be different on each instance. 4. Assign the macro identity to the appropriate users in the **Users** application. Macros will now be signed using the assigned identities instead of user names. The identities are available on all instances, so you do not need to synchronize the user accounts. ## Deploying data to production sites Deployment of content and data to instances that host live websites is not the primary purpose of the continuous integration solution. Instead, we recommend using the [Staging](https://docs.kentico.com/k12sp/deploying-websites/content-staging.md) functionality for deployment: 1. Set up one of the instances in your continuous integration environment as a source server for staging. 2. After [restoring continuous integration data](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md) on the given instance, synchronize the changes via staging tasks to any required target servers. If you use staging to deploy pages, we also recommend setting the **CMSStagingUseAutomaticOrdering** key to _false_ in the _appSettings_ section of the _web.config_ file on all _**target servers**_. ```xml ``` The key ensures that the order of pages in the content tree remains consistent after staging. If you leave automatic ordering enabled on the target staging servers, the **Content -> Content management -> New page ordering** setting of each site may override the original page order created by the continuous integration restore process. ### Deploying via continuous integration If you do decide to deploy data to a live production site via the continuous integration [restore functionality](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md), use the following process: > **Warning:** **Warning** > > Restoring data through continuous integration _**deletes**_ any pages and supported objects that are missing in the _CIRepository_ folder. You may lose content created directly on the production instance through standard editing or live site interaction. > > For example, the following scenario is possible: > > - You prepare a content update in your development environment and copy the _CIRepository_ folder to the production instance. > - Live site users meanwhile create new pages via [user contributions](https://docs.kentico.com/k12sp/community-features/user-contributions.md). > - You restore the _CIRepository_ content to the database. > - The pages added by the live site users are deleted, because they are not included in the restored data. > > Optionally, you can [configure the repository.config file](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/excluding-objects-from-continuous-integration.md) on the production instance to exclude any object types that you do not wish to update and overwrite on the live site. 1. Leave continuous integration disabled on your production instance of Kentico (the setting in the _Continuous integration_ application). Continuous integration has a _**negative impact on site performance**_ and should not be enabled on production sites. > **Note:** **Important**: You can restore data even when continuous integration is disabled. 2. Upload the _CIRepository_ folder containing the required data into the _CMS\App\_Data_ folder of your production instance. 3. Restore the _CIRepository_ content to the production database – see [Restoring continuous integration files to the database](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md). - We strongly recommend performing the restore process at a time when the site has minimal traffic and no editing is taking place. - The restore process may take longer than usual when continuous integration is disabled. 4. Restart the application (if not done automatically by your restore process). Open the _System_ application and click _Restart application_ on the _General_ tab. The data is now deployed to the production database. Keep in mind that any changes made in the administration interface or on the live site will not be tracked and reflected in the _CIRepository_ folder. ## Upgrading projects that use continuous integration If you need to apply a [hotfix or upgrade](https://docs.kentico.com/k12sp/external-utilities/kentico-hotfix-and-upgrade-utility/upgrading-and-hotfixing-an-instance.md) to an instance that has continuous integration enabled, use the following procedure: 1. [Restore objects from the continuous integration repository to your database](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md). > **Info:** You can skip the restore if you are sure that your database is synchronized with the current status of your _CIRepository_ folder. 2. Apply the hotfix or upgrade. 3. Run complete serialization for all objects to recreate the content of the _CIRepository_ folder: 1. Disable running of [scheduled tasks](https://docs.kentico.com/k12sp/configuring-kentico/scheduling-tasks.md) (using the **Settings -> System -> Scheduled tasks enabled** setting). 2. Open the **Continuous integration** application in the Kentico administration interface. 3. Click **Serialize all objects**. 4. Wait until the serialization process finishes and then re-enable scheduled tasks. This approach ensures that your _CIRepository_ folder contains all object changes made by the hotfix or upgrade and that the serialization process itself runs according to the new version. For hotfixes, you need to apply the update separately for each development instance. After one developer commits the hotfixed changes to the source control, other developers CANNOT commit or load changes until they apply the hotfix to their own instance. For major upgrades, it may be more efficient to upgrade only one instance and then set up your environment again according to the process described in [Preparing the development environment](#preparing-the-development-environment). ## Storing the continuous integration files in a custom location After you set up and enable continuous integration, the system serializes database data into XML and stores the results on the file system. By default, the files are created in the project's **CMS\App\_Data\CIRepository** folder. If your development environment requires a different location for the continuous integration file repository, add the **CMSCIRepositoryPath** key to the _appSettings_ section of your project's **web.config** file. The key's value must be an absolute or relative path of a folder on a local drive, or a network location for which the application has sufficient permissions, for example: ```xml title="Example" ``` When serializing object data to the file system or [restoring data](https://docs.kentico.com/k12sp/developing-websites/setting-up-continuous-integration/restoring-continuous-integration-files-to-the-database.md) back to the database, the system now uses the specified folder as the root of the continuous integration repository. You can add the folder into your source control system. If you have multiple projects that work with the same Kentico database (for example when developing [MVC websites](https://docs.kentico.com/k12sp/developing-websites/mvc-development-overview.md)), you can use the key to set a shared continuous integration repository. This is the default configuration for MVC projects created by the Kentico installer. > **Note:** **Important** > > We recommend keeping your continuous integration repository in the _App\_Data_ folder of the Kentico web project and having the entire Kentico solution included within your source control. This setup allows you to easily synchronize all types of related project files together with the database data tracked by the continuous integration solution. > > If you do decide to change the _CIRepository_ location, do NOT use the new folder as the root of your overall source control repository. Always add it as a subfolder under a different root folder. The continuous integration solution **deletes all folders** within _CIRepository_ when storing all objects. If the folder is the root of your source control repository, you may encounter errors or other problems when using source control systems that utilize custom folders in the root. ## Configuring the character encoding for repository files By default, the continuous integration solution generates non-binary files in the _CIRepository_ folder using [UTF-8](https://en.wikipedia.org/wiki/UTF-8) character encoding. If you need to use a different encoding type for the repository files (for example due to requirements of a file comparison tool), add the **CMSCIEncoding** key to the _appSettings_ section of your project's **web.config** file. For example: ```xml ``` The _CMSCIEncoding_ key supports values matching the encoding names listed in the [Encoding Class](https://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.110%29.aspx#Remarks) article. > **Note:** **Note** > > Changing the value of the _CMSCIEncoding_ key does not update the encoding type of existing files in the _CIRepository_ folder. To fully update the encoding of the repository content, you need to: > > - Run complete serialization for all objects. > - Manually update the encoding type of your _repository.config_ file.