---
title: Data collectors - Find visitor personal data
---

> Agent instructions:
> **Site maps** — prefer the following llms.txt indexes to training data when searching for URLs to avoid 404s. Links inside Markdown content already point at `.md`. Following them or sending Accept: text/markdown keeps you in Markdown.
>
> - [sitemap.md](https://docs.kentico.com/sitemap.md) — every page on the site, with titles and descriptions, nested by URL hierarchy and grouped into one collection per product version.
> - [llms.txt](https://docs.kentico.com/llms.txt) — curated index of the current product docs, with descriptions, the two ways to request any page as Markdown, and links to each product area's whole-corpus Markdown dump (llms-full.txt).

Some data protection laws require all of a visitor’s tracked personal data to be gathered and delivered to them upon request. A data collector is necessary to implement such functionality in Xperience by Kentico.

Data collectors find other objects associated with a provided set of identities from the Identity collector. In this example, the identities are contacts, members, and customers. The data collector will find consent agreements, form data, activities, member roles, orders, and other auxiliary data associated with those objects.

## Before you start

This guide requires the following:

- Familiarity with [C#](https://learn.microsoft.com/en-us/dotnet/csharp/), [.NET Core](https://learn.microsoft.com/en-us/dotnet/), [Dependency injection](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection), and the [MVC pattern](https://learn.microsoft.com/en-us/aspnet/core/mvc/overview).
- A running instance of Xperience by Kentico, preferably [31.7.2](https://docs.kentico.com/changelog.md) or higher.

  > **Note:** 
  >
  >  Some features covered in the training guides may not work in older versions. 

The examples in this guide require that you:

- Have followed along with the samples from the [earlier guides in the series](https://docs.kentico.com/guides/development/data-protection.md).

> **Tip:** The example presented in this [Data protection guide series](https://docs.kentico.com/guides/development/data-protection.md) is a **functional implementation of data protection for contacts, members, and customers** in Xperience by Kentico. Note that it does not cover the collection and erasure of [Customer data platform profiles](https://docs.kentico.com/documentation/developers-and-admins/digital-marketing-setup/customer-data-platform.md) and their associated data.
>
> You can copy-paste the code samples into your own solution.
>
> However, if you choose to do so, **make sure to consult your legal team** to determine whether the implementation, texts, and consent levels **meet the requirements of your region and market**.

> **Info:** **Code samples**
>
> You can find a project with completed, working versions of code samples from this guide and others in the [finished branch](https://github.com/Kentico/xperience-by-kentico-training-guides/tree/finished) of the _Training guides_ repository.
>
> The [main branch](https://github.com/Kentico/xperience-by-kentico-training-guides) of the repository provides a starting point to code along with the guides.
>
> The code samples in this guide are for [.NET 8](https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotnet-8/overview) only.
>
> They come from a project that uses [implicit using directives](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/overview#implicit-using-directives). You may need to add additional `using` directives to your code if your project does not use this feature.

## Create data writers

Based on whether data collection is requested through the **Data portability** or **Right to access** tab in the Xperience UI, it will invoke registered personal data collectors with the `outputFormat` parameter set to a constant string value, either `PersonalDataFormat.HUMAN_READABLE` or `PersonalDataFormat.MACHINE_READABLE`.

Data collectors are meant to output a string result.

> **Tip:** While it is possible to include logic to print in different formats within the code that collects the data, separating it out into dedicated writer classes keeps the code clearer and less redundant.

### Lay the groundwork

Before you define the first writer, create a class that represents a column that should be collected and displayed, as well as an interface that both writers will implement.

1. In the _TrainingGuides.Web/Features/DataProtection/Collectors_ folder, create a `CollectedColumn` class.
2. Add properties to hold the display name and a code name.
3. Define a constructor that takes both properties as parameters.

   ```csharp title="CollectedColumn.cs"
   namespace TrainingGuides.Web.Features.DataProtection.Collectors;

   public class CollectedColumn
   {
       public string Name { get; }

       public string DisplayName { get; }

       public CollectedColumn(string name, string displayName)
       {
           Name = name;
           DisplayName = displayName;
       }
   }
   ```
4. Create a _Writers_ subfolder in _\~/Features/DataProtection_ and add an interface that both the human and machine-readable writer classes can implement.
5. Similar to the [example in the documentation](https://docs.kentico.com/documentation/developers-and-admins/data-protection/personal-data-collection.md#implement-writer-classes), add signatures for void methods that create beginning and ending sections and info objects.
6. Create an additional method signature for writing values.
7. Add a method signature for a string method that outputs the result of the writer’s operations.

   ```csharp title="IPersonalDataWriter.cs"
   using CMS.DataEngine;
   using TrainingGuides.Web.Features.DataProtection.Collectors;

   namespace TrainingGuides.Web.Features.DataProtection.Writers;

   public interface IPersonalDataWriter : IDisposable
   {
       void WriteStartSection(string sectionName, string sectionDisplayName);
       void WriteBaseInfo(BaseInfo baseInfo, List<CollectedColumn> columns, Func<string, object, object>? valueTransformationFunction = null);
       void WriteSectionValue(string sectionName, string sectionDisplayName, string value);
       void WriteEndSection();
       string GetResult();
   }
   ```

### Add a human-readable writer

1. In the same folder, define a `HumanReadablePersonalDataWriter` that implements the `IPersonalDataWriter` interface.
2. Define it similarly to the [documentation example](https://docs.kentico.com/documentation/developers-and-admins/data-protection/personal-data-collection.md#implement-writer-classes), utilizing a private string builder property.
3. Add a method called `WriteSectionValue` that prints a key and value, ignoring the section name, which the machine-readable counterpart can utilize.
4. Leave the `Dispose` method empty.

   > **Info:** You don't need the method here, but it is important for the machine-readable data writer covered in the [next step](#create-a-machine-readable-writer).

```csharp title="HumanReadablePersonalDataWriter.cs"
using System.Globalization;
using System.Text;
using CMS.Base;
using CMS.DataEngine;
using TrainingGuides.Web.Features.DataProtection.Collectors;

namespace TrainingGuides.Web.Features.DataProtection.Writers;

public class HumanReadablePersonalDataWriter : IPersonalDataWriter
{
    private readonly StringBuilder stringBuilder;
    private int indentationLevel;
    private bool ignoreNewLine;

    private static readonly string decimalPrecision = new('#', 26);
    private static readonly string decimalFormat = "{0:0.00" + decimalPrecision + "}";

    public CultureInfo Culture { get; set; } = new CultureInfo(SystemContext.SYSTEM_CULTURE_NAME);

    public HumanReadablePersonalDataWriter()
    {
        stringBuilder = new StringBuilder();
        indentationLevel = 0;
        ignoreNewLine = false;
    }

    public void WriteStartSection(string sectionName, string sectionDisplayName)
    {
        ignoreNewLine = false;
        Indent();

        stringBuilder.AppendLine(sectionDisplayName + ": ");
        indentationLevel++;
    }

    private void Indent() => stringBuilder.Append('\t', indentationLevel);

    public void WriteBaseInfo(BaseInfo baseInfo, List<CollectedColumn> columns,
        Func<string, object, object>? valueTransformationFunction = null)
    {
        if (baseInfo == null)
        {
            throw new ArgumentNullException(nameof(baseInfo));
        }

        foreach (var column in columns)
        {
            string columnName = column.Name;
            string columnDisplayName = column.DisplayName;

            if (string.IsNullOrWhiteSpace(columnDisplayName) ||
                columnName.Equals(baseInfo.TypeInfo.IDColumn, StringComparison.Ordinal) ||
                columnName.Equals(baseInfo.TypeInfo.GUIDColumn, StringComparison.Ordinal))
            {
                continue;
            }

            object value = baseInfo.GetValue(columnName);

            if (value == null)
            {
                continue;
            }

            if (valueTransformationFunction != null)
            {
                value = valueTransformationFunction(columnName, value);
            }

            WriteKeyValue(columnDisplayName, value);
        }
    }

    public void WriteSectionValue(string sectionName, string sectionDisplayName, string value)
    {
        Indent();

        stringBuilder.Append($"{sectionDisplayName}: {value}");
        stringBuilder.AppendLine();
    }

    private void WriteKeyValue(string keyName, object value)
    {
        Indent();
        stringBuilder.Append($"{keyName}: ");

        string format = "{0}";

        if (value is decimal)
        {
            format = decimalFormat;
        }

        stringBuilder.AppendFormat(Culture, format, value);
        stringBuilder.AppendLine();

        ignoreNewLine = true;
    }

    public void WriteEndSection()
    {
        indentationLevel--;

        if (ignoreNewLine)
            return;

        Indent();
        stringBuilder.AppendLine();
        ignoreNewLine = true;
    }

    public string GetResult() => stringBuilder.ToString();

    public void Dispose()
    {
    }
}
```

### Create a machine-readable writer

Add an XML writer that closely mirrors the [example in the documentation](https://docs.kentico.com/documentation/developers-and-admins/data-protection/personal-data-collection.md#implement-writer-classes).

> **Note:** In this case, `WriteSectionValue` needs to use the `sectionName` property for the XML tags rather than the display name.

```csharp title="XmlPersonalDataWriter.cs"
using System.Text;
using System.Xml;
using CMS.DataEngine;
using CMS.Helpers;
using TrainingGuides.Web.Features.DataProtection.Collectors;

namespace TrainingGuides.Web.Features.DataProtection.Writers;

public class XmlPersonalDataWriter : IPersonalDataWriter
{
    private readonly StringBuilder stringBuilder;
    private readonly XmlWriter xmlWriter;

    public XmlPersonalDataWriter()
    {
        stringBuilder = new StringBuilder();
        xmlWriter = XmlWriter.Create(stringBuilder, new XmlWriterSettings { Indent = true, OmitXmlDeclaration = true });
    }

    public void WriteStartSection(string sectionName, string sectionDisplayName) =>
        xmlWriter.WriteStartElement(TransformElementName(sectionName));

    private string TransformElementName(string originalName) => originalName.Replace('.', '_');

    public void WriteBaseInfo(BaseInfo baseInfo, List<CollectedColumn> columns,
        Func<string, object, object>? valueTransformationFunction = null)
    {
        if (baseInfo == null)
        {
            throw new ArgumentNullException(nameof(baseInfo));
        }

        foreach (var columnTuple in columns)
        {
            string columnName = columnTuple.Name;

            if (string.IsNullOrWhiteSpace(columnTuple.DisplayName))
            {
                continue;
            }

            object value = baseInfo.GetValue(columnName);
            if (value == null)
            {
                continue;
            }

            if (valueTransformationFunction != null)
            {
                value = valueTransformationFunction(columnName, value);
            }

            xmlWriter.WriteStartElement(columnName);
            xmlWriter.WriteValue(XmlHelper.ConvertToString(value, value.GetType()));
            xmlWriter.WriteEndElement();
        }
    }

    public void WriteSectionValue(string sectionName, string sectionDisplayName, string value)
    {
        xmlWriter.WriteStartElement(sectionName);
        xmlWriter.WriteString(value);
        xmlWriter.WriteEndElement();
    }

    public void WriteEndSection() => xmlWriter.WriteEndElement();

    public string GetResult()
    {
        xmlWriter.Flush();

        return stringBuilder.ToString();
    }

    public void Dispose() => xmlWriter.Dispose();
}
```

## Collect data

### Prepare to collect forms

While many of the objects that need to be collected are relatively straightforward, containing a set of known columns, including a reference to the _contact_ type, form submissions are more complicated.

The columns of forms are defined by editors in the Xperience administration interface. For this reason, there is no way to anticipate which columns the forms will consist of, and they cannot be explicitly referenced in code in advance.

You need logic to iterate through the forms on the website and determine if any of them have one or more email columns.

Create a class called `FormDefinition` in the _DataProtection/Collectors_ folder, which contains a list of email column names and a list of other columns to display.

```csharp title="FormDefinition.cs"
namespace TrainingGuides.Web.Features.DataProtection.Collectors;

public class FormDefinition
{
    public const string FORM_CONSENT_COLUMN_NAME = "Consent";
    public IEnumerable<string> EmailColumns { get; }
    public List<CollectedColumn> FormColumns { get; }

    public FormDefinition(List<string> emailColumns, List<CollectedColumn> formColumns)
    {
        EmailColumns = emailColumns;
        FormColumns = formColumns;
    }
}
```

When it comes to populating `FormDefinition` objects, another service class will be useful.

1. In the _\~/Features/DataProtection/Services_ folder of the _TrainingGuides.Web_ project, add an `IFormCollectionService` interface and a `FormCollectionService` class that implements it.
2. Add a public method called `GetForms`, which retrieves all of the forms in the database and converts them to a `Dictionary`, with `Guid` objects as keys and `FormDefinition` objects as values. Within this method, do the following:
   a. Find the `DataClassInfo` object associated with each form, and parse through the XML of the `ClassFormDefinition` and `ClassContactMapping` columns.
   b. Use this data to assemble a collection of email fields. For this example, target fields that are either mapped to the `ContactEmail` field or use the `Kentico.EmailInput` form component.
   > **Tip:** In your projects, you can define other sets of conditions to broaden or narrow the search for matching email addresses.
   c. Make a separate collection of fields that are not email fields or ID fields.
   d. Use these two collections to assemble `FormDefinition` for the form and add it to the dictionary.
3. Create another public method called `GetBizFormItems` to retrieve all form submissions associated with a contact. The method needs to do the following:
   a. Construct an `ObjectQuery<BizFormItem> bizFormItems` to retrieve all form submissions that meet one of the following conditions:
   i. The submission has an email field with one of the provided email addresses.\
   or
   ii. The submission references one of the supplied `ConsentAgreementInfo` objects.
   b. Use a `FormDefinition` parameter to determine which email fields to check for the supplied addresses.
   c. Use a `DataClassInfo` object to determine items of which form to retrieve.

```csharp title="IFormCollectionService.cs"
using System.Data;
using CMS.DataEngine;
using CMS.DataProtection;
using CMS.OnlineForms;
using TrainingGuides.Web.Features.DataProtection.Collectors;

namespace TrainingGuides.Web.Features.DataProtection.Services;

public interface IFormCollectionService
{
    Dictionary<Guid, FormDefinition> GetForms();
    ObjectQuery<BizFormItem> GetBizFormItems(ICollection<string> emails, ObjectQuery<ConsentAgreementInfo> consentAgreementGuids, DataRow row, FormDefinition formDefinition);
}
```

```csharp title="FormCollectionService.cs"
using System.Data;
using System.Xml.Linq;
using CMS.DataEngine;
using CMS.DataProtection;
using CMS.OnlineForms;
using TrainingGuides.Web.Features.DataProtection.Collectors;

namespace TrainingGuides.Web.Features.DataProtection.Services;

public class FormCollectionService : IFormCollectionService
{
    private readonly IInfoProvider<BizFormInfo> bizFormInfoProvider;

    public FormCollectionService(IInfoProvider<BizFormInfo> bizFormInfoProvider)
    {
        this.bizFormInfoProvider = bizFormInfoProvider;
    }

    private bool IsField(XElement element) => element.Name.ToString() == "field";

    private bool UsesComponent(XElement fieldElement, string componentIdentifier)
    {
        var matchingComponents = fieldElement
            .Descendants()
            .Where(desc => desc.Name == "componentidentifier" && desc.Value == componentIdentifier);
        return matchingComponents.Count() > 0;
    }

    private bool IsInList(XElement fieldElement, IEnumerable<string> list) =>
        list.Contains(fieldElement.Attribute("column")?.Value, StringComparer.OrdinalIgnoreCase);

    private bool IsNotInCollection(XElement fieldElement, IEnumerable<XElement> excludedFieldElements) =>
        !excludedFieldElements.Contains(fieldElement);

    private bool IsNotID(XElement fieldElement)
    {
        string? isPK = fieldElement.Attribute("isPK")?.Value;
        return !bool.Parse(isPK ?? "false");
    }

    private string GetCaption(XElement fieldElement) =>
        fieldElement
            .Descendants()
            .Where(desc => desc.Name == "fieldcaption")
            .FirstOrDefault()?
            .Value
        ?? string.Empty;

    /// <summary>
    /// Gets information about all forms in the database, including which of the form's fields are email fields
    /// </summary>
    /// <returns>A <see cref="Dictionary{TKey, TValue}"/> mapping a form's <see cref="Guid"/> to a <see cref="FormDefinition"/>, which specifies the form's email fields and non-email fields.</returns>
    public Dictionary<Guid, FormDefinition> GetForms()
    {
        var result = new Dictionary<Guid, FormDefinition>();
        var bizForms = bizFormInfoProvider.Get();
        foreach (var bizForm in bizForms)
        {
            var dataClassInfo = DataClassInfoProvider.GetDataClassInfo(bizForm.FormClassID);
            if (dataClassInfo == null || string.IsNullOrEmpty(dataClassInfo.ClassFormDefinition))
            {
                continue;
            }

            IEnumerable<string> mappedEmailFields = [];

            if (!string.IsNullOrEmpty(dataClassInfo.ClassContactMapping))
            {
                var contactMapping = XElement.Parse(dataClassInfo.ClassContactMapping);

                //gets lowercase names of fields mapped to contact email (there should be only one unless they mess with the database directly)
                mappedEmailFields = contactMapping?.Elements()
                    .Where(field => field.Attribute("column")?.Value == "ContactEmail")
                    .Select(field => field.Attribute("mappedtofield")?.Value ?? string.Empty)
                ?? [];
            }

            var formDefinition = XElement.Parse(dataClassInfo.ClassFormDefinition);

            //gets all form fields which are either mapped to the ContactEmail column, or which use the Kentico.EmailInput form control
            var emailFields = formDefinition?.Elements()
                .Where(element => IsField(element) && (IsInList(element, mappedEmailFields) || UsesComponent(element, "Kentico.EmailInput"))) ?? [];

            //gets the remaining fields
            var otherFields = formDefinition?.Elements()
                .Where(element => IsField(element) && IsNotInCollection(element, emailFields) && IsNotID(element)) ?? [];

            result.Add(
                bizForm.FormGUID,
                new FormDefinition(
                    emailFields
                        .Select(element => element.Attribute("column")?.Value ?? string.Empty)
                        .ToList(),
                    otherFields
                        .Select(element => new CollectedColumn(element.Attribute("column")?.Value ?? string.Empty, GetCaption(element)))
                        .ToList()
                    )
                );
        }
        return result;
    }

    /// <summary>
    /// Gets an object query for all bizform items with any of the supplied emails in any of its email columns, or with any of the supplied consent agreements
    /// </summary>
    /// <param name="emails">The email addresses to look for</param>
    /// <param name="consentAgreementGuids">The guids of consents agreed to by the contact</param>
    /// <param name="row">a DataRow representing the DataClassInfo associated with the given form</param>
    /// <param name="formDefinition">The form definition of the supplied bizform</param>
    /// <returns>An <see cref="ObjectQuery"/>for all <see cref="BizFormItem"/> objects related to the provided contact through a consent agreement or email address</returns>
    public ObjectQuery<BizFormItem> GetBizFormItems(ICollection<string> emails, ObjectQuery<ConsentAgreementInfo> consentAgreementGuids, DataRow row, FormDefinition formDefinition)
    {
        var formClass = new DataClassInfo(row);

        var bizFormItems = BizFormItemProvider.GetItems(formClass.ClassName);
        foreach (string column in formDefinition.EmailColumns)
        {
            bizFormItems.Or().WhereIn(column, emails);
        }

        var consentColumns = formDefinition.FormColumns
            .Where(column => column.Name
            .Contains(FormDefinition.FORM_CONSENT_COLUMN_NAME)).Select(column => column.Name);

        if (formClass.ClassFormDefinition.Contains(FormDefinition.FORM_CONSENT_COLUMN_NAME))
        {
            foreach (string columnName in consentColumns)
            {
                bizFormItems.Or().WhereIn(columnName, consentAgreementGuids);
            }
        }
        return bizFormItems;
    }
}
```

> **Note:** Remember to **register the service with the dependency injection container** in `AddTrainingGuidesServices` method inside _TrainingGuides.Web/ServiceCollectionExtensions.cs_.
>
> ```csharp title="ServiceCollectionExtensions.cs"
> // ... existing code ...
> using TrainingGuides.Web.Features.DataProtection.Services;
> // ... existing code ...
> services.AddSingleton<IFormCollectionService, FormCollectionService>();
> // ... existing code ...
> ```

### Gather the data

You have prepared what you need to gather form submissions. In the next steps, you will add the main collection logic to your project.

1. In the _TrainingGuides.Web/Features/DataProtection/Collectors_ folder, create a class called `DataCollectorCore`.
   > **Info:** This class will hold all of the data collection logic, relying on the `IPersonalDataWriter` interface and separating the data collection code from the code that determines which writer to use.
2. Define private variables to hold the data writer, lists of the relevant columns of all the classes to collect, and a dictionary of forms populated by the `FormCollectionService` from earlier.
3. Add transformation methods to turn certain types of raw data references into an informative format.
4. Populate the writer field from a parameter in the constructor.
5. Define the `CollectData` method with a `string?` return type, using the data writer to print details about all the object types.

   > **Note:** **Relationships between object types**
   >
   > Object types may be more complicated than they appear at a glance. For example, alongside consents, you must find associated consent agreements (`ConsentAgreementInfo`) and consent archives (`ConsentArchiveInfo`).
   >
   > Additionally, some related objects may not have direct references. Look for **form submissions, customer addresses, and order addresses** that use the provided email address, **even if no contact or customer exists**. It is possible for visitors with no associated contact to submit a form, and for customers to enter addresses of other people.

```csharp title="DataCollectorCore.cs"
using System.Data;
using System.Xml;
using CMS.Activities;
using CMS.Commerce;
using CMS.ContactManagement;
using CMS.DataEngine;
using CMS.DataProtection;
using CMS.Globalization;
using CMS.Helpers;
using CMS.Membership;
using CMS.OnlineForms;
using TrainingGuides.Web.Features.DataProtection.Services;
using TrainingGuides.Web.Features.DataProtection.Writers;

namespace TrainingGuides.Web.Features.DataProtection.Collectors;

public class DataCollectorCore
{
    private readonly List<CollectedColumn> contactInfoColumns =
    [
        new CollectedColumn("ContactFirstName", "First name"),
        new CollectedColumn("ContactMiddleName", "Middle name"),
        new CollectedColumn("ContactLastName", "Last name"),
        new CollectedColumn("ContactJobTitle", "Job title"),
        new CollectedColumn("ContactAddress1", "Address"),
        new CollectedColumn("ContactCity", "City"),
        new CollectedColumn("ContactZIP", "ZIP"),
        new CollectedColumn("ContactStateID", ""),
        new CollectedColumn("ContactCountryID", ""),
        new CollectedColumn("ContactMobilePhone", "Mobile phone"),
        new CollectedColumn("ContactBusinessPhone", "Business phone"),
        new CollectedColumn("ContactEmail", "Email"),
        new CollectedColumn("ContactBirthday", "Birthday"),
        new CollectedColumn("ContactGender", "Gender"),
        new CollectedColumn("ContactNotes", "Notes"),
        new CollectedColumn("ContactGUID", "GUID"),
        new CollectedColumn("ContactLastModified", "Last modified"),
        new CollectedColumn("ContactCreated", "Created"),
        new CollectedColumn("ContactCampaign", "Campaign"),
        new CollectedColumn("ContactCompanyName", "Company name")
    ];

    private readonly List<CollectedColumn> consentAgreementInfoColumns =
    [
        new CollectedColumn("ConsentAgreementGuid", "GUID"),
        new CollectedColumn("ConsentAgreementRevoked", "Consent action"),
        new CollectedColumn("ConsentAgreementTime", "Performed on")
    ];

    private readonly List<CollectedColumn> consentInfoColumns =
    [
        new CollectedColumn("ConsentGUID", "GUID"),
        new CollectedColumn("ConsentDisplayName", "Consent name"),
        new CollectedColumn("ConsentContent", "Full text")
    ];

    private readonly List<CollectedColumn> consentArchiveInfoColumns =
    [
        new CollectedColumn("ConsentArchiveGUID", "GUID"),
        new CollectedColumn("ConsentArchiveContent", "Full text")
    ];

    private readonly List<CollectedColumn> activityInfoColumns =
    [
        new CollectedColumn("ActivityId", "ID"),
        new CollectedColumn("ActivityCreated", "Created"),
        new CollectedColumn("ActivityType", "Type"),
        new CollectedColumn("ActivityUrl", "URL"),
        new CollectedColumn("ActivityTitle", "Title"),
        new CollectedColumn("ActivityItemId", "")
    ];

    private readonly List<CollectedColumn> countryInfoColumns =
    [
        new CollectedColumn("CountryDisplayName", "Country")
    ];

    private readonly List<CollectedColumn> stateInfoColumns =
    [
        new CollectedColumn("StateDisplayName", "State")
    ];

    private readonly List<CollectedColumn> contactGroupInfoColumns =
    [
        new CollectedColumn("ContactGroupGUID", "GUID"),
        new CollectedColumn("ContactGroupName", "Contact group name"),
        new CollectedColumn("ContactGroupDescription", "Contact group description")
    ];

    private readonly List<CollectedColumn> memberInfoColumns =
    [
        new CollectedColumn("MemberEmail", "Email"),
        new CollectedColumn("MemberEnabled", "Enabled"),
        new CollectedColumn("MemberCreated", "Created"),
        new CollectedColumn("MemberGuid", "GUID"),
        new CollectedColumn("MemberName", "Name"),
        new CollectedColumn("MemberIsExternal", "Is external"),
        new CollectedColumn("GuidesMemberGivenName", "Given name"),
        new CollectedColumn("GuidesMemberFamilyName", "Family name"),
        new CollectedColumn("GuidesMemberFamilyNameFirst", "Family name first"),
        new CollectedColumn("GuidesMemberFavoriteCoffee", "Favorite coffee")
    ];

    private readonly List<CollectedColumn> memberRoleInfoColumns =
    [
        new CollectedColumn("MemberRoleGUID", "GUID"),
        new CollectedColumn("MemberRoleDisplayName", "Name"),
        new CollectedColumn("MemberRoleDescription", "Description"),
        new CollectedColumn("GuidesRoleBenefitsSummary", "Benefits")
    ];

    private readonly List<CollectedColumn> customerInfoColumns =
    [
        new CollectedColumn("CustomerGUID", "GUID"),
        new CollectedColumn("CustomerCreatedWhen", "Created"),
        new CollectedColumn("CustomerFirstName", "First name"),
        new CollectedColumn("CustomerLastName", "Last name"),
        new CollectedColumn("CustomerEmail", "Email"),
        new CollectedColumn("CustomerPhone", "Phone")
    ];

    private readonly List<CollectedColumn> orderInfoColumns =
    [
        new CollectedColumn("OrderGUID", "GUID"),
        new CollectedColumn("OrderNumber", "Order number"),
        new CollectedColumn("OrderCreatedWhen", "Created"),
        new CollectedColumn("OrderModifiedWhen", "Last modified"),
        new CollectedColumn("OrderTotalPrice", "Total price"),
        new CollectedColumn("OrderTotalShipping", "Total shipping"),
        new CollectedColumn("OrderTotalTax", "Total tax"),
        new CollectedColumn("OrderGrandTotal", "Grand total"),
        new CollectedColumn("OrderPaymentMethodDisplayName", "Payment method"),
        new CollectedColumn("OrderShippingMethodDisplayName", "Shipping method"),
        new CollectedColumn("OrderShippingMethodPrice", "Shipping method price")
    ];

    private readonly List<CollectedColumn> orderItemInfoColumns =
    [
        new CollectedColumn("OrderItemGUID", "GUID"),
        new CollectedColumn("OrderItemSKU", "SKU"),
        new CollectedColumn("OrderItemName", "Name"),
        new CollectedColumn("OrderItemQuantity", "Quantity"),
        new CollectedColumn("OrderItemUnitPrice", "Unit price"),
        new CollectedColumn("OrderItemTotalPrice", "Total price"),
        new CollectedColumn("OrderItemTotalTax", "Total tax"),
        new CollectedColumn("OrderItemTaxRate", "Tax rate")
    ];

    private readonly List<CollectedColumn> customerAddressInfoColumns =
    [
        new CollectedColumn("CustomerAddressGUID", "GUID"),
        new CollectedColumn("CustomerAddressFirstName", "First name"),
        new CollectedColumn("CustomerAddressLastName", "Last name"),
        new CollectedColumn("CustomerAddressCompany", "Company"),
        new CollectedColumn("CustomerAddressEmail", "Email"),
        new CollectedColumn("CustomerAddressPhone", "Phone"),
        new CollectedColumn("CustomerAddressLine1", "Address line 1"),
        new CollectedColumn("CustomerAddressLine2", "Address line 2"),
        new CollectedColumn("CustomerAddressCity", "City"),
        new CollectedColumn("CustomerAddressZip", "ZIP"),
        new CollectedColumn("CustomerAddressCountryID", ""),
        new CollectedColumn("CustomerAddressStateID", "")
    ];

    private readonly List<CollectedColumn> orderAddressInfoColumns =
    [
        new CollectedColumn("OrderAddressGUID", "GUID"),
        new CollectedColumn("OrderAddressType", "Address type"),
        new CollectedColumn("OrderAddressFirstName", "First name"),
        new CollectedColumn("OrderAddressLastName", "Last name"),
        new CollectedColumn("OrderAddressCompany", "Company"),
        new CollectedColumn("OrderAddressEmail", "Email"),
        new CollectedColumn("OrderAddressPhone", "Phone"),
        new CollectedColumn("OrderAddressLine1", "Address line 1"),
        new CollectedColumn("OrderAddressLine2", "Address line 2"),
        new CollectedColumn("OrderAddressCity", "City"),
        new CollectedColumn("OrderAddressZip", "ZIP"),
        new CollectedColumn("OrderAddressCountryID", ""),
        new CollectedColumn("OrderAddressStateID", "")
    ];

    private readonly IPersonalDataWriter personalDataWriter;
    private readonly IFormCollectionService formCollectionService;
    private readonly Dictionary<Guid, FormDefinition> forms;

    private readonly IInfoProvider<ConsentAgreementInfo> consentAgreementInfoProvider;
    private readonly IInfoProvider<ActivityInfo> activityInfoProvider;
    private readonly IInfoProvider<BizFormInfo> bizFormInfoProvider;
    private readonly IInfoProvider<CountryInfo> countryInfoProvider;
    private readonly IInfoProvider<StateInfo> stateInfoProvider;
    private readonly IInfoProvider<OrderInfo> orderInfoProvider;
    private readonly IInfoProvider<OrderItemInfo> orderItemInfoProvider;
    private readonly IInfoProvider<MemberRoleInfo> memberRoleInfoProvider;
    private readonly IInfoProvider<MemberRoleMemberInfo> memberRoleMemberInfoProvider;
    private readonly IInfoProvider<CustomerAddressInfo> customerAddressInfoProvider;
    private readonly IInfoProvider<OrderAddressInfo> orderAddressInfoProvider;

    public DataCollectorCore(IPersonalDataWriter personalDataWriter,
        IFormCollectionService formCollectionService,
        IInfoProvider<ConsentAgreementInfo> consentAgreementInfoProvider,
        IInfoProvider<ActivityInfo> activityInfoProvider,
        IInfoProvider<BizFormInfo> bizFormInfoProvider,
        IInfoProvider<CountryInfo> countryInfoProvider,
        IInfoProvider<StateInfo> stateInfoProvider,
        IInfoProvider<OrderInfo> orderInfoProvider,
        IInfoProvider<OrderItemInfo> orderItemInfoProvider,
        IInfoProvider<MemberRoleInfo> memberRoleInfoProvider,
        IInfoProvider<MemberRoleMemberInfo> memberRoleMemberInfoProvider,
        IInfoProvider<CustomerAddressInfo> customerAddressInfoProvider,
        IInfoProvider<OrderAddressInfo> orderAddressInfoProvider)
    {
        this.personalDataWriter = personalDataWriter;
        this.formCollectionService = formCollectionService;
        this.consentAgreementInfoProvider = consentAgreementInfoProvider;
        this.activityInfoProvider = activityInfoProvider;
        this.bizFormInfoProvider = bizFormInfoProvider;
        this.countryInfoProvider = countryInfoProvider;
        this.stateInfoProvider = stateInfoProvider;
        this.orderInfoProvider = orderInfoProvider;
        this.orderItemInfoProvider = orderItemInfoProvider;
        this.memberRoleInfoProvider = memberRoleInfoProvider;
        this.memberRoleMemberInfoProvider = memberRoleMemberInfoProvider;
        this.customerAddressInfoProvider = customerAddressInfoProvider;
        this.orderAddressInfoProvider = orderAddressInfoProvider;
        forms = this.formCollectionService.GetForms();
    }

    public string? CollectData(IEnumerable<BaseInfo> identities)
    {
        var contacts = identities.OfType<ContactInfo>().ToList();
        var members = identities.OfType<MemberInfo>().ToList();
        var customers = identities.OfType<CustomerInfo>().ToList();

        if (!(contacts.Any() || members.Any() || customers.Any()))
        {
            return null;
        }

        personalDataWriter.WriteStartSection("AllData", "All visitor data");

        string before = personalDataWriter.GetResult();

        if (contacts.Any())
        {
            var contactIDs = contacts.Select(c => c.ContactID).ToList();
            var contactEmails = contacts.Select(c => c.ContactEmail).ToList();

            var contactActivities = activityInfoProvider.Get()
                .Columns(activityInfoColumns.Select(t => t.Name))
                .WhereIn("ActivityContactID", contactIDs).ToList();

            // Includes plain contact groups and recipient lists
            var contactContactGroups = contacts
                .SelectMany(c => c.ContactGroups)
                .GroupBy(c => c.ContactGroupID)
                .Select(group => group.First());

            personalDataWriter.WriteStartSection("OnlineMarketingData", "Online marketing data");
            WriteContacts(contacts);
            WriteConsents(contactIDs);
            WriteContactActivities(contactActivities);
            WriteContactGroups(contactContactGroups);
            WriteSubmittedFormsData(contactEmails, contactIDs);
            personalDataWriter.WriteEndSection();
        }
        if (members.Any())
        {
            personalDataWriter.WriteStartSection("MembershipData", "Membership data");
            WriteMembers(members);
            WriteMemberRoles(members);
            personalDataWriter.WriteEndSection();
        }
        if (customers.Any())
        {
            var customerIds = customers.Select(c => c.CustomerID).ToList();
            var customerEmails = customers.Select(c => c.CustomerEmail).ToList();

            personalDataWriter.WriteStartSection("CommerceData", "Commerce data");
            WriteCustomers(customers);
            WriteCustomerOrders(customerIds);
            WriteThirdPartyCustomerAddresses(customerEmails, customerIds);
            WriteThirdPartyOrderAddresses(customerEmails, customerIds);
            personalDataWriter.WriteEndSection();
        }

        string after = personalDataWriter.GetResult();

        //return nothing if the previous methods did not find any data to write
        if (string.IsNullOrWhiteSpace(after) || after.Equals(before))
        {
            return null;
        }

        personalDataWriter.WriteEndSection();

        return personalDataWriter.GetResult();
    }

    private object TransformGenderValue(string columnName, object columnValue)
    {
        if (columnName.Equals("ContactGender", StringComparison.InvariantCultureIgnoreCase))
        {
            int? gender = columnValue as int?;
            return gender switch
            {
                1 => "male",
                2 => "female",
                _ => "undefined",
            };
        }

        return columnValue;
    }

    private object TransformConsentText(string columnName, object columnValue)
    {
        if (columnName.Equals("ConsentContent", StringComparison.InvariantCultureIgnoreCase) ||
            columnName.Equals("ConsentArchiveContent", StringComparison.InvariantCultureIgnoreCase))
        {
            var consentXml = new XmlDocument();
            consentXml.LoadXml((columnValue as string) ?? string.Empty);

            var xmlNode =
                consentXml.SelectSingleNode("/ConsentContent/ConsentLanguageVersions/ConsentLanguageVersion/FullText");

            string result = HTMLHelper.StripTags(xmlNode?.InnerText);

            return result;
        }

        return columnValue;
    }

    private object TransformConsentAction(string columnName, object columnValue)
    {
        if (columnName.Equals("ConsentAgreementRevoked", StringComparison.InvariantCultureIgnoreCase))
        {
            bool revoked = (bool)columnValue;

            return revoked ? "Revoked" : "Agreed";
        }

        return columnValue;
    }

    private void WriteContacts(IEnumerable<ContactInfo> contacts)
    {
        foreach (var contactInfo in contacts.Where(contact => contact.ContactID > 0))
        {
            personalDataWriter.WriteStartSection(ContactInfo.OBJECT_TYPE, "Contact");
            personalDataWriter.WriteBaseInfo(contactInfo, contactInfoColumns, TransformGenderValue);

            WriteCountryAndOrState(contactInfo.ContactCountryID, contactInfo.ContactStateID);

            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteCountryAndOrState(int countryId, int stateId)
    {
        if (countryId != 0)
        {
            personalDataWriter.WriteBaseInfo(countryInfoProvider.Get(countryId), countryInfoColumns);
        }

        if (stateId != 0)
        {
            personalDataWriter.WriteBaseInfo(stateInfoProvider.Get(stateId), stateInfoColumns);
        }
    }

    private void WriteConsents(ICollection<int> contactIDs)
    {
        var consentsDataSet = consentAgreementInfoProvider.Get()
            .Source(s => s.Join<ConsentInfo>("CMS_ConsentAgreement.ConsentAgreementConsentID", "ConsentID"))
            .Source(s =>
                s.LeftJoin<ConsentArchiveInfo>("CMS_ConsentAgreement.ConsentAgreementConsentHash",
                    "ConsentArchiveHash"))
            .WhereIn("ConsentAgreementContactID", contactIDs)
            .OrderBy("ConsentID")
            .OrderByDescending("ConsentAgreementTime")
            .Result;

        if (DataHelper.DataSourceIsEmpty(consentsDataSet))
        {
            return;
        }
        var consents = new Dictionary<int, ConsentInfo>();
        var consentRevocations = new Dictionary<int, List<ConsentAgreementInfo>>();

        var consentContentArchives = new Dictionary<string, ConsentArchiveInfo>(StringComparer.OrdinalIgnoreCase);
        var consentContentAgreements =
            new Dictionary<string, List<ConsentAgreementInfo>>(StringComparer.OrdinalIgnoreCase);

        foreach (var row in consentsDataSet.Tables[0].AsEnumerable())
        {
            var consentAgreementInfo = new ConsentAgreementInfo(row);

            if (!consents.TryGetValue(consentAgreementInfo.ConsentAgreementConsentID, out var consentInfo))
            {
                consentInfo = new ConsentInfo(row);
                consents.Add(consentAgreementInfo.ConsentAgreementConsentID, consentInfo);
            }

            if (consentAgreementInfo.ConsentAgreementRevoked)
            {
                var revocationsOfSameConsent = GetRevocationsOfSameConsent(consentRevocations,
                    consentAgreementInfo.ConsentAgreementConsentID);
                revocationsOfSameConsent.Add(consentAgreementInfo);
            }
            else
            {
                var agreementsOfSameConsentContent = GetAgreementsOfSameConsentContent(consentContentAgreements,
                    consentAgreementInfo.ConsentAgreementConsentHash);
                agreementsOfSameConsentContent.Add(consentAgreementInfo);

                if (IsAgreementOfDifferentConsentContent(consentAgreementInfo, consentInfo) &&
                    !consentContentArchives.ContainsKey(consentAgreementInfo.ConsentAgreementConsentHash))
                {
                    consentContentArchives.Add(consentAgreementInfo.ConsentAgreementConsentHash,
                        new ConsentArchiveInfo(row));
                }
            }
        }

        WriteConsents(consents, consentContentArchives, consentContentAgreements, consentRevocations);
    }

    private static List<ConsentAgreementInfo> GetRevocationsOfSameConsent(
        Dictionary<int, List<ConsentAgreementInfo>> consentRevocations, int consentId)
    {
        if (!consentRevocations.TryGetValue(consentId, out var revocationsOfSameConsent))
        {
            revocationsOfSameConsent = [];
            consentRevocations.Add(consentId, revocationsOfSameConsent);
        }

        return revocationsOfSameConsent;
    }

    private static List<ConsentAgreementInfo> GetAgreementsOfSameConsentContent(
        Dictionary<string, List<ConsentAgreementInfo>> consentContentAgreements, string consentHash)
    {
        if (!consentContentAgreements.TryGetValue(consentHash, out var agreementsOfSameConsent))
        {
            agreementsOfSameConsent = [];
            consentContentAgreements.Add(consentHash, agreementsOfSameConsent);
        }

        return agreementsOfSameConsent;
    }

    private static bool IsAgreementOfDifferentConsentContent(ConsentAgreementInfo consentAgreementInfo,
        ConsentInfo consentInfo) => consentAgreementInfo.ConsentAgreementConsentHash != consentInfo.ConsentHash;

    private void WriteConsents(Dictionary<int, ConsentInfo> consents,
        Dictionary<string, ConsentArchiveInfo> consentContentArchives,
        Dictionary<string, List<ConsentAgreementInfo>> consentContentAgreements,
        Dictionary<int, List<ConsentAgreementInfo>> consentRevocations)
    {
        foreach (var agreementsOfSameConsentContent in consentContentAgreements.Values)
        {
            var consentAgreement = agreementsOfSameConsentContent.First();
            var consentInfo = consents[consentAgreement.ConsentAgreementConsentID];

            consentContentArchives.TryGetValue(consentAgreement.ConsentAgreementConsentHash, out var consentArchiveInfo);
            consentRevocations.TryGetValue(consentAgreement.ConsentAgreementConsentID, out var revocationsOfSameConsent);

            WriteConsent(consentInfo, consentArchiveInfo, agreementsOfSameConsentContent, revocationsOfSameConsent);
        }
    }

    private void WriteConsent(ConsentInfo consentInfo, ConsentArchiveInfo? consentArchiveInfo,
        IEnumerable<ConsentAgreementInfo> consentAgreements,
        IEnumerable<ConsentAgreementInfo>? consentRevocations)
    {
        personalDataWriter.WriteStartSection(ConsentInfo.OBJECT_TYPE, "Consent");

        var agreedConsentLastModified =
            consentArchiveInfo?.ConsentArchiveLastModified ?? consentInfo.ConsentLastModified;

        WriteConsentContent(consentInfo, consentArchiveInfo);
        WriteConsentAgreements(consentAgreements);
        WriteConsentRevocations(consentRevocations?.Where(cr => cr.ConsentAgreementTime > agreedConsentLastModified));

        personalDataWriter.WriteEndSection();
    }

    private void WriteConsentContent(ConsentInfo consentInfo, ConsentArchiveInfo? consentArchiveInfo)
    {
        if (consentArchiveInfo == null)
        {
            personalDataWriter.WriteBaseInfo(consentInfo, consentInfoColumns, TransformConsentText);
        }
        else
        {
            personalDataWriter.WriteSectionValue("ConsentDisplayName", "Consent name", consentInfo.ConsentDisplayName);
            personalDataWriter.WriteBaseInfo(consentArchiveInfo, consentArchiveInfoColumns, TransformConsentText);
        }
    }

    private void WriteConsentAgreements(IEnumerable<ConsentAgreementInfo> consentAgreements)
    {
        foreach (var consentAgreement in consentAgreements)
        {
            personalDataWriter.WriteBaseInfo(consentAgreement, consentAgreementInfoColumns, TransformConsentAction);
        }
    }

    private void WriteConsentRevocations(IEnumerable<ConsentAgreementInfo>? consentRevocations)
    {
        if (consentRevocations == null)
        {
            return;
        }

        foreach (var consentRevocation in consentRevocations)
        {
            personalDataWriter.WriteBaseInfo(consentRevocation, consentAgreementInfoColumns, TransformConsentAction);
        }
    }

    private void WriteContactActivities(IEnumerable<ActivityInfo> contactActivities)
    {
        foreach (var contactActivityInfo in contactActivities)
        {
            personalDataWriter.WriteStartSection(ActivityInfo.OBJECT_TYPE, "Activity");
            personalDataWriter.WriteBaseInfo(contactActivityInfo, activityInfoColumns);
            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteContactGroups(IEnumerable<ContactGroupInfo> contactContactGroups)
    {
        foreach (var contactGroupInfo in contactContactGroups)
        {
            personalDataWriter.WriteStartSection(ContactGroupInfo.OBJECT_TYPE, "Contact group");
            personalDataWriter.WriteBaseInfo(contactGroupInfo, contactGroupInfoColumns);
            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteSubmittedFormsData(ICollection<string> emails, ICollection<int> contactIDs)
    {
        var consentAgreementGuids = consentAgreementInfoProvider.Get()
            .Columns("ConsentAgreementGuid")
            .WhereIn("ConsentAgreementContactID", contactIDs);

        var formClasses = bizFormInfoProvider.Get()
            .Source(s => s.InnerJoin<DataClassInfo>("CMS_Form.FormClassID", "ClassID"))
            .WhereIn("FormGUID", forms.Keys);

        formClasses.ForEachRow(row =>
        {
            var bizForm = new BizFormInfo(row);
            var formDefinition = forms[bizForm.FormGUID];

            var bizFormItems = formCollectionService.GetBizFormItems(emails, consentAgreementGuids, row, formDefinition);

            foreach (var bizFormItem in bizFormItems)
            {
                personalDataWriter.WriteStartSection("SubmittedForm", "Submitted form");

                personalDataWriter.WriteSectionValue("FormDisplayName", "Form display name", bizForm.FormDisplayName);
                personalDataWriter.WriteSectionValue("FormGUID", "Form GUID", bizForm.FormGUID.ToString());
                personalDataWriter.WriteBaseInfo(bizFormItem, formDefinition.FormColumns);

                personalDataWriter.WriteEndSection();
            }
        });
    }

    private void WriteMembers(IEnumerable<MemberInfo> members)
    {
        foreach (var memberInfo in members.Where(member => member.MemberID > 0))
        {
            personalDataWriter.WriteStartSection(MemberInfo.OBJECT_TYPE, "Member");
            personalDataWriter.WriteBaseInfo(memberInfo, memberInfoColumns);
            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteMemberRoles(IEnumerable<MemberInfo> members)
    {
        foreach (var memberInfo in members.Where(member => member.MemberID > 0))
        {
            // Nested query, do not enumerate to prevent multiple database calls
            var memberRoleBindings = memberRoleMemberInfoProvider.Get()
                .WhereEquals(nameof(MemberRoleMemberInfo.MemberRoleMemberMemberID), memberInfo.MemberID)
                .Column(nameof(MemberRoleMemberInfo.MemberRoleMemberMemberRoleID))
                .Distinct();

            var memberRoles = memberRoleInfoProvider.Get()
                .WhereIn(nameof(MemberRoleInfo.MemberRoleID), memberRoleBindings);

            foreach (var memberRoleInfo in memberRoles)
            {
                personalDataWriter.WriteStartSection("MemberRole", "Member role");
                personalDataWriter.WriteBaseInfo(memberRoleInfo, memberRoleInfoColumns);
                personalDataWriter.WriteEndSection();
            }
        }
    }

    private void WriteCustomers(IEnumerable<CustomerInfo> customers)
    {
        var allCustomerAddresses = customerAddressInfoProvider.Get()
            .WhereIn(nameof(CustomerAddressInfo.CustomerAddressCustomerID), customers.Select(c => c.CustomerID))
            .ToList();

        foreach (var customerInfo in customers.Where(customer => customer.CustomerID > 0))
        {
            personalDataWriter.WriteStartSection(CustomerInfo.OBJECT_TYPE, "Customer");
            personalDataWriter.WriteBaseInfo(customerInfo, customerInfoColumns);

            var customerAddresses = allCustomerAddresses.Where(ca => ca.CustomerAddressCustomerID == customerInfo.CustomerID);
            foreach (var customerAddressInfo in customerAddresses)
            {
                personalDataWriter.WriteStartSection(CustomerAddressInfo.OBJECT_TYPE, "Customer address");
                personalDataWriter.WriteBaseInfo(customerAddressInfo, customerAddressInfoColumns);
                WriteCountryAndOrState(customerAddressInfo.CustomerAddressCountryID, customerAddressInfo.CustomerAddressStateID);

                personalDataWriter.WriteEndSection();
            }

            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteCustomerOrders(IEnumerable<int> customerIds)
    {
        var orders = orderInfoProvider.Get()
            .WhereIn(nameof(OrderInfo.OrderCustomerID), customerIds)
            .ToList();

        var allOrderItems = orderItemInfoProvider.Get()
            .WhereIn(nameof(OrderItemInfo.OrderItemOrderID), orders.Select(o => o.OrderID))
            .ToList();

        var allOrderAddresses = orderAddressInfoProvider.Get()
            .WhereIn(nameof(OrderAddressInfo.OrderAddressOrderID), orders.Select(o => o.OrderID))
            .ToList();

        foreach (var orderInfo in orders)
        {
            personalDataWriter.WriteStartSection(OrderInfo.OBJECT_TYPE, "Order");
            personalDataWriter.WriteBaseInfo(orderInfo, orderInfoColumns);

            var items = allOrderItems.Where(oi => oi.OrderItemOrderID == orderInfo.OrderID);
            foreach (var orderItemInfo in items)
            {
                personalDataWriter.WriteStartSection(OrderItemInfo.OBJECT_TYPE, "Order item");
                personalDataWriter.WriteBaseInfo(orderItemInfo, orderItemInfoColumns);
                personalDataWriter.WriteEndSection();
            }

            var orderAddresses = allOrderAddresses.Where(oa => oa.OrderAddressOrderID == orderInfo.OrderID);
            foreach (var orderAddressInfo in orderAddresses)
            {
                personalDataWriter.WriteStartSection(OrderAddressInfo.OBJECT_TYPE, "Order address");
                personalDataWriter.WriteBaseInfo(orderAddressInfo, orderAddressInfoColumns);
                WriteCountryAndOrState(orderAddressInfo.OrderAddressCountryID, orderAddressInfo.OrderAddressStateID);
                personalDataWriter.WriteEndSection();
            }

            personalDataWriter.WriteEndSection();
        }
    }

    private void WriteThirdPartyCustomerAddresses(IEnumerable<string> customerEmails, IEnumerable<int> customerIds)
    {
        // Customer addresses referencing the data subject's email on other customers' accounts
        var thirdPartyCustomerAddresses = customerAddressInfoProvider.Get()
            .WhereIn(nameof(CustomerAddressInfo.CustomerAddressEmail), customerEmails)
            .WhereNotIn(nameof(CustomerAddressInfo.CustomerAddressCustomerID), customerIds)
            .ToList();

        if (!thirdPartyCustomerAddresses.Any())
        {
            return;
        }

        personalDataWriter.WriteStartSection("ThirdPartyCustomerAddresses", "Customer addresses referencing your data");

        foreach (var customerAddressInfo in thirdPartyCustomerAddresses)
        {
            personalDataWriter.WriteStartSection(CustomerAddressInfo.OBJECT_TYPE, "Customer address");
            personalDataWriter.WriteBaseInfo(customerAddressInfo, customerAddressInfoColumns);
            WriteCountryAndOrState(customerAddressInfo.CustomerAddressCountryID, customerAddressInfo.CustomerAddressStateID);
            personalDataWriter.WriteEndSection();
        }

        personalDataWriter.WriteEndSection();
    }

    private void WriteThirdPartyOrderAddresses(IEnumerable<string> customerEmails, IEnumerable<int> customerIds)
    {
        // Nested query for order IDs belonging to the data subject's own customers
        var ownOrderIds = orderInfoProvider.Get()
            .WhereIn(nameof(OrderInfo.OrderCustomerID), customerIds)
            .Column(nameof(OrderInfo.OrderID));

        // Order addresses referencing the data subject's email on other customers' orders
        var thirdPartyOrderAddresses = orderAddressInfoProvider.Get()
            .WhereIn(nameof(OrderAddressInfo.OrderAddressEmail), customerEmails)
            .WhereNotIn(nameof(OrderAddressInfo.OrderAddressOrderID), ownOrderIds)
            .ToList();

        if (!thirdPartyOrderAddresses.Any())
        {
            return;
        }

        personalDataWriter.WriteStartSection("ThirdPartyOrderAddresses", "Order addresses referencing your data");

        foreach (var orderAddressInfo in thirdPartyOrderAddresses)
        {
            personalDataWriter.WriteStartSection(OrderAddressInfo.OBJECT_TYPE, "Order address");
            personalDataWriter.WriteBaseInfo(orderAddressInfo, orderAddressInfoColumns);
            WriteCountryAndOrState(orderAddressInfo.OrderAddressCountryID, orderAddressInfo.OrderAddressStateID);
            personalDataWriter.WriteEndSection();
        }

        personalDataWriter.WriteEndSection();
    }
}
```

### Implement the collector

The `DataCollectorCore` handles all the heavy lifting, which means that the `IPersonalDataCollector` implementation can be relatively small.

1. Define the `DataCollector` class in the _DataProtection_ folder.
2. Determine the correct `DataWriter` to use based on the `outputFormat` parameter provided to the `Collect` method by Xperience, and use it to instantiate a `DataCollectorCore` object.

```csharp title="DataCollector.cs"
using CMS.DataEngine;
using CMS.DataProtection;
using TrainingGuides.Web.Features.DataProtection.Writers;

namespace TrainingGuides.Web.Features.DataProtection.Collectors;

public class DataCollector(IServiceProvider serviceProvider) : IPersonalDataCollector
{
    public PersonalDataCollectorResult Collect(IEnumerable<BaseInfo> identities, string outputFormat)
    {
        using var personalDataWriter = CreateWriter(outputFormat);

        var dataCollector = ActivatorUtilities.CreateInstance<DataCollectorCore>(serviceProvider, personalDataWriter);

        return new PersonalDataCollectorResult
        {
            Text = dataCollector.CollectData(identities)
        };
    }

    private IPersonalDataWriter CreateWriter(string outputFormat) =>
        outputFormat.ToLowerInvariant() switch
        {
            PersonalDataFormat.MACHINE_READABLE => new XmlPersonalDataWriter(),
            _ => new HumanReadablePersonalDataWriter()
        };
}
```

### Register the Data collector

Register the data collector so that it is called by the system on the **Right to access** and **Data portability** tabs of the **Data protection** application in Xperience.

1. Add the following highlighted code to the `OnInit` method of the `DataProtectionRegistrationModule` class, using the `IServiceProvider` from earlier to activate the collector.

   ```csharp title="DataProtectionRegistrationModule.cs" highlight="30-31"
   using CMS;
   using CMS.Core;
   using CMS.DataEngine;
   using CMS.DataProtection;
   using TrainingGuides.Web.Features.DataProtection.Collectors;
   using TrainingGuides.Web.Features.DataProtection.Shared;

   [assembly: RegisterModule(
       type: typeof(DataProtectionRegistrationModule))]

   namespace TrainingGuides.Web.Features.DataProtection.Shared;

   public class DataProtectionRegistrationModule : Module
   {
       public DataProtectionRegistrationModule()
           : base("DataProtectionRegistration")
       {
       }

       // Contains initialization code that is executed when the application starts
       protected override void OnInit(ModuleInitParameters parameters)
       {
           var serviceProvider = parameters.Services.GetRequiredService<IServiceProvider>();

           base.OnInit(parameters);

           // Adds the IdentityCollector to the collection of registered identity collectors
           IdentityCollectorRegister.Instance.Add(ActivatorUtilities.CreateInstance<IdentityCollector>(serviceProvider));

           // Adds the DataCollector to the collection of registered personal data collectors
           PersonalDataCollectorRegister.Instance.Add(ActivatorUtilities.CreateInstance<DataCollector>(serviceProvider));
       }
   }
   ```

   > **Tip:** This code can be placed anywhere within the scope of the method, though it may be best to put it after the existing code from the previous guide in order to mirror the order in which the collectors are called.

Now, the system will use this data collector on the **Data portability** and **Right to access** tabs of the **Data protection** application in the **Configuration** section of the Xperience administration interface.

If you search for an email address on one of these tabs, Xperience will use this data collector, in conjunction with the identity collector from [earlier in this series](https://docs.kentico.com/guides/development/data-protection/identity-collectors-gather-contacts-associated-with-a-visitor.md), to retrieve all identities and data associated with the email. See an example in the following video:

🎬 [Video](https://docs.kentico.com/docsassets/guides/data-collectors-find-contact-personal-data/Right-to-access.mp4)

## What's next?

The [next guide in this series](https://docs.kentico.com/guides/development/data-protection/data-erasers-clear-contact-information-from-the-site.md) will cover the process of creating a _Data eraser_ for removing visitors' personal data from the site.