Pages

01 November, 2022

Sitecore Scheduled Publishing Module - Update

The purpose of Scheduled Publish Module is to give the content editor the option to delay the publishing of an item for a future point in time. Thus, a page or a feature that should go live at a specific time can be created and populated in Sitecore and previewed long before it goes live without the risk of an accidental publish before the specific time. Moreover, there is no need for a content-editor to go to Sitecore and manually publish something at an inconvenient hour, e.g. a New Year’s announcement. Scheduled Publish intends to give the content-editor all features of a normal publish with the addition of automation, timing and notifications.

Updated to Sitecore 10.2

This module was built by Hedgehog few years ago and it was not updated to Sitecore 10. I forked it and updated to work with Sitecore 10.2. As part of this task,

  • TDS project has been removed and converted to Sitecore Content Serialization with version 5.1.x. 
  • Updated the .NET version, referenced latest Sitecore nuget packages and fixed build issues.
  • Updated the documentation from an old source to GitHub readme file.
  • Created a Sitecore Package for installing it without the source.
You can view it here in my GitHub repository. https://github.com/nehemiahj/SCScheduledPublishing.

Docker Asset Image Creation

Lately, I was exploring options to create a Docker asset image for this module. Robbert Hock has created an excellent Docker asset image creator script which can intake the Sitecore package and convert it into an asset image. When you run the script, you may face this issue "The SQL provider cannot run with dacpac option because of a missing dependency". You can follow the below steps to fix it. Otherwise, it was super easy to create the image using this script. 

To fix the dacpac missing dependency issue:
  • Install the following applications by downloading it from here
    • Microsoft SQL Server 2012 Data-Tier Application Framework
    • SQL Server 2012 Transact-SQL ScriptDom
    • SQL Server System CLR Types 11.0
  • Register it using gacutil
    • "%ProgramFiles(x86)%\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\gacutil" /i "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\SQLDB\DAC\130\Microsoft.SqlServer.TransactSql.ScriptDom.dll


Docker Asset Image:

For the Sitecore Scheduled Publishing module, the Docker image has been published to DockerHub and tagged it for Sitecore 10.2.

Image repository: nehemiah/sitecore-scheduled-publish

Tags: 10.2-1809

Dockerfile instructions:
  • cm:
COPY --from=<sitecore_scheduled_publishing_image> \module\cm\content .\


Reference:

DacFx Issue: https://stackoverflow.com/a/41777959/3016347
sitecore-module-docker-asset-image-creator: https://github.com/KayeeNL/sitecore-module-docker-asset-image-creator

30 September, 2022

Find and Delete xDB Contact Outliers and Automate it via API

This article is about finding and deleting xDB contact outliers automatically using Analytics Database Manager and PowerShell. Everyone talks about Sitecore SmartHub nowadays but huge number of clients are still with Sitecore XP and may face issues with xConnect performance. 

Let's talk about xDB contacts and Interactions. 

xDB Contacts

A contact represents an individual who interacts with or may potentially interact with your organization. Contacts are represented by the Sitecore.XConnect.Contact class, and are uniquely identified by ID (of type Guid) within the xDB. ref

Interactions

An interaction describes any point at which a contact interfaces with a brand, either online or offline. Examples of interactions include:

  • Purchasing a something from a physical store
  • Using an app
  • Browsing a website
  • A phone conversation

Xdb Contact Outliers

Outliers - contacts with abnormal number of interactions. 



Error
2022-05-27 05:05:30.910 -07:00 [Error] ["XdbContextLoggingPlugin"] XdbContext Batch Execution Exception
Sitecore.Xdb.Collection.Failures.DataProviderException: Execution Timeout Expired. 
The timeout period elapsed prior to completion of the operation or the server is not responding. ---> 
System.Data.SqlClient.SqlException: Execution Timeout Expired. 
The timeout period elapsed prior to completion of the operation or the server is not responding. ---> 
System.ComponentModel.Win32Exception: The wait operation timed out

If you have used Sitecore xConnect, you might have faced above error sometimes. Kelly Rusk has an excellent post on how excessive interactions can be prevented.

As per Sitecore support, contacts with >1000 interactions can be considered 'excessive'. so with this large number of interactions I suspect it was causing the SQL performance issues. 

There is a module called Sitecore Analytics Database Manager (ADM) which allows performing various operations with collection database:

  • Removing contact and interactions data.
  • Viewing database statistics.
  • Rebuild calculated facets.
As per the documentation, the current version of ADM has the following functionalities:

  1. Remove all data for contacts that do not have interactions after the specified date or for date range.
  2. Remove only interactions after the specified date or for date range. Rebuild of all calculated facets is triggered automatically for all contacts when the process finishes. Contacts are not removed in this case.
  3. Rebuild the selected calculated facets.
  4. Remove all data for a specific contact defined by the contact id or source and identifier.
  5. Check if the contact with specific id or source and identifier exists.
  6. Retrieve the xDB statistics.
  7. Retrieve xDB Index statistics.
  8. Find outliers in the database.
Once we install the ADM module, there is a set of APIs available to automate the xDB maintenance. We can use a PowerShell script to call this API and schedule the deletion of xDB contact Outliers. 

  1. Authentication:

    In order for calling any API, we need to authenticate the request. To get the access token, we can enable non-interactive client login. We can refer this article to enable it in Identity server. Once enabled, we will get Client_ID and Client_Secret. With that, we can call login API to get the Access Token.



  2. Generate Latest xDB Outliers List

    Using this API, xDB outliers list can be generated. It is a task and with this API, you can let ADM start this process.  



    This process may take minutes to hours based on the size of the xDB. It is better to start this task and we can get the status of the task using another API. Before we run the 3 step, we can execute the below API to get the status and if it is completed, we can start the next step. 


  3. Delete the Outliers

    In this step, there is an API to get the Outliers. Once we get the Outliers, we can call another API to delete the contact with the Outliers contact IDs. Deletion is also a task so once we execute the delete request, we need to check if the task is completed using the 3rd API which will give the status of the deletion request. Once it is completed, we can repeat the whole process. 

    Get the Outliers:



    Delete the Contact by IDs



    Check if the Deletion request is completed

    Repeat this step till we clear the contacts with abnormal number of interactions. This process can give us an overview on how we can find the Outliers using ADM and then delete it. 

26 August, 2022

IIS 301 Redirects with Cache Control

With IIS rewrite module, we can create 301 redirects. Once the 301 redirect response is sent to the browser, client browser caches it indefinitely until browser cache is cleared or hard reload happens. This can tweaked by setting proper cache control header. 

Sometimes 301 redirects need to be changed (yes, it happens!). Even though we change it in rewrite rule in web.config, browser does not send the request to the server to know whether there is an update in the destination url. We can avoid it by setting cache-control response header. 

The problem with the IIS Rewrite Module is that once the "redirect" rule is hit, it does not process further rule. StopProcessing flag is respected only for Rewrites and not for Redirects. If it is respected, we can add outbound rules to modify the response header whenever the response status code is 301. 

A probable solution is to create an intermediate page "redirect.aspx". This page will redirect the request to target URL with proper cache control. Please make sure that instead of Action type as Redirect, we need to change it to Rewrite in IIS Rewrite rule. 


In the aspx page, we can set cache control so that the 301 redirects can be cached in proxy (CDN) for a specific days or hours. In the IIS rewrite rule, we need to change from Redirect to Rewrite. 

Below aspx page will not allow any layer to cache the 301 redirects. Please note that this may increase the traffic to our web server since all the 301 redirects will hit the web server to get the target URL all the time. It is important to set the proper cache control to avoid heavy load on the web server. Based on current infrastructure, we can set cache control header to cache it in CDN or the web server itself.



24 August, 2022

Docker - Increase web.config size

Since we have many IIS rewrites, the web.config file size has exceeded the default 250 KB. We moved all the redirects to separate rewriteMaps configuration but the overall web.config file size has been more than 2.5 MB. In order for the application to work, we went ahead and set the registry changes in Docker images. 

While building the Sitecore CM image, we added few lines in Dockerfile to set the max size. Below lines will increase the web.config max size to 10 MB.

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

# Set the web.config size to 10MB to support rewriteconfig
RUN New-Item -Path 'HKLM:\SOFTWARE\Wow6432Node\Microsoft\InetStp\Configuration'
RUN New-Item -Path 'HKLM:\SOFTWARE\Microsoft\InetStp\Configuration'
RUN Reg Add HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\InetStp\Configuration /v MaxWebConfigFileSizeInKB /t REG_DWORD /d 10240
RUN Reg Add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\InetStp\Configuration /v MaxWebConfigFileSizeInKB /t REG_DWORD /d 10240


Docker - Install IIS Rewrite Module

Recently I wanted to test the IIS redirects in the local Sitecore Docker environment. In order to install IIS rewrite module, we can add few lines in CM Dockerfile. Please note that we need to rebuild the CM image. 

  • First, we need to get the MSI file for Microsoft IIS Rewrite site. You can get it from here
  • Then add the above lines to CM Dockerfile which will download the MSI file and install it silently. 
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

# Install IIS Rewrite Module
WORKDIR /install
ADD https://download.microsoft.com/download/1/2/8/128E2E22-C1B9-44A4-BE2A-5859ED1D4592/rewrite_amd64_en-US.msi rewrite_amd64_en-US.msi
RUN Write-Host 'Installing URL Rewrite' ; Start-Process msiexec.exe -ArgumentList '/i', 'rewrite_amd64_en-US.msi', '/quiet', '/norestart' -NoNewWindow -Wait;

blockquote { margin: 0; } blockquote p { padding: 15px; background: #eee; border-radius: 5px; } blockquote p::before { content: '\201C'; } blockquote p::after { content: '\201D'; }