Pages

18 August, 2021

Sitecore Upgrade - Codebase Upgrade Series - Framework upgrade - .NET

As part of the Sitecore codebase upgrade series, in this blog, we are going to deal with upgrading .NET Framework version. 

Option 1: Using PowerShell, we can quickly update the version in all the csproj files. The script has been added in this blog. This script will update both legacy and SDK project format.

Option 2: To upgrade .NET framework, you can open the csproj files in Notepad++ and replace the version number from 4.x to 4.8 (or desired version as per Sitecore). 

  • SDK Style Projects
    Should replace <TargetFramework>net4xx</TargetFramework> with the targeted version. 
  • Legacy Style Projects
    Should replace <TargetFrameworkVersion>v4.x.x</TargetFrameworkVersion> with the targeted version. 

Option 3: There is a Visual Studio marketplace module called TargetFrameworkMigrator. Once you install it, you can change the target framework to the desired version. This will work only for the Legacy Style Projects. SDK Style Project support will be added in the next version as per the roadmap. 

  • Once you install the VS Extension. Open the extension from Tools menu --> Target Framework Migrator.
  • Choose the projects you want to migrate, then click Migrate. 


17 August, 2021

Update .NET Version in SDK and Legacy Project Format Type

As part of Sitecore Upgrade, we had to upgrade .NET version in 100+ Helix projects. The below script was used to update the version in the projects quickly. 

Since Microsoft introduced SDK project format, the XML tag in the project file and also the version format are different. The below updates .NET Target Framework version in both SDK and Legacy Project format types. 


22 July, 2021

Coveo for Sitecore 5 with Sitecore Horizon 10

When we try to use Sitecore Horizon (10.0.1) with with a solution having Coveo integration, Sitecore Horizon throws the below exception in the UI. It does not load any sites or languages in the dropdown. When we try to remove site definition with name "coveo_website", it is loading fine. As per Coveo, we should not remove this site definition as it will impact Coveo integration. 

Reached out to awesome Sitecore community in Sitecore Slack for help and Jeff (@jflh) provided a simple workaround patch configuration in the Sitecore.Demo.Platform repo. Basically we are changing the Coveo_website site definition to an item that does not exist. Horizon will not consider this site as a valid site to be editable in the editor. 

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:coveo="http://www.sitecore.net/xmlconfig/coveo/">
	<sitecore coveo:require="!disabled">
		<sites>
			<!--
				DEMO WORKAROUND
				Coveo for Sitecore and Horizon modules are incompatible by default. Horizon handles the coveo_website as a content site.
				We are applying a workaround similar to what Unicorn had done to fix the same issue: https://github.com/SitecoreUnicorn/Unicorn/issues/398
				We change the coveo_website rootPath to an item that does not exist.
			-->
			<site name="coveo_website">
				<patch:attribute name="rootPath" value="/coveo/for/sitecore/module" />
			</site>
		</sites>
	</sitecore>
</configuration>

After adding this configuration patch, Horizon started to load our sites with Coveo_website site. 

Sitecore Slack Chat: https://sitecorechat.slack.com/archives/C0CF16R9C/p1626812378104900

20 July, 2021

Coveo for Sitecore Upgrade - HtmlContentInBodyWithRequestsProcessor to FetchPageContentProcessor - Length cannot be less than zero

After upgrading from Coveo for Sitecore 4 to version 5, it is noted that there is a new processor which replaces the old HtmlContentInBodyWithRequestsProcessor. New processor in Coveo 5 is called as FetchPageContentProcessor. Similar to the previous processor, it executes an HTTP request, get the page response and then sends the data to the Coveo cloud. Enabling this processor delays the indexing. 

In Coveo 4 for Sitecore

<configuration xmlns:x="http://www.sitecore.net/xmlconfig/" 
  xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:coveo="http://www.sitecore.net/xmlconfig/coveo/">
  <sitecore coveo:require="!disabled">
    <pipelines>
      <coveoPostItemProcessingPipeline>
        <processor type="Coveo.SearchProvider.Processors.HtmlContentInBodyWithRequestsProcessor, Coveo.SearchProviderBase">
          <StartCommentText>BEGIN NOINDEX</StartCommentText>
          <EndCommentText>END NOINDEX</EndCommentText>
        </processor>
      </coveoPostItemProcessingPipeline>
    </pipelines>
  </sitecore>
</configuration>

In Coveo 5 for Sitecore (recommended by Coveo)

<coveoPostItemProcessingPipeline>
  <processor type="Coveo.SearchProvider.Processors.ExecuteGetBinaryDataPipeline, Coveo.SearchProviderBase" />
</coveoPostItemProcessingPipeline>
<coveoGetBinaryData>
  <processor type="Coveo.SearchProvider.Processors.FetchPageContentProcessor, Coveo.SearchProviderBase">
    <inboundFilter hint="list:AddInboundFilter">
      <itemsWithLayout type="Coveo.SearchProvider.Processors.FetchPageContent.Filters.ItemsWithLayout, Coveo.SearchProviderBase" />
    </inboundFilter>
    <preAuthentication hint="list:AddPreAuthenticator" />
    <postProcessing hint="list:AddPostProcessing">
      <processor type="Coveo.SearchProvider.Processors.FetchPageContent.PostProcessing.CleanHtml, Coveo.SearchProviderBase">
        <startComment>BEGIN NOINDEX</startComment>
        <endComment>END NOINDEX</endComment>
      </processor>
    </postProcessing>
  </processor>
</coveoGetBinaryData>

In Coveo 5, there is a post processing processor called CleanHtml which will be executed on the fetched content after the HTTP request. This processor helps you to guide Coveo to index only a certain section of your web page. 

For an example, if you want to remove header, footer, navigation from the index document, you can mark the section using Start Comment and End Comment. In this configuration, it will be <!-- BEGIN NOINDEX --> and <!-- END NOINDEX -->

In a Sitecore instance with Coveo 4, we had nested comments as below. Coveo indexing with the HtmlContentInBodyWithRequestsProcessor processor were able to handle the nested comments and remove the section and send the HTML content to Coveo. 

<!-- BEGIN NOINDEX -->
    <!-- BEGIN NOINDEX -->
    	Content 1
    <!-- END NOINDEX -->
    <!-- BEGIN NOINDEX -->
    	Content 2
    <!-- END NOINDEX -->
<!-- END NOINDEX -->

In Coveo 5, the new processor CleanHtml throws below exception if there is a nested comments. I have even decompiled both the processor and tested the output of the HTML with nested comments and CleanHtml processor throws exception while removing content. 

ManagedPoolThread #19 02:19:52 ERROR An error occurred while trying to clean the HTML, no cleaning will be done.
Exception: System.ArgumentOutOfRangeException
Message: Length cannot be less than zero.
Parameter name: length
Source: mscorlib
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Coveo.SearchProvider.Utils.HtmlCleaner.CleanHtmlContent(String p_HtmlContent, String p_StartCommentText, String p_EndCommentText)
   at Coveo.SearchProvider.Processors.HtmlContentInBodyWithRequestsProcessor.CollectHttpWebResponsesForAllClickableUris(List`1 p_CoveoIndexableItems, Dictionary`2 p_CleanedBinaryDataByUri)

We do not see a way to prevent this error when having a nested comments. When overriding the CleanHtml processor with the old processor method which cleans the HTML, it works but I do not think it is a good way to use the old code and patch it with the new processor. Raised a Coveo ticket to see if there is any workaround. 

Update: As per Coveo, there is no workaround in the Sitecore side, nested tags will need to be removed. 

  1. Jeff (@jflh) suggested that we could use a custom processor which cleans the HTML based on CSS selector instead of CleanHtml processor.
  2. He also provided a suggestion to use a Chrome extension which can help you to decide the Html elements to clean by the processor.Very useful. 
  3. Coveo has an Indexing Pipeline Extension (IPE) which works in the same fashion as mentioned in the first point (custom processor). This requires us to remove all BEGIN and END NOINDEX tags. We need to add a custom class coveo-no-index and IPE will select those sections and remove it. 

Sitecore Slack Chathttps://sitecorechat.slack.com/archives/C0CF16R9C/p1626979876116500

Reference

  1. Index Page Content With the FetchPageContentProcessor
  2. Nested NOINDEX tags preventing HTML content from being fetched during indexing

18 July, 2021

Sitecore Solr - multiple values encountered for non multiValued field

In case if you have faced an error "multiple values encountered for non multiValued field" in Solr, you will need to find the fieldType element in the Sitecore Solr Index Configuration and set it as stringCollection in returnType.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <indexConfigurations>
        <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">
          <fieldMap ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration/fieldMap">
            <fieldNames hint="raw:AddFieldByFieldName">
              <fieldType fieldName="tags" returnType="stringCollection"/>
            </fieldNames>
          </fieldMap>
        </defaultSolrIndexConfiguration>
      </indexConfigurations>
    </contentSearch>
  </sitecore>
</configuration>

Error:

org.apache.solr.common.SolrException: ERROR: [doc=sitecore://master/{37ac1cc6-109b-41ee-9f94-3f527ff22c12}?lang=en&ver=1&ndx=sitecore_master_index] multiple values encountered for non multiValued field tags_t: [594396e563f046b98418b3a8a52d4aa2, 5e918d1e4412450298a8eae0ee65da65, 9fe55245aa6641b79ae55cd47874ee0e]
	at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:153)
	at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:109)
	at org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:975)
	at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:345)
	at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:292)
	at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:239)
	at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
	at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:259)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:489)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:339)
	at org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:339)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:225)
	at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
	at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:261)
	at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2596)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:427)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:321)
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
	at java.base/java.lang.Thread.run(Thread.java:829)