Pages

22 July, 2021

Coveo for Sitecore 5 with Sitecore Horizon 10

When we try to use Sitecore Horizon (10.0.1) with with a solution having Coveo integration, Sitecore Horizon throws the below exception in the UI. It does not load any sites or languages in the dropdown. When we try to remove site definition with name "coveo_website", it is loading fine. As per Coveo, we should not remove this site definition as it will impact Coveo integration. 

Reached out to awesome Sitecore community in Sitecore Slack for help and Jeff (@jflh) provided a simple workaround patch configuration in the Sitecore.Demo.Platform repo. Basically we are changing the Coveo_website site definition to an item that does not exist. Horizon will not consider this site as a valid site to be editable in the editor. 

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:coveo="http://www.sitecore.net/xmlconfig/coveo/">
	<sitecore coveo:require="!disabled">
		<sites>
			<!--
				DEMO WORKAROUND
				Coveo for Sitecore and Horizon modules are incompatible by default. Horizon handles the coveo_website as a content site.
				We are applying a workaround similar to what Unicorn had done to fix the same issue: https://github.com/SitecoreUnicorn/Unicorn/issues/398
				We change the coveo_website rootPath to an item that does not exist.
			-->
			<site name="coveo_website">
				<patch:attribute name="rootPath" value="/coveo/for/sitecore/module" />
			</site>
		</sites>
	</sitecore>
</configuration>

After adding this configuration patch, Horizon started to load our sites with Coveo_website site. 

Sitecore Slack Chat: https://sitecorechat.slack.com/archives/C0CF16R9C/p1626812378104900

20 July, 2021

Coveo for Sitecore Upgrade - HtmlContentInBodyWithRequestsProcessor to FetchPageContentProcessor - Length cannot be less than zero

After upgrading from Coveo for Sitecore 4 to version 5, it is noted that there is a new processor which replaces the old HtmlContentInBodyWithRequestsProcessor. New processor in Coveo 5 is called as FetchPageContentProcessor. Similar to the previous processor, it executes an HTTP request, get the page response and then sends the data to the Coveo cloud. Enabling this processor delays the indexing. 

In Coveo 4 for Sitecore

<configuration xmlns:x="http://www.sitecore.net/xmlconfig/" 
  xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:coveo="http://www.sitecore.net/xmlconfig/coveo/">
  <sitecore coveo:require="!disabled">
    <pipelines>
      <coveoPostItemProcessingPipeline>
        <processor type="Coveo.SearchProvider.Processors.HtmlContentInBodyWithRequestsProcessor, Coveo.SearchProviderBase">
          <StartCommentText>BEGIN NOINDEX</StartCommentText>
          <EndCommentText>END NOINDEX</EndCommentText>
        </processor>
      </coveoPostItemProcessingPipeline>
    </pipelines>
  </sitecore>
</configuration>

In Coveo 5 for Sitecore (recommended by Coveo)

<coveoPostItemProcessingPipeline>
  <processor type="Coveo.SearchProvider.Processors.ExecuteGetBinaryDataPipeline, Coveo.SearchProviderBase" />
</coveoPostItemProcessingPipeline>
<coveoGetBinaryData>
  <processor type="Coveo.SearchProvider.Processors.FetchPageContentProcessor, Coveo.SearchProviderBase">
    <inboundFilter hint="list:AddInboundFilter">
      <itemsWithLayout type="Coveo.SearchProvider.Processors.FetchPageContent.Filters.ItemsWithLayout, Coveo.SearchProviderBase" />
    </inboundFilter>
    <preAuthentication hint="list:AddPreAuthenticator" />
    <postProcessing hint="list:AddPostProcessing">
      <processor type="Coveo.SearchProvider.Processors.FetchPageContent.PostProcessing.CleanHtml, Coveo.SearchProviderBase">
        <startComment>BEGIN NOINDEX</startComment>
        <endComment>END NOINDEX</endComment>
      </processor>
    </postProcessing>
  </processor>
</coveoGetBinaryData>

In Coveo 5, there is a post processing processor called CleanHtml which will be executed on the fetched content after the HTTP request. This processor helps you to guide Coveo to index only a certain section of your web page. 

For an example, if you want to remove header, footer, navigation from the index document, you can mark the section using Start Comment and End Comment. In this configuration, it will be <!-- BEGIN NOINDEX --> and <!-- END NOINDEX -->

In a Sitecore instance with Coveo 4, we had nested comments as below. Coveo indexing with the HtmlContentInBodyWithRequestsProcessor processor were able to handle the nested comments and remove the section and send the HTML content to Coveo. 

<!-- BEGIN NOINDEX -->
    <!-- BEGIN NOINDEX -->
    	Content 1
    <!-- END NOINDEX -->
    <!-- BEGIN NOINDEX -->
    	Content 2
    <!-- END NOINDEX -->
<!-- END NOINDEX -->

In Coveo 5, the new processor CleanHtml throws below exception if there is a nested comments. I have even decompiled both the processor and tested the output of the HTML with nested comments and CleanHtml processor throws exception while removing content. 

ManagedPoolThread #19 02:19:52 ERROR An error occurred while trying to clean the HTML, no cleaning will be done.
Exception: System.ArgumentOutOfRangeException
Message: Length cannot be less than zero.
Parameter name: length
Source: mscorlib
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Coveo.SearchProvider.Utils.HtmlCleaner.CleanHtmlContent(String p_HtmlContent, String p_StartCommentText, String p_EndCommentText)
   at Coveo.SearchProvider.Processors.HtmlContentInBodyWithRequestsProcessor.CollectHttpWebResponsesForAllClickableUris(List`1 p_CoveoIndexableItems, Dictionary`2 p_CleanedBinaryDataByUri)

We do not see a way to prevent this error when having a nested comments. When overriding the CleanHtml processor with the old processor method which cleans the HTML, it works but I do not think it is a good way to use the old code and patch it with the new processor. Raised a Coveo ticket to see if there is any workaround. 

Update: As per Coveo, there is no workaround in the Sitecore side, nested tags will need to be removed. 

  1. Jeff (@jflh) suggested that we could use a custom processor which cleans the HTML based on CSS selector instead of CleanHtml processor.
  2. He also provided a suggestion to use a Chrome extension which can help you to decide the Html elements to clean by the processor.Very useful. 
  3. Coveo has an Indexing Pipeline Extension (IPE) which works in the same fashion as mentioned in the first point (custom processor). This requires us to remove all BEGIN and END NOINDEX tags. We need to add a custom class coveo-no-index and IPE will select those sections and remove it. 

Sitecore Slack Chathttps://sitecorechat.slack.com/archives/C0CF16R9C/p1626979876116500

Reference

  1. Index Page Content With the FetchPageContentProcessor
  2. Nested NOINDEX tags preventing HTML content from being fetched during indexing

18 July, 2021

Sitecore Solr - multiple values encountered for non multiValued field

In case if you have faced an error "multiple values encountered for non multiValued field" in Solr, you will need to find the fieldType element in the Sitecore Solr Index Configuration and set it as stringCollection in returnType.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <indexConfigurations>
        <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">
          <fieldMap ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration/fieldMap">
            <fieldNames hint="raw:AddFieldByFieldName">
              <fieldType fieldName="tags" returnType="stringCollection"/>
            </fieldNames>
          </fieldMap>
        </defaultSolrIndexConfiguration>
      </indexConfigurations>
    </contentSearch>
  </sitecore>
</configuration>

Error:

org.apache.solr.common.SolrException: ERROR: [doc=sitecore://master/{37ac1cc6-109b-41ee-9f94-3f527ff22c12}?lang=en&ver=1&ndx=sitecore_master_index] multiple values encountered for non multiValued field tags_t: [594396e563f046b98418b3a8a52d4aa2, 5e918d1e4412450298a8eae0ee65da65, 9fe55245aa6641b79ae55cd47874ee0e]
	at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:153)
	at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:109)
	at org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:975)
	at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:345)
	at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:292)
	at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:239)
	at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
	at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:259)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:489)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:339)
	at org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:339)
	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:225)
	at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
	at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:261)
	at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188)
	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2596)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:427)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:321)
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
	at java.base/java.lang.Thread.run(Thread.java:829)

17 July, 2021

Experience Editor Slow or Hang for non-English languages

As part of the Sitecore Upgrade to Sitecore 10.0.1, we faced an issue related to experience editor. When we try to edit the page in experience editor for non-english version of the item, the CM website hangs and then it will just block every other requests. Entire CM instance will be unavailable. 

Troubleshooting:

  1. Checked the Vanilla Sitecore 10.0.1 version and experience editor is working as expected. 
  2. Debugged the solution and found out that Dictionary API (Sitecore.Globalization.Translate.Text) is the root cause for the slowness. 
Solution:
Sitecore has reported a similar issue as a known issue for Sitecore 9.3 onwards and the fix was included in Sitecore 10.1.0. Sitecore article has a fix for version 9.3 in their page

Reached out to Sitecore to get the fix for Sitecore 10.0.1 version. For this version, the hotfix has a set of DLLs and a config file. They are not support DLLs. So we should include this as part of the CI/CD pipeline so that it does not get overwritten next time. 

After applying the fix, the experience editor started to respond for all the other languages as well. 

Reach out to Sitecore if your Sitecore version (9.3 to 10.0.1) has a similar issue on experience editor and they will be providing the hotfix for the particular version.