0 comments on “Sitecore PaaS and Ansible”

Sitecore PaaS and Ansible

Sitecore PaaS and Azure is a good match and the idea is to blend in Ansible for Sitecore PaaS infrastructure set up on Azure and vanilla Site deployment.

Why would you use Ansible? Using Powershell scripts with parameter files is the common approach. Ansible is a very valid alternate approach for organisations who have Ansible in their tech stack already or for those that prefer it over Powershell.

Let’s start with a brief overview of Ansible. Ansible is an automation tool to orchestrate configuration and deployment of software. Ansible is based on agent less architecture by leveraging the SSH daemon. The Ansible playbook is a well defined collection of scripts that defines the work for server configuration and deployment. They are written in YAML and consists of multiple plays each defining the work to be done for a configuration on a managed server. 

Ansible Playbooks help solve the following problems:

  1. Provision of Azure Infrastructure required to run Sitecore and the deployment of Sitecore. Ansible supports the ability to seperate the provision of the infrastructure from the deployment of the Sitecore packages into “roles”. These roles can then be shared between different playbooks essentially allowing for re-use and the configuration of different playbooks for different purposes.
  2. Modularise the environment spin up into tasks/plays instead of one monolithic command doing everything in one go.
  3. By executing a single playbook, all the required tasks are coordinated to be executed to result in a fully operational instance of Sitecore up and running and ready to be customised by the organisations development team

Ansible Playbooks help with workflow between teams:

  1. Provide flexibility for Developers and DevOps teams to work together on separate piece of work to achieve a common goal. A DevOps team can work on the Azure Infrastructure set up and Developers can work on Application set up and vanilla deployment  
  2. Once the environment is provisioned hand it over to Development team for each site to deploy the custom code and configuration on Vanilla site.

Ansible has list of pre-built modules for Azure that can be leveraged for Azure Infrastructure Spin up and deployment. A list is available here https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html#azure.  

We have used the azure_rm_deployment module during the setup journey. The best thing I liked about Ansible was the ability to structure the parameters in a clean and organised fashion to ensure future extensibility is maintained. Ansible supports the use of multiple parameter file. This allows for both shared and environment specific parameter files. You will see the example later in the blog. 

All the ARM templates, play books and tasks are source controlled and Ansible tower can be hooked into the Source control of your choice.

This allows/enforces all changes to the templates, play books and tasks to be made locally and then commited to the source control repository using familiar tools. Asnible will then retrieve the lastest versions of these files from source control as the initial step on execution.

This option is more streamlined than having to manually upload the updated files to an online repository like a storage account and have Ansible/Azure access them using URLs.

Below is one of the example of the playbook. The roles mentioned here are just an example. You will need more roles for a complete azure infrastructure and Sitecore deployment 

Note the variables {{ env }} and {{ scname }}. They are passed from the Ansible tower job template into the playbook. This variables needs to be configured in the Extra VARIABLES field as shown below in the example job template. 

The env name is your target environment for which you want to spin up the Sitecore Azure environment. This could be dev, test, regression or production and the site name is the name of your website. This allows you to use same playbook to spin up multiple sites for multiple environment based on the extra variables passed in the job template in Ansible tower. This combination forms the path to the yml file which contains the definition of the parameters, per site, per environment. Below is the snapshot of the variable file structure. 

  • Each role in the Playbook is a Play/Task and the naming convention is fairly self-explanatory. 
  • Each task has a yml file and ARM template (json file). However it is not mandatory to have an ARM template for each of the tasks.
  1. Create the resource group just to have the tasks yml file and no arm template. 

2. Create the Redis Cache resource that will contain both the tasks yml file and the ARM template. 

There are tons of resources available in the Azure ARM template repo https://github.com/Azure/azure-quickstart-templates to get you started. You can then customize it to suit your projects requirements. Sitecore ARM templates are a good starting point which you can utilize to get some ideas. The idea is that you can grab snippets from these example to form your own ARM template. 

I will be writing more blogs on Azure and Sitecore so stay tuned.

0 comments on “SXA Speedy – Supercharge your SXA Page Speed Scores in Google”

SXA Speedy – Supercharge your SXA Page Speed Scores in Google

We are excited to preview our latest Open Source module. Before jumping into the actual technical details here are some of the early results we are seeing against the Habitat SXA Demo.


Results:

Results

Before:

After

After:

Before
* Results based on Mobile Lighthouse Audit in chrome. 
* Results are based on a local developer machine. Production results usually incur an additional penalty due to network latency.

Want to know more about our latest open source SXA Sitecore module …. read on ….


I’m continually surprised by the number of new site launches that fail to implement Google recommendations for Page Speed. If you believe what Niel Patel has to say this score is vitally important to SEO and your search ranking. At Aceik it’s one of the key benchmarks we use to measure the projects we launch and the projects we inherit and have to fix.

The main issue is often a fairly low mobile score, desktop tends to be easier to cater for. In particular, pick out any SXA project that you know has launched recently and even with bundling properly turned on its unlikely to get over 70 / 100 (mobile score). The majority we tried came in somewhere around the 50 to 60 out 100 mark.

Getting that page score into the desired zone (which I would suggest is 90+) is not easy but here is a reasonable checklist to get close.

1) Introduce image lazy loading
2) Ensure a cache strategy is in place and verify its working.
3) Dianoga is your friend for image compression
4) Use responsive images (must serve up smaller images sizes for mobile)
5) Introduce Critical CSS and deferred CSS files
7) Javascript is not a page speed friend. Defer Defer Defer

The last two items are the main topics that I believe are the hardest to get right. These are the focus of our new module.

Critical_plus_defer

Check out the GitHub repository.

I have also done an installation and usage video.

So how will the module help you get critical and JS defer right?

Deferred Javascript Load

For Javascript, it uses a deferred loading technique mentioned here. I attempted a few different techniques before finding this blog and the script he provides (closer to the bottom of the article) seems to get the best results.  It essentially incorporates some clever tactics (as mentioned in the article) that defer script load without compromising load order.

I also added in one more technique that I have found useful and that is to use a cookie to detect a first or second-time visitor. Second-time visitors naturally will have all external resources cached locally, so we can, therefore, provide a completely different loading experience on the 2nd pass. It stands to reason that only on the very first-page load we need to provide a deferred experience.

Critical + Deferred CSS Load

For CSS we incorporated the Critical Viewport technique that has been recommended by Google for some time. This technique was mentioned in this previous blog post. Generating the Critical CSS is not something we want to be doing manually and there is an excellent gulp based package that does this for you.

It can require some intervention and tweaking of the Critical CSS once generated, but the Gulp scripts provided in the module do seek to address/automate this.

Our module has a button added into the Configure panel inside the Sitecore CMS. So Content Editors can trigger off the re-generation of the Critical CSS when ever needed.

Generate Critical button added to Configure.

Local vs Production Scores

It’s also important to remember that the scores you achieve via Lighthouse built into Chrome on localhost and your non-public development servers can be vastly different than production. In fact, it’s probably safest to assume that non-production boxes give false positives in the region of 10 to 20 points. So it’s best to assume that your score on production will be a little worse than expected.

Conclusion

It’s a fair statement that you can’t just install the module and expect Page Load to be perfect in under 10 minutes.  Achieving top Page Load Speed’s requires many technical things to work together. By ensuring that the previously mentioned checklists are done (Adequate Servers, Sitecore Cache, Image Loading techniques) you are partway over the line. By introducing the deferred load techniques in the module (as recommended by Google) you should then be a step closer to top score.

For more hints please see the Wiki on Github.

This module has been submitted to the Sitecore Marketplace and is awaiting approval.


Author: Thomas Tyack – Solutions Architect / Sitecore MVP 2019

0 comments on “Monitoring and Debugging Interaction Processing in Sitecore 9 on Azure PaaS”

Monitoring and Debugging Interaction Processing in Sitecore 9 on Azure PaaS

When configuring a new instance of Sitecore XP or maintaining an existing one, you may encounter a situation where your interactions report shows far fewer interactions than expected.

low-interactions
Where are my interactions?

One possible cause is interaction processing which hasn’t kept up with the interactions being logged on your website. In some cases this can be so slow that it appears collection, processing, and reporting aren’t working at all. Here are a few things you can look at to help you diagnose your issue.

 

Are interactions being recorded?

SELECT TOP 10 * FROM xdb_collection.Interactions ORDER BY StartDateTime ASC
Run this command in each of your shard databases to see the recent interactions which have been recorded. Compare the interactions being logged with the expected number and frequency of interactions in the environment you’re looking at.

 

How many interactions are waiting to be processed?

SELECT COUNT(*) FROM xdb_processing_pools.InteractionLiveProcessingPool
This command will indicate the number of interactions waiting to be processed. Monitoring the number of records in this table can give you an indication of the number of new records being created and the number of new interactions which are being queued for processing.
If the number of records is steadily building up, either processing isn’t working or it’s working too slowly to handle the workload.
If you’re collecting interactions but not seeing the size of the live interaction processing pool change at all, there might be an issue with aggregation.

If Analytics reports don’t look quite right, there are some things you can try:

Disable device detection

We encountered an issue with slow processing on a recent project. After logging an issue with Sitecore support, they advised:
Device detection has been known to cause the slowness in rebuilding reporting DB.
Try disabling device detection to determine if this has been impacting the speed of processing.

 

Check the CPU usage on your processing role

If you’re consistently seeing a high level of activity, you may need to scale your processing instances up or out.

high-average-cpu
Time for more instances…

Check connection strings

Use the Server Role Configuration Reference to ensure you have the correct settings on each of your servers

Check Application Insights errors

Check in Application Insights for any repeated error messages that might indicate misconfiguration.

 

millions-of-interactions
That’s more like it!

Helpful links

0 comments on “Sitecore Azure Search: Top 10 Tips”

Sitecore Azure Search: Top 10 Tips

Its been a while since I first wrote about Azure Search and we have a few more tips and tricks on how to optimise Azure Search implementations.

Before proceeding if you missed our previous posts check out some tools we created for Azure Search Helix setup and Geo-Spatial Searching.

Also, check out the slides from our presentation at last years Melbourne Sitecore User Group.

Ok let us jump into the top 10 tips:

Tip 1) Create custom indexes for targeted searching

The default out of the box indexes will attempt to cover just about everything in your Sitecore databases. They do so to support Sitecore CMS UI searches out of the box.  It’s not a problem if you want to use the default indexes (web, master) to search with, however for optimal searches and faster re-indexing time a custom index will help performance.

By stepping back and looking at the different search requirements across the site you can map out your custom indexes and the data that each will require.

Consider also that if the custom indexes need to be used across multiple Feature Helix modules the configuration files and search repositories may need to live in an appropriate Foundation module. More about feature vs foundation can be found here.

Tip 2) Keep your indexes lean

This tip follows on from the first Tip.

Essentially the default Azure Search configuration out of the box will have:

<indexAllFields>true</indexAllFields>

This can include a lot of fields and your probably not going to need every single Sitecore field in order to present the user with meaningful data on the front end interfaces.

The other option is to specify only the fields that you need in your indexes:

<include hint="list:IncludeField"> 
<Text>{A60ACD61-A6DB-4182-8329-C957982CEC74}</Text> 
</include>

The end result will limit the amount of JSON payload that needs to be sent across the network and also the amount of payload that the Sitecore Azure Search Provider needs to process.

Particularly if you are returning thousands of search results you can see what happens when “IndexAllFields” is on via Fiddler.

This screenshot is via a local development machine and Azure Search instance at the Microsoft hosting centre.

Fiddler Index

JSONFIelds

  • So for a single query “IndexAllFields” can result in:
    • 2 MB plus JSON payload size.
    • Document results with all Sitecore metadata included. That could be around 100 fields.

If your query results in Document counts in the thousands obviously the payload will grow rapidly. By reducing the fields in your indexes (removing un-necessary data)  you can speed up query, transfer and processing times and get the data displayed quicker.

Tip 3) Make use of direct azure connections

Sitecore has done a lot of the heavy lifting for you in the Sitecore Azure Search Provider. It’s a bit like a wrapper that does all the hard work for you. In some cases however you may find that writing your own queries that connect via the Azure Search DLL gives you better performance.

Tip 4) Monitor performance via Azure Search Portal

It’s really important to monitor your Azure Search Instance via Azure Portal. This will give you critical clues as to whether your scaling settings are appropriate.

In particular look out for high latency times as this will indicate that your search queries are getting throttled. As a result, you may need to scale up your Azure Search Instance.

In order to monitor your latency times go to:

  1. Login to Azure Portal
  2. Navigate to your Azure Search Instance.
  3. Click on metrics in the left-hand navigation
    • metrics
  4. Select the “Search Latency” checkbox and scan over the last week.
    • graph
  5. You will see some peaks these usually indicate heavy periods of re-indexing. During re-indexing, the Azure Search instance is under heavy load. As long as your peaks under 0.5-second mark your ok.  If you see Search Latency up into the 2-second timeframe you probably need to either adjust how your indexes are used (caching and re-indexing) or scale up to avoid the flow on effects of slow search.

Tip 5) Cache Wrappers

In the code that uses Azure Search, it would be advisable to use cache wrappers around the searches when possible. For your most common searches, this should prevent Azure Search getting hit repeatedly with the same query.

For a full example of cache wrapper checkout the section titled Sitecore.Caching.CustomCache in my previous blog post.

Tip 6) Disable Indexing on CD

This is a hot tip that we got from Sitecore Support when we started to encounter high search latency during re-indexing.

Most likely in your production setup, you will have a single Azure Search instance shared between CM and CD environments.

You need to factor in that CM should be the server that controls the re-indexing (writing) and CD will most likely be the server doing the queries (reading).

Re-indexing is triggered via the event queue and every server subscribes and reacts to these events. Each server with the out of the box search configuration will cause the Azure Search indexes to be updated.  In a shared Azure Search (or SOLR instance) this only needs to be updated by a single server. Each additional re-index is overkill and just doubling up on re-indexing workload.

You can, therefore, adjust the configuration on the CD servers so that it does not cause re-indexing to happen.

The trick is in your index configuration files to use Configuration Roles to specify the indexing strategy on each server.

 <strategies hint="list:AddStrategy">
 <!--
 NOTE: order of these is controls the execution order 
 -->
 <strategy role:require="Standalone OR ContentManagement" ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync"/>
 <strategy role:require="ContentDelivery" ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual"/>
 </strategies>

Setting the index update strategy to manual on your CD servers will take a big load off your remote indexes.

Particularly if you have multiple CD servers using the same indexes. Each additional CD server would cause additional updates to the index without the above setting.

Tip 7) Rigid Indexes – Have a deployment plan

If your deployment includes additions and changes to the indexes and you need 100% availability of search data, a deployment plan for re-indexing will be required.

Grant chatted about the problem in his post here. To get around this you could consider using the blue / green paradigm during deployments.

  • This would mean having a set blue indexes and a set of green indexes.
  • Using slot swaps for your deployments.
    • One slot points to green in configuration.
    • One slot (production) points to blue in configuration.
  • To save on costs you could decommission the staging slot between deployments.

Tip 8) HttpClient should be a singleton or static

The basic idea here is that you should keep the number of HttpClient instances in your code to an absolute minimum if you want optimal performance.

The Sitecore Azure Search provider actually spins up 2 x HttpClient connections for every single index. This in itself is not ideal and unfortunately, there is not a lot you can do about this code in the core product itself.

In your own connections to other APIs, however, HttpClient SendAsync is perfectly thread safe.

By using HttpClient singletons you stand to gain big in the performance stakes. One great blog article worth reading runs you through the performance benefits. 

It’s also worth noting that in the Azure Search documentation Microsoft themselves say you should treat HttpClient as a singleton.

Tip 9) Monitor your resources

In Azure web apps you have finite resources with your app server plans. Opening multiple connections with HttpClient and not disposing of them properly can have severe consequences.

For instance, we found a bug in the core Sitecore product that was caused by the connection retryer. It held open ports forever whenever we hit out Azure Search plan usage limits.  The result was that we hit outbound open connection limits for sockets and this caused our Sitecore instance to ground to a slow halt.

Sitecore has since resolved the issue mentioned above after a lengthy investigation working alongside the Aceik team. This was tracked under reference number 203909.

To monitor the number of sockets in Azure we found a nice page on the MSDN site.

Tip 10) Make use of OData Expressions

This tip relates strongly to tip 3.  Azure search has some really powerful OData Expressions that you can make use of by a direct connection.  Once you have had a play with direct connections it is surprisingly easy to spin up really fast queries.

Operators include:

  • OrderBy, Filter (by field), Search
  • Logical operators (and, or, not).
  • Comparison expressions (eq, ne, gt, lt, ge, le).
  • any with no parameters. This tests whether a field of type Collection(Edm.String) contains any elements.
  • any and all with limited lambda expression support.
  • Geospatial functions geo.distance and geo.intersects. The geo.distance function returns the distance in kilometres between two points.

See the complete list here.


 

Q&A

Q) Anything on multiple region setups? Or latency considerations?

A) Multi-region setups:   Although I can’t comment from experience the configuration documentation does state that you can specify multiple Azure Search instances using a pipe separator in the connection string.

<add name="cloud.search" connectionString="serviceUrl=https://searchservice1.search.windows.net;apiVersion=2015-02-28;apiKey=AdminKey1|serviceUrl=https://searchservice2.search.windows.net;apiVersion=2015-02-28;apiKey=AdminKey2" /> 

Unfortunately, the documentation does not go into much detail. It simply states that “Sitecore supports a Search service with geo-replicated scenarios” which one would hope means under the hood it has all the smarts to take care of this.

I’m curious about this as well and opened a stack overflow ticket. Let’s see if anyone else in the community can answer this for us.

Search Latency: 

Search latency can be directly improved by adding more replicas via the scaling setting in Azure Portal

replicas

Two replicas should be your starting point for an Azure Search instance to support Sitecore. Once you launch your site you will need to follow the instruction in tip 4 above monitor search latency.  If the latency graph is showing consistent spikes and high latency times above 0.5 seconds it’s probably time to add some more replicas.