0 comments on “Sitecore Page Speed: Part 3: Eliminate JS Loading Time”

Sitecore Page Speed: Part 3: Eliminate JS Loading Time

In part 1 & part 2 of our Sitecore page speed blog, we covered off:

  • The Google Page Speed Insights tool.
  • We looked at a node tool called critical that could generate above the fold (critical viewport) CSS code that is minified.
  • We referenced the way in which Google recommends deferring CSS loading.
  • We showed a way to integrate “Above The Fold” CSS into a Helix based project and achieve a page free of render blocking CSS.

In this 3rd part of the series, we will introduce a way to defer the load of all external javascript assets (async).

A reminder that I have committed the sample code for this blog into a fork of the helix habitat example project. You can find the sample here. For a direct comparison of changes made to achieve these page load enhancements, view a side by side comparison here.

Dynamic JS loading Installation Steps:

  1. Inside Sitecore add a new view Rendering that reference the file /Views/Common/Assets/Scripts-3.2.1.cshtml
    • Note down the ID of this rendering and replace in the ID of the rendering in the next step.
  2. Update the Default.cshtml layout to include a new cached rendering.
  3.  @*Scripts Legacy Jquery jquery-3.2.1 *@
     @Html.Sitecore().CachedRendering("{B0DD36CE-EE4A-4D01-9986-7BEF114196DD}", new RenderingCachingSettings { Cacheable = true, CacheKey = cacheKey + "_bottom_scripts" })
    • cacheKey = This variable is something unique that will identify the page. You could use the Sitecore context Item ID or path for example.

Explanation:

The rendering Scripts-3.2.1.cshtml will render out the following javascript onto the page:

var scriptsToLoad = ['//cdnjs.cloudflare.com/ajax/libs/modernizr/2.8.3/modernizr.min.js','//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js','/assets/js/slick.min.js','/assets/js/global.js','/assets/js/Script.js','/assets-legacy/js/lib/lazyload.min.js'];
src="/assets-legacy/js/lib/jquery-3.2.1.min.js" async defer>
  • First of all, it prints out a JS array of all the scripts that this page requires.
    • This is the array of JS files that comes from Themes and Page Assets inside the CMS. If you are familiar with Habitat Helix this list can be content managed inside the CMS.
  • It then instructs the jquery library to be loaded async (which will not block the network download of the page response).
  • Once jquery is loaded, this modified version of jquery contains some code at the end that will read in the list of scripts dynamically and apply them to the page.
    • This is achieved with fairly simple AJAX load calls to the script URLs.

Outcome:

Once integrated successfully you will end up with a page that does not contain any blocking JS network calls.  The Google Page Speed tool should give you a nice score boost for your achievement in reducing initial load time.


Hints and Tips:

Bootstrapping JQuery Code:

  • jquery Document.Ready() function calls may not fire inside dynamically loaded JS files. This is because the JS file is loaded after DOM is ready and it’s too late for the Document.Ready() event at this stage.
  • As a workaround, you could code your JS files to bootstrap on both the Document.Ready() or whenever $ is not undefined.
  • In the case of dynamic loading in this manner, because jquery was loaded first, $ should not be undefined and your code should be bootstrapped successfully.

Debugging in chrome:

  • When dynamically loading JS files they may strangely not appear in the chrome console debugger as you would normally expect.
  • The workaround for this is to add a comment to the top of each JS library
  • //# sourceURL=global.js
  • This will cause the chrome debugger to list the file in the source tab under the “(no domain)” heading.
  • You will then be able to debug the file as per normal.
0 comments on “Sitecore Page Speed: Part 2: Inlining CSS into Helix”

Sitecore Page Speed: Part 2: Inlining CSS into Helix

In part 1 of this Sitecore page speed blog, we covered off:

  • The Google Page Speed Insights tool.
  • We looked at a node tool called critical that could generate above the fold (critical viewport) CSS code that is minified.
  • We referenced the way in which Google recommends deferring CSS loading.

In this second part of the Sitecore Page Speed series, I am going to cover off how I would go about achieving this in my Sitecore layout.

Before we dive in I have committed the sample code for this blog into a fork of the helix habitat example project. You can find the sample here. For a direct comparison of changes made to achieve these page load enhancements, view a side by side comparison here.

Installation Steps:

MinimisedCode

  1. For each page that you want to render above the fold CSS we take the minimised code (we generated in the first blog) and put it on the page within the CMS.
  2. Inside the CMS we also create some new renderings in the common project.
    • These renderings are used in the default.cshtml layout.
    • They point to the CSS rendering code.
    • This wrapping technique provides the ability to cache the rendering so that the code in RenderAssetsService.cs does not need to be executed on every single page load.
    • Take note of the IDs of each rendering you will need to copy them over to the CachedRendering IDs shown in step 3 below.
  3. Update the default.cshtml layout with two key renderings.
    • One in the <head> tag that points to /Views/Common/Assets/InlineStyles.cshtml
    •  @*Inline Styles Rendering*@
       @Html.Sitecore().CachedRendering("{B14DA82E-F844-4945-8F31-4577A52861E1}", new RenderingCachingSettings { Cacheable = true, CacheKey = cacheKey + "_critical_styles" })
    • One just before the </body> closing tag that points to /Views/Common/Assets/StylesDeferred.cshtml
    •  @*Styles Rendering Deferred Styles *@
       @Html.Sitecore().CachedRendering("{F04C562A-CBF9-40CF-8CA9-8CE83FDF0BFA}", new RenderingCachingSettings { Cacheable = true, CacheKey = cacheKey + "_bottom_styles" })

StylesDeferred.cshtml  contains logic that will check for inline CSS on every page. If the page contains Inline CSS then the main CSS files will have their network download deferred until later. On the other hand, if the page does not contain any inline CSS then the main CSS files will be loaded as blocking assets. Doing so ensures that the page displays normally in both situations.

  • The cacheKey variable passed to our CachedRendering is simply something to identify the page as unique.  You could use the Sitecore context item ID or path for example.

If done correctly you should end up with pages that look normal even with the main CSS files deleted (only do this as a test). The CSS will no longer load via another network that blocks the page and your Google Page Speed rank should recieve a boost.

0 comments on “Sitecore Azure Search: Top 10 Tips”

Sitecore Azure Search: Top 10 Tips

Its been a while since I first wrote about Azure Search and we have a few more tips and tricks on how to optimise Azure Search implementations.

Before proceeding if you missed our previous posts check out some tools we created for Azure Search Helix setup and Geo-Spatial Searching.

Also, check out the slides from our presentation at last years Melbourne Sitecore User Group.

Ok let us jump into the top 10 tips:

Tip 1) Create custom indexes for targeted searching

The default out of the box indexes will attempt to cover just about everything in your Sitecore databases. They do so to support Sitecore CMS UI searches out of the box.  It’s not a problem if you want to use the default indexes (web, master) to search with, however for optimal searches and faster re-indexing time a custom index will help performance.

By stepping back and looking at the different search requirements across the site you can map out your custom indexes and the data that each will require.

Consider also that if the custom indexes need to be used across multiple Feature Helix modules the configuration files and search repositories may need to live in an appropriate Foundation module. More about feature vs foundation can be found here.

Tip 2) Keep your indexes lean

This tip follows on from the first Tip.

Essentially the default Azure Search configuration out of the box will have:

<indexAllFields>true</indexAllFields>

This can include a lot of fields and your probably not going to need every single Sitecore field in order to present the user with meaningful data on the front end interfaces.

The other option is to specify only the fields that you need in your indexes:

<include hint="list:IncludeField"> 
<Text>{A60ACD61-A6DB-4182-8329-C957982CEC74}</Text> 
</include>

The end result will limit the amount of JSON payload that needs to be sent across the network and also the amount of payload that the Sitecore Azure Search Provider needs to process.

Particularly if you are returning thousands of search results you can see what happens when “IndexAllFields” is on via Fiddler.

This screenshot is via a local development machine and Azure Search instance at the Microsoft hosting centre.

Fiddler Index

JSONFIelds

  • So for a single query “IndexAllFields” can result in:
    • 2 MB plus JSON payload size.
    • Document results with all Sitecore metadata included. That could be around 100 fields.

If your query results in Document counts in the thousands obviously the payload will grow rapidly. By reducing the fields in your indexes (removing un-necessary data)  you can speed up query, transfer and processing times and get the data displayed quicker.

Tip 3) Make use of direct azure connections

Sitecore has done a lot of the heavy lifting for you in the Sitecore Azure Search Provider. It’s a bit like a wrapper that does all the hard work for you. In some cases however you may find that writing your own queries that connect via the Azure Search DLL gives you better performance.

Tip 4) Monitor performance via Azure Search Portal

It’s really important to monitor your Azure Search Instance via Azure Portal. This will give you critical clues as to whether your scaling settings are appropriate.

In particular look out for high latency times as this will indicate that your search queries are getting throttled. As a result, you may need to scale up your Azure Search Instance.

In order to monitor your latency times go to:

  1. Login to Azure Portal
  2. Navigate to your Azure Search Instance.
  3. Click on metrics in the left-hand navigation
    • metrics
  4. Select the “Search Latency” checkbox and scan over the last week.
    • graph
  5. You will see some peaks these usually indicate heavy periods of re-indexing. During re-indexing, the Azure Search instance is under heavy load. As long as your peaks under 0.5-second mark your ok.  If you see Search Latency up into the 2-second timeframe you probably need to either adjust how your indexes are used (caching and re-indexing) or scale up to avoid the flow on effects of slow search.

Tip 5) Cache Wrappers

In the code that uses Azure Search, it would be advisable to use cache wrappers around the searches when possible. For your most common searches, this should prevent Azure Search getting hit repeatedly with the same query.

For a full example of cache wrapper checkout the section titled Sitecore.Caching.CustomCache in my previous blog post.

Tip 6) Disable Indexing on CD

This is a hot tip that we got from Sitecore Support when we started to encounter high search latency during re-indexing.

Most likely in your production setup, you will have a single Azure Search instance shared between CM and CD environments.

You need to factor in that CM should be the server that controls the re-indexing (writing) and CD will most likely be the server doing the queries (reading).

Re-indexing is triggered via the event queue and every server subscribes and reacts to these events. Each server with the out of the box search configuration will cause the Azure Search indexes to be updated.  In a shared Azure Search (or SOLR instance) this only needs to be updated by a single server. Each additional re-index is overkill and just doubling up on re-indexing workload.

You can, therefore, adjust the configuration on the CD servers so that it does not cause re-indexing to happen.

The trick is in your index configuration files to use Configuration Roles to specify the indexing strategy on each server.

 <strategies hint="list:AddStrategy">
 <!--
 NOTE: order of these is controls the execution order 
 -->
 <strategy role:require="Standalone OR ContentManagement" ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync"/>
 <strategy role:require="ContentDelivery" ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual"/>
 </strategies>

Setting the index update strategy to manual on your CD servers will take a big load off your remote indexes.

Particularly if you have multiple CD servers using the same indexes. Each additional CD server would cause additional updates to the index without the above setting.

Tip 7) Rigid Indexes – Have a deployment plan

If your deployment includes additions and changes to the indexes and you need 100% availability of search data, a deployment plan for re-indexing will be required.

Grant chatted about the problem in his post here. To get around this you could consider using the blue / green paradigm during deployments.

  • This would mean having a set blue indexes and a set of green indexes.
  • Using slot swaps for your deployments.
    • One slot points to green in configuration.
    • One slot (production) points to blue in configuration.
  • To save on costs you could decommission the staging slot between deployments.

Tip 8) HttpClient should be a singleton or static

The basic idea here is that you should keep the number of HttpClient instances in your code to an absolute minimum if you want optimal performance.

The Sitecore Azure Search provider actually spins up 2 x HttpClient connections for every single index. This in itself is not ideal and unfortunately, there is not a lot you can do about this code in the core product itself.

In your own connections to other APIs, however, HttpClient SendAsync is perfectly thread safe.

By using HttpClient singletons you stand to gain big in the performance stakes. One great blog article worth reading runs you through the performance benefits. 

It’s also worth noting that in the Azure Search documentation Microsoft themselves say you should treat HttpClient as a singleton.

Tip 9) Monitor your resources

In Azure web apps you have finite resources with your app server plans. Opening multiple connections with HttpClient and not disposing of them properly can have severe consequences.

For instance, we found a bug in the core Sitecore product that was caused by the connection retryer. It held open ports forever whenever we hit out Azure Search plan usage limits.  The result was that we hit outbound open connection limits for sockets and this caused our Sitecore instance to ground to a slow halt.

Sitecore has since resolved the issue mentioned above after a lengthy investigation working alongside the Aceik team. This was tracked under reference number 203909.

To monitor the number of sockets in Azure we found a nice page on the MSDN site.

Tip 10) Make use of OData Expressions

This tip relates strongly to tip 3.  Azure search has some really powerful OData Expressions that you can make use of by a direct connection.  Once you have had a play with direct connections it is surprisingly easy to spin up really fast queries.

Operators include:

  • OrderBy, Filter (by field), Search
  • Logical operators (and, or, not).
  • Comparison expressions (eq, ne, gt, lt, ge, le).
  • any with no parameters. This tests whether a field of type Collection(Edm.String) contains any elements.
  • any and all with limited lambda expression support.
  • Geospatial functions geo.distance and geo.intersects. The geo.distance function returns the distance in kilometres between two points.

See the complete list here.


 

Q&A

Q) Anything on multiple region setups? Or latency considerations?

A) Multi-region setups:   Although I can’t comment from experience the configuration documentation does state that you can specify multiple Azure Search instances using a pipe separator in the connection string.

<add name="cloud.search" connectionString="serviceUrl=https://searchservice1.search.windows.net;apiVersion=2015-02-28;apiKey=AdminKey1|serviceUrl=https://searchservice2.search.windows.net;apiVersion=2015-02-28;apiKey=AdminKey2" /> 

Unfortunately, the documentation does not go into much detail. It simply states that “Sitecore supports a Search service with geo-replicated scenarios” which one would hope means under the hood it has all the smarts to take care of this.

I’m curious about this as well and opened a stack overflow ticket. Let’s see if anyone else in the community can answer this for us.

Search Latency: 

Search latency can be directly improved by adding more replicas via the scaling setting in Azure Portal

replicas

Two replicas should be your starting point for an Azure Search instance to support Sitecore. Once you launch your site you will need to follow the instruction in tip 4 above monitor search latency.  If the latency graph is showing consistent spikes and high latency times above 0.5 seconds it’s probably time to add some more replicas.

0 comments on “Sitecore Helix: Lets Talk Layers”

Sitecore Helix: Lets Talk Layers

Here are some notes from the decision-making process our team uses with regard to what goes where and in which layer. Of course, the helix documentation does go over the guidelines but it’s not until you start working with the architecture that things begin to become clear.

Project Layer

Definition: The Project layer provides the context of the solution. This means the actual cohesive website or channel output from the implementation, such as the page types, layout and graphical design. It is on this layer that all the features of the solution are stitched together into a cohesive solution that fits the requirements.

Comment: The project layer is probably the most straightforward layer to understand. In our project, the modules in this layer remained lightweight and mostly contain razor view files that allow content editors to build up the HTML structure of the pages.

The website content (under home), page templates and component templates are serialized by Unicorn and also live in this layer.

It’s also worth mentioning one particular gotcha you may hit in development to do with template inheritance and you can read more in this blog post.

Feature or Foundation?

Feature Definition: The Feature layer contains concrete features of the solution as understood by the business owners and editors of the solution, for example news, articles, promotions, website search, etc. The features are expressed as seen in the business domain of the solution and not by technology, which means that the responsibility of a Feature layer module is defined by the intent of the module as seen by a business user and not by the underlying technology. Therefore, the module’s responsibility and naming should never be decided by specific technologies but rather by the module’s business value or business responsibility.

Discussion: For our feature modules, we aimed for single concrete features that are independent of each other. They may contain views, templates, controllers, renderings, configuration changes and related business logic code to tie it all together. The point is to always stick to the rule: “Classes that change together are packaged together”.

When building feature modules, it’s also very handy to think about the feature modules removal as you build it. Keep asking yourself how easy would it be to roll back this module and what would I need to do. Doing so will help you to keep those dependencies under control.

Foundation Definition: The lowest level layer in Helix is the Foundation layer, which as the name suggests forms the foundation of your solution. When a change occurs in one of these modules it can impact many other modules in the solution. This mean that these modules should be the most stable in your solution in term of the Stable Dependencies Principle.

 

Discussion: We found that our foundation modules usually consist of frameworks or code that provide a structural functionality to support the web application as a whole. Each foundation module may be used by multiple feature modules to provide them with the support they need to run properly. Our foundation modules contain API calls, configuration, ORM structures (Glass Mapper), initialisation code, interfaces and abstract base classes.

An important point is that unlike feature layer modules, the foundation layer modules can have dependencies to other foundation layer modules. If this was not the case it would be very difficult to construct the foundation layer in the first place.

For the most part, the team can make some fairly quick decisions about what goes where in the initial project planning. And what goes where is fairly obvious after you get familiar with the habitat example project. The main dilemma you’re going to encounter is around where your repositories and services (key business logic) might need to sit.

Business LogicWhat goes where! Help!

Let’s consider the definitions above, they seem straightforward enough. However, in agile projects where things may change rapidly or requirements are not immediately clear (which happens a lot), you’re inevitably going to need to make some judgment calls.

What am I talking about with the above statement, well let’s say one developer codes up a feature module at the beginning of that project. At first, it seems like that particular portion of code is only required by that particular feature. Down the track a requirement surfaces whereby the same business logic needs to be used in another Feature module. Helix rules dictate:

  • Classes that change together are packaged together.
  • No dependencies between feature projects should exist.

A lesser developer may be tempted at this point to duplicate the code in both feature modules to get the job done quickly. This, however, breaks some fairly important fundamental coding standards that many of us try to stick to. Step back to consider the technical debt that duplicate code leave behind vs dependencies between your Helix feature modules.

The solution to this dilemma; it’s time to refactor that feature logic and move it into the foundation layer. After which any feature modules that needs to reference the same code can reference it in the foundation layer.

Remember “with great power comes great responsibility” and this is especially true when touching code in the foundation layer. The code you touch in the foundation layer should be highly stable and well tested as the helix guidelines suggest.

Was the original decision a mistake?

On the flip side of the coin its worth considering that it wasn’t a mistake to put that piece of code in the feature layer to start with. Technically if no one else needed to use the code at the time and it was reasonably unforeseen that anyone else would need to use it, then it probably was the correct call.

Accept that things may change

The team members on your helix project will need to be flexible and accepting of change. I think it’s worth being prepared for some open discussions within your team about what goes in the foundation layer and what goes in the feature layer. It’s certainly going to be open to interpretation and a topic of debate. A team that can work together and be open to a change of direction within their code structure will help the code base stay within the helix guidelines as the project evolves.

Good luck!

 

0 comments on “Azure FED Files missing in Helix”

Azure FED Files missing in Helix

When deploying Sitecore to Azure if you come across an issue where your font files are marked as red in the chrome network tab (essentially they are not found), this may help.

Basically, you need to tell the web server to serve files with a particular extension.

In our case, the font files ending with the extension .woff were not getting picked up which results in a number of strange looking fonts around a deployed website.

The solution is to add this to your web.config:

<configuration>
    <system.webServer>
        <staticContent>
            <mimeMap fileExtension="woff" mimeType="application/font-woff" />
            <mimeMap fileExtension="woff2" mimeType="application/font-woff" /> 
         </staticContent>
    </system.webServer>
</configuration> 

This will tell the Azure web application to serve up files with the given extension as expected.

If you’re working with the Helix Architecture you might like to include this in your Foundation layer.

You could introduce it into a web.config.transform inside your Foundation assets project:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <system.webServer xdt:Transform="InsertIfMissing">
      <staticContent xdt:Transform="InsertIfMissing">
          <mimeMap xdt:Transform="InsertIfMissing" xdt:Locator="Match(fileExtension)" fileExtension="woff" mimeType="application/font-woff" />
          <mimeMap xdt:Transform="InsertIfMissing" xdt:Locator="Match(fileExtension)" fileExtension="woff2" mimeType="application/font-woff" />
      </staticContent>
   </system.webServer>
</configuration>
0 comments on “Helix Development Settings and GIT”

Helix Development Settings and GIT

When you first start out working with the Helix architecture two files will standout as needing to be unique for each developer. Both of these file are part of the habitat example project that you can find here.

  • gulp-config.js (in the root of the project)
  • z.Website.DevSettings.config (\src\Project\Website\code\App_Config\Include\Project\)

These files contain directory locations that are probably going to be unique depending on how a team member sets up their projects.

The trick is that we want these files in git but we also don’t want the developer to check these files in again with there own settings each time.

The solution:

In the respective folders containing the files run the following git command in the cmd prompt:

git update-index --assume-unchanged

Your probably going to want to document this as one of the required setup steps after checking out the project from you git repository.


What if a developer ever needs to change this file and check it in again. They can reverse the process with:

git update-index --no-assume-unchanged