Next Gen Image Compression in Sitecore

Spoiler: This post is not a post about Dianoga, I take a deep dive into Tiny PNG and Kraken.IO integrations into Sitecore. The results are worth checking out at the bottom.


At the start of the year, I’ve picked up where I left off, on page speed. Last year I took a deep dive into attempting to improve the page speed on Sitecore SXA sites by using some of Google’s recommended techniques to structure the page. If you haven’t already seen it, head on over the Sitecore Speedy and see some of the results we achieved.

I’ll be the first to admit that getting really good page speed scores isn’t easy. It takes a lot of different factors to come together. Just as a reminder, here is the main list that I would consider you need to check off to be winning at this game.

1) Introduce image lazy loading

2) Ensure a cache strategy is in place and verify its working.

3) Dianoga is your friend for image compression

4) Use responsive images (must serve up smaller images sizes for mobile)

5) Introduce Critical CSS and deferred CSS files

6) Javascript is not a page speed friend. Defer Defer Defer

For this post, i’m going to look at an alternative to Dianoga. I’m a big fan of Dianoga and have used it over the years to crunch loads of oversized images introduced by Content Editors. I will, however, say that it can add complexity to deployments and CI/CD pipelines and while some claim to have had success in Azure Apps, others have not.

On the flip side, content editors love Tiny PNG, which is one of the most popular image compression website utilities going around. Tiny PNG also has a developer API, so we have used this to build in a compression tool that can be used directly from your Sitecore toolbar.

The button below is hooked up to chat to Tiny PNG API. It will send across your image data and receive a compressed image back for storage.


Full disclosure, I’m not the first person to hook up Tiny PNG to the image library. I could find two other implementations

One will allow you to run a powershell script to connect to the Tiny PNG API and the other is a module to connect to the API on upload.


This implementation of the Tiny PNG API introduces the following variances:

  • A button in the CMS to crunch any single image.
  • A scheduled task that will process any image not already processed.
  • Error handling for when the API limits are reached
  • Logging that outlines which images were processed.
  • Before and After compression information stored in any Image field of choice.
  • A feature toggle to turn the whole feature on/off

All the source code is available at: https://github.com/Aceik/ImageCompression

Now let’s jump in have a look at the results just from crunching a few images down:

Without image compression:

Click to Enlarge Image

To compress the images on the page, we head on over to the “Compress” button in the Media tab that we have introduced.

Click to enlarge

A few examples of compression results taken from homepage images:

Before: 158.4 KB | After: 110.6 KB

Before: 197.8 KB | After: 135.3 KB

Before: 640.0 KB | After: 120.7 KB

After compressing all the images on the page the saving can be seen below.

Click to enlarge

So our total image size saving is 2.4MB – 1.3MB = 1.1MB

A pretty decent saving from just pressing the compress button on 27 homepage images. Also, consider that the user won’t notice any difference in image quality as this method uses lossless compression.


The compression achieved is great for helping us tick off one of the requirements for fast pages with Google. But as we are about to find out Google will likely still complain about two other criteria. When it comes to Google Page Speed insights a page that does not have properly processed images will bring up the following three recommendations:

Here is a break down of how we address each one:

  1. Serve image in next-gen formats – Image formats like JPEG 2000, JPEG XR, and WebP often provide better compression than PNG or JPEG, which means faster downloads and less data consumption. Learn more.
  2. Properly Size images – Your CSS layouts should be responsive and use modern image retrieval techniques that adapt the image size requested based on screen size. Read More
  3. Efficiently encode images – The Tiny PNG integration above will take care of this. This is all about compressing the image to as small as it can get without a visible loss of quality.

So assuming you have already achieved number three using the Tiny PNG integration or another source, let us look at how we can solve the next-gen image requirement.

As a quick side note the testing I did after converting the images to next-gen also ticked item number two above. I don't think this should be relied on however and its best to incorporate responsive images into your projects from the beginning.  

When looking into how to convert images to a next-gen format I opted to target webp. Google has a nice little page explaining the format here.

WebP is natively supported in Google Chrome, Firefox, Edge, the Opera browser, and by many other tools and software libraries.

Once again I opted to look for an API that would provide the conversion for me so that Sitecore could easily connect, send the image and then store the result. All without any extra hosting requirements. I opted to go with Kraken.IO image APIs as they have a free 100MB trial offer and well free is a good price when building proof of concepts. The integration is all available on Aceik’s github repository. Just signup for your own API keys add them to the module settings (in the CMS) and start converting.

To test out just how much this would impact the image payload size for the whole page, I once again converted all the images on the SXA habitat homepage.

Here are the results:

Click to enlarge

So our total image size saving is now 2.4MB – 0.79MB = 1.61MB

The reduction in size from a non-compressed image to a webp formatted image is truly impressive.

A few examples:


Conclusion

I can only conclude by saying that if page speed is really an important factor for your Sitecore project take a look at Tiny PNG. If you want to go next level with your image formats and achieve great compression try out the Kraken.IO API integration as it could be well worth the small subscription fee.


Results

CompressionTotal Image SizeSaving
None2.4 MB
Tiny PNG1.3 MB1.1MB
Kraken.IO (webp)0.79MB1.61MB

Notes:

The module and code mentioned in this blog post are available on Aceik’s Github account. This also contains installations instructions.

GitHub: https://github.com/Aceik/ImageCompression

After installation, your content editors will simply be able to compress and convert images as needed from within the CMS.

Click to enlarge

The Github Readme contains a run down and the standard settings inside Sitecore as shown below:

SXA Speedy – Supercharge your SXA Page Speed Scores in Google

We are excited to preview our latest Open Source module. Before jumping into the actual technical details here are some of the early results we are seeing against the Habitat SXA Demo.


Results:

Results

Before:

After

After:

Before
* Results based on Mobile Lighthouse Audit in chrome. 
* Results are based on a local developer machine. Production results usually incur an additional penalty due to network latency.

Want to know more about our latest open source SXA Sitecore module …. read on ….


I’m continually surprised by the number of new site launches that fail to implement Google recommendations for Page Speed. If you believe what Niel Patel has to say this score is vitally important to SEO and your search ranking. At Aceik it’s one of the key benchmarks we use to measure the projects we launch and the projects we inherit and have to fix.

The main issue is often a fairly low mobile score, desktop tends to be easier to cater for. In particular, pick out any SXA project that you know has launched recently and even with bundling properly turned on its unlikely to get over 70 / 100 (mobile score). The majority we tried came in somewhere around the 50 to 60 out 100 mark.

Getting that page score into the desired zone (which I would suggest is 90+) is not easy but here is a reasonable checklist to get close.

1) Introduce image lazy loading
2) Ensure a cache strategy is in place and verify its working.
3) Dianoga is your friend for image compression
4) Use responsive images (must serve up smaller images sizes for mobile)
5) Introduce Critical CSS and deferred CSS files
7) Javascript is not a page speed friend. Defer Defer Defer

The last two items are the main topics that I believe are the hardest to get right. These are the focus of our new module.

Critical_plus_defer

Check out the GitHub repository.

I have also done an installation and usage video.

So how will the module help you get critical and JS defer right?

Deferred Javascript Load

For Javascript, it uses a deferred loading technique mentioned here. I attempted a few different techniques before finding this blog and the script he provides (closer to the bottom of the article) seems to get the best results.  It essentially incorporates some clever tactics (as mentioned in the article) that defer script load without compromising load order.

I also added in one more technique that I have found useful and that is to use a cookie to detect a first or second-time visitor. Second-time visitors naturally will have all external resources cached locally, so we can, therefore, provide a completely different loading experience on the 2nd pass. It stands to reason that only on the very first-page load we need to provide a deferred experience.

Critical + Deferred CSS Load

For CSS we incorporated the Critical Viewport technique that has been recommended by Google for some time. This technique was mentioned in this previous blog post. Generating the Critical CSS is not something we want to be doing manually and there is an excellent gulp based package that does this for you.

It can require some intervention and tweaking of the Critical CSS once generated, but the Gulp scripts provided in the module do seek to address/automate this.

Our module has a button added into the Configure panel inside the Sitecore CMS. So Content Editors can trigger off the re-generation of the Critical CSS when ever needed.

Generate Critical button added to Configure.

Local vs Production Scores

It’s also important to remember that the scores you achieve via Lighthouse built into Chrome on localhost and your non-public development servers can be vastly different than production. In fact, it’s probably safest to assume that non-production boxes give false positives in the region of 10 to 20 points. So it’s best to assume that your score on production will be a little worse than expected.

Conclusion

It’s a fair statement that you can’t just install the module and expect Page Load to be perfect in under 10 minutes.  Achieving top Page Load Speed’s requires many technical things to work together. By ensuring that the previously mentioned checklists are done (Adequate Servers, Sitecore Cache, Image Loading techniques) you are partway over the line. By introducing the deferred load techniques in the module (as recommended by Google) you should then be a step closer to top score.

For more hints please see the Wiki on Github.

This module has been submitted to the Sitecore Marketplace and is awaiting approval.


Author: Thomas Tyack – Solutions Architect / Sitecore MVP 2019

Responsive Images and Sitecore

responsive

By February 2018 Australians already spent more than double the amount of time on smartphones than on their desktop1. With the greater variety of devices consumers use to access sites, it’s important to serve images which appropriately cater to those devices.

“Responsive images” describes a technique where an image is served to the browser depending (usually) on the width of the browser window. Desktop browsers would generally receive a larger version of the image with a greater download size, and mobile devices a smaller version better suiting the smaller display size of the device and being quicker to download.

One of our clients has a Sitecore 7.2 site which uses both MVC and legacy code in ASP.NET WebForms. The legacy code had multiple ways of handling image resizing, mostly inline HTML hard coding of dimensions. My aim was to provide a single C# library function that could work both with legacy code and new development and allow consistent results.

imagetag

In the newer part of the website, the designer had used a JavaScript based image processor which would allow the browser to determine the optimal resolution of the image required and only request an appropriately sized image from the server.

As you can see from the above HTML sample each image tag has multiple URLs in the data-srcset attribute. From this list the JavaScript library determines the optimal size for the image, based on device and browser size, and only requests that version of the image from the server.

Sitecore provides a tool to provide the resized image files based on the request URL and also caches these resized images. We created a helper function that could take a Sitecore image as input and return all of the URLs representing valid image sizes as a delimited string. The valid sizes are determined by a setting in a config file. This return value is used to populate the data-srcset attribute of our img.

ImageCodeSample

Sitecore 7.5 and beyond require a hash code to be added to the URL2 to prevent Denial of Service attacks by making numerous requests for images of various sizes. Because our new helper function now provides a centralised and configurable way to deliver the required URL string, this will be quite easy to change after the upgrade.

References

  1. http://www.nielsen.com/au/en/press-room/2018/february-2018-digital-ratings.html
  2. http://www.seanholmesby.com/images-not-resizing-in-sitecore-7-5-sitecore-8-0/

Sitecore Page Speed: Part 3: Eliminate JS Loading Time

In part 1 & part 2 of our Sitecore page speed blog, we covered off:

  • The Google Page Speed Insights tool.
  • We looked at a node tool called critical that could generate above the fold (critical viewport) CSS code that is minified.
  • We referenced the way in which Google recommends deferring CSS loading.
  • We showed a way to integrate “Above The Fold” CSS into a Helix based project and achieve a page free of render blocking CSS.

In this 3rd part of the series, we will introduce a way to defer the load of all external javascript assets (async).

A reminder that I have committed the sample code for this blog into a fork of the helix habitat example project. You can find the sample here. For a direct comparison of changes made to achieve these page load enhancements, view a side by side comparison here.

Dynamic JS loading Installation Steps:

  1. Inside Sitecore add a new view Rendering that reference the file /Views/Common/Assets/Scripts-3.2.1.cshtml
    • Note down the ID of this rendering and replace in the ID of the rendering in the next step.
  2. Update the Default.cshtml layout to include a new cached rendering.
  3.  @*Scripts Legacy Jquery jquery-3.2.1 *@
     @Html.Sitecore().CachedRendering("{B0DD36CE-EE4A-4D01-9986-7BEF114196DD}", new RenderingCachingSettings { Cacheable = true, CacheKey = cacheKey + "_bottom_scripts" })
    • cacheKey = This variable is something unique that will identify the page. You could use the Sitecore context Item ID or path for example.

Explanation:

The rendering Scripts-3.2.1.cshtml will render out the following javascript onto the page:

var scriptsToLoad = ['//cdnjs.cloudflare.com/ajax/libs/modernizr/2.8.3/modernizr.min.js','//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js','/assets/js/slick.min.js','/assets/js/global.js','/assets/js/Script.js','/assets-legacy/js/lib/lazyload.min.js'];
src="/assets-legacy/js/lib/jquery-3.2.1.min.js" async defer>
  • First of all, it prints out a JS array of all the scripts that this page requires.
    • This is the array of JS files that comes from Themes and Page Assets inside the CMS. If you are familiar with Habitat Helix this list can be content managed inside the CMS.
  • It then instructs the jquery library to be loaded async (which will not block the network download of the page response).
  • Once jquery is loaded, this modified version of jquery contains some code at the end that will read in the list of scripts dynamically and apply them to the page.
    • This is achieved with fairly simple AJAX load calls to the script URLs.

Outcome:

Once integrated successfully you will end up with a page that does not contain any blocking JS network calls.  The Google Page Speed tool should give you a nice score boost for your achievement in reducing initial load time.


Hints and Tips:

Bootstrapping JQuery Code:

  • jquery Document.Ready() function calls may not fire inside dynamically loaded JS files. This is because the JS file is loaded after DOM is ready and it’s too late for the Document.Ready() event at this stage.
  • As a workaround, you could code your JS files to bootstrap on both the Document.Ready() or whenever $ is not undefined.
  • In the case of dynamic loading in this manner, because jquery was loaded first, $ should not be undefined and your code should be bootstrapped successfully.

Debugging in chrome:

  • When dynamically loading JS files they may strangely not appear in the chrome console debugger as you would normally expect.
  • The workaround for this is to add a comment to the top of each JS library
  • //# sourceURL=global.js
  • This will cause the chrome debugger to list the file in the source tab under the “(no domain)” heading.
  • You will then be able to debug the file as per normal.