Next Gen Image Compression in Sitecore

Spoiler: This post is not a post about Dianoga, I take a deep dive into Tiny PNG and Kraken.IO integrations into Sitecore. The results are worth checking out at the bottom.


At the start of the year, I’ve picked up where I left off, on page speed. Last year I took a deep dive into attempting to improve the page speed on Sitecore SXA sites by using some of Google’s recommended techniques to structure the page. If you haven’t already seen it, head on over the Sitecore Speedy and see some of the results we achieved.

I’ll be the first to admit that getting really good page speed scores isn’t easy. It takes a lot of different factors to come together. Just as a reminder, here is the main list that I would consider you need to check off to be winning at this game.

1) Introduce image lazy loading

2) Ensure a cache strategy is in place and verify its working.

3) Dianoga is your friend for image compression

4) Use responsive images (must serve up smaller images sizes for mobile)

5) Introduce Critical CSS and deferred CSS files

6) Javascript is not a page speed friend. Defer Defer Defer

For this post, i’m going to look at an alternative to Dianoga. I’m a big fan of Dianoga and have used it over the years to crunch loads of oversized images introduced by Content Editors. I will, however, say that it can add complexity to deployments and CI/CD pipelines and while some claim to have had success in Azure Apps, others have not.

On the flip side, content editors love Tiny PNG, which is one of the most popular image compression website utilities going around. Tiny PNG also has a developer API, so we have used this to build in a compression tool that can be used directly from your Sitecore toolbar.

The button below is hooked up to chat to Tiny PNG API. It will send across your image data and receive a compressed image back for storage.


Full disclosure, I’m not the first person to hook up Tiny PNG to the image library. I could find two other implementations

One will allow you to run a powershell script to connect to the Tiny PNG API and the other is a module to connect to the API on upload.


This implementation of the Tiny PNG API introduces the following variances:

  • A button in the CMS to crunch any single image.
  • A scheduled task that will process any image not already processed.
  • Error handling for when the API limits are reached
  • Logging that outlines which images were processed.
  • Before and After compression information stored in any Image field of choice.
  • A feature toggle to turn the whole feature on/off

All the source code is available at: https://github.com/Aceik/ImageCompression

Now let’s jump in have a look at the results just from crunching a few images down:

Without image compression:

Click to Enlarge Image

To compress the images on the page, we head on over to the “Compress” button in the Media tab that we have introduced.

Click to enlarge

A few examples of compression results taken from homepage images:

Before: 158.4 KB | After: 110.6 KB

Before: 197.8 KB | After: 135.3 KB

Before: 640.0 KB | After: 120.7 KB

After compressing all the images on the page the saving can be seen below.

Click to enlarge

So our total image size saving is 2.4MB – 1.3MB = 1.1MB

A pretty decent saving from just pressing the compress button on 27 homepage images. Also, consider that the user won’t notice any difference in image quality as this method uses lossless compression.


The compression achieved is great for helping us tick off one of the requirements for fast pages with Google. But as we are about to find out Google will likely still complain about two other criteria. When it comes to Google Page Speed insights a page that does not have properly processed images will bring up the following three recommendations:

Here is a break down of how we address each one:

  1. Serve image in next-gen formats – Image formats like JPEG 2000, JPEG XR, and WebP often provide better compression than PNG or JPEG, which means faster downloads and less data consumption. Learn more.
  2. Properly Size images – Your CSS layouts should be responsive and use modern image retrieval techniques that adapt the image size requested based on screen size. Read More
  3. Efficiently encode images – The Tiny PNG integration above will take care of this. This is all about compressing the image to as small as it can get without a visible loss of quality.

So assuming you have already achieved number three using the Tiny PNG integration or another source, let us look at how we can solve the next-gen image requirement.

As a quick side note the testing I did after converting the images to next-gen also ticked item number two above. I don't think this should be relied on however and its best to incorporate responsive images into your projects from the beginning.  

When looking into how to convert images to a next-gen format I opted to target webp. Google has a nice little page explaining the format here.

WebP is natively supported in Google Chrome, Firefox, Edge, the Opera browser, and by many other tools and software libraries.

Once again I opted to look for an API that would provide the conversion for me so that Sitecore could easily connect, send the image and then store the result. All without any extra hosting requirements. I opted to go with Kraken.IO image APIs as they have a free 100MB trial offer and well free is a good price when building proof of concepts. The integration is all available on Aceik’s github repository. Just signup for your own API keys add them to the module settings (in the CMS) and start converting.

To test out just how much this would impact the image payload size for the whole page, I once again converted all the images on the SXA habitat homepage.

Here are the results:

Click to enlarge

So our total image size saving is now 2.4MB – 0.79MB = 1.61MB

The reduction in size from a non-compressed image to a webp formatted image is truly impressive.

A few examples:


Conclusion

I can only conclude by saying that if page speed is really an important factor for your Sitecore project take a look at Tiny PNG. If you want to go next level with your image formats and achieve great compression try out the Kraken.IO API integration as it could be well worth the small subscription fee.


Results

CompressionTotal Image SizeSaving
None2.4 MB
Tiny PNG1.3 MB1.1MB
Kraken.IO (webp)0.79MB1.61MB

Notes:

The module and code mentioned in this blog post are available on Aceik’s Github account. This also contains installations instructions.

GitHub: https://github.com/Aceik/ImageCompression

After installation, your content editors will simply be able to compress and convert images as needed from within the CMS.

Click to enlarge

The Github Readme contains a run down and the standard settings inside Sitecore as shown below:

Accessing the JSS Dictionary in C#

This is a quick post to guide developers through gaining access to the JSS Dictionary in the backend C# code.

Why would you want to be able to do this?

The reason we originally had to do this was that our JSS Angular application had editable content from the dictionary that we also wanted to access in C#. In our particular case, it was to inject the content into an email template that would be sent to the user. To save duplicating content it made sense for both the front end and C# to have access to the same dictionary.

Where do we start?

The following assumes you have a Sitecore instance with JSS installed and a JSS application you are working on. Grab your favourite de-compilation tool (I use ILSpy) and locate the following DLL in the bin folder of your running Sitecore instance:

Sitecore.JavaScriptServices.Globalization.dll

Once you have that open in ILSpy you want to have a search for DictionaryServiceController

public class DictionaryServiceController : ApiController

The following method is what we want to use in our C# code:

public DictionaryServiceResult GetDictionary(string appName, string language)

It takes the unique application name (that belongs to your application) and the language (“en”) as a parameter. As a result, you will get back a dictionary object that you can use to lookup up your content.

This is the Controller that would normally be called via an API on the front end. So how do we call it from normal C# service for instance?

Firstly, the controller has a constructor that has three parameters that are injected via DI (Dependency Injection).

IConfigurationResolver configurationResolver, 
BaseLanguageManager languageManager, 
IApplicationDictionaryReader appDictionaryReader

Using ILSpy once again you can find that the above three parameters are all set up in the DI container via RegisterDependencies.cs in various JSS assemblies. The Controller itself is already registered in the DI Container as well, which is very handy.

If you have a look at showconfig.aspx in the admin tools you can see that a lot of the dependencies are registered via RegisterDependencies.cs

For example:

<configurator type="Sitecore.JavaScriptServices.AppServices.RegisterDependencies, Sitecore.JavaScriptServices.AppServices" patch:source="Sitecore.JavaScriptServices.AppServices.config"/>

Dependency injection is a whole other topic so I will leave that to your personal preference as to how you achieve it. For the purposes of the following complete code example, I have used the Services Attribute style setup. If you want to keep consistency with Sitecore you could also setup via the RegisterDependencies.cs class of your own and use a patch file to kick it off.


Example Service:

using Sitecore.Foundation.DependencyInjection; // Borrowed from habitat
using Sitecore.Diagnostics;
using Sitecore.JavaScriptServices.Globalization.Controllers;

namespace Sitecore.Foundation.JSS.Services
{
    public interface ITranslationService
    {
        string TranslateKey(string key);
    }

    [Service(typeof(ITranslationService), Lifetime = Lifetime.Transient)] 
    public class TranslationService : ITranslationService
    {
        private readonly DictionaryServiceController _controller;
        
        public TranslationService(DictionaryServiceController controller)
        {
            this._controller = controller;
        }

        public string TranslateKey(string key)
        {
            var dictionary = GetDictionary();
            if (dictionary.phrases.ContainsKey(key))
                return dictionary.phrases[key];
            Log.Error("Dictionary key {key} not found", this);
            return string.Empty;
        }

        private DictionaryServiceResult GetDictionary(string appName = "myAppName", string language = "en")
        {
           return _controller.GetDictionary(appName, "en");
        }
    }
}

Above is a simple service that can be used from just about anywhere in your C# code.

Simply change the appName and language as required to access the correct JSS dictionary. Also, remember to publish your app dictionary to the web database or you may get no results.


There we have it, accessing the JSS dictionary from C# in a nutshell. I hope this helps some other folks get this done quickly on JSS builds.

An Introduction to Sitecore Pipelines

What is a Sitecore Pipeline?

In Sitecore pipelines describe a series of discrete steps that are taken to achieve some objective.  If you think about writing code to handle an HTTP request for example you could create a monolithic class that does the job from end to end; however the pipeline approach is a number of classes that can be invoked in order.  First do this and then do that etc. 

There are many Pipelines in Sitecore.  You can view existing pipelines and the processors that they call using Sitecore Rocks in Visual Studio.  Right click on the site connection and choose Manage. Click on the Pipelines tab in the Visual Studio edit pane.  Click on one of the listed pipelines to see all the processors that are executed as part of the pipeline.  There are pipelines with a single processor at one end of the scale to the httpRequestBegin pipeline with 45 distinct steps at the other.

Why are they useful?

Thinking about the monolithic class above it would be very difficult to maintain or modify, It would also be truly massive.  So modularising it would make it much easier to maintain. 

It also makes it much easier to customise.  For example, if we consider a pipeline that has three processors it might be drawn something like

To add to the existing functionality we could include a new step like

Where we add some custom function after Step 1 and before Step 2. 

We could also replace an existing step completely

How are they defined?

Pipelines are defined using XML in sitecore.config.  In the example below we can see the httpRequestEnd pipeline definition.  The three processors are called in the order in which they are listed.  A parameters object is passed between them to provide continuity.  The final processor is also receiving four additional parameters from the config file.

<?xml version="1.0" encoding="utf-8"?>
<sitecore database="SqlServer" xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:security="http://www.sitecore.net/xmlconfig/security/">
…
<pipelines>   
… 
<httpRequestEnd>
      <processor type="Sitecore.Pipelines.PreprocessRequest.CheckIgnoreFlag, Sitecore.Kernel" />
      <processor type="Sitecore.Pipelines.HttpRequest.EndDiagnostics, Sitecore.Kernel" role:require="Standalone or ContentManagement" />
      <!--<processor type="Sitecore.Pipelines.HttpRequest.ResizePicture, Sitecore.Kernel"/>-->
      <processor type="Sitecore.Pipelines.HttpRequest.StopMeasurements, Sitecore.Kernel">
        <ShowThresholdWarnings>false</ShowThresholdWarnings>
        <TimingThreshold desc="Milliseconds">1000</TimingThreshold>
        <ItemThreshold desc="Item count">1000</ItemThreshold>
        <MemoryThreshold desc="KB">10000</MemoryThreshold>
      </processor>
    </httpRequestEnd>
…
    </pipelines>
…
  </sitecore>

How can I work with Sitecore Pipelines?

Existing Sitecore pipelines can be customised as outlined above and it is also possible to create a brand new pipeline from scratch

Customise existing Pipelines

First thing to do is to create a configuration patch to add the new processor class into the pipeline at the desired location.  As you can see from the code it is possible to pass variables to the processor.  Here we are adding a processor called NewsArticleLogEntryProcessor into the httpRequestBegin pipeline after the ItemResolver

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <processor type="Fourbyclub.CustomCode.CustomCode.Pipelines.httpRequestBegin.NewsArticleLogEntryProcessor,Fourbyclub.CustomCode" patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']">
          <NewsArticleTemplateID>{B871115E-609F-44BB-91A4-A37F5E881CA6}</NewsArticleTemplateID>
        </processor> 
      </httpRequestBegin>
    </pipelines>
  </sitecore>
</configuration>

Then we need to create the processor.  Inherit the HttpRequestProcessor and Implement the Process method.  All we are doing here is writing to the log if the requested item is a NewsArticle.

namespace Fourbyclub.CustomCode.CustomCode.Pipelines.httpRequestBegin
{
    using Sitecore.Pipelines.HttpRequest;
    using Sitecore.Diagnostics;

    // TODO: \App_Config\include\NewsArticleLogEntryProcessor.config created automatically when creating NewsArticleLogEntryProcessor class.

    public class NewsArticleLogEntryProcessor : HttpRequestProcessor
    {
        
        // Declare a property of type string:
        private string _newsArticleTemplateID;
        public string NewsArticleTemplateID { get { return _newsArticleTemplateID; } set { _newsArticleTemplateID = value; } }

        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if ((Sitecore.Context.Item != null) && (!string.IsNullOrEmpty(_newsArticleTemplateID)))
            {
                Assert.IsNotNull(Sitecore.Context.Item, "No item in parameters");
                // use util to get id from string property
                if (Sitecore.Context.Item.TemplateID == Sitecore.MainUtil.GetID(_newsArticleTemplateID))
                {
                    // view in log file later, so add FourbyclubCustomCode
                    Log.Info(string.Format("FourbyclubCustomCode: News Article requested is {0} and the item path is {1}", Sitecore.Context.Item.DisplayName, Sitecore.Context.Item.Paths.FullPath), this);
                }
            }
        }

    }
}

Create a new Pipleline

To create and new pipeline is a little more work but still very simple.  The first thing to do is to declare the pipeline with a configuration patch

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <logWriter>
        <processor type="Fourbyclub.CustomCode.CustomCode.Pipelines.logWriter.logWriterProcessor,Fourbyclub.CustomCode" />
      </logWriter>
    </pipelines>
  </sitecore>
</configuration>

The XML above will create a pipeline called logWriter that has a single processor called logWriterProcessor, which will be in the Fourbyclub.CustomCode.dll.

Pipelines must pass a PipelineArgs object to each processor as it is called so that needs to be defined

using Sitecore.Pipelines;

namespace Fourbyclub.CustomCode.CustomCode.Pipelines.logWriter
{
    public class LogWriterPipelineArgs : PipelineArgs
    {
        public string LogMessage { get; set; }
    }
}

At least one processor is needed to do the work of our pipeline

using Sitecore.Diagnostics;

namespace Fourbyclub.CustomCode.CustomCode.Pipelines.logWriter
{
    public class logWriterProcessor
    {
        public void Process(LogWriterPipelineArgs args)
        {
            Log.Info(string.Format("FourbyclubCustomCode: The message was {0}", args.LogMessage), this);
        }
    }
}

Finally we need to invoke the pipeline in our code somewhere.  Instantiate the LogWriterPipelineArgs and set the LogMessage.  Then call CorePipeline.Run and pass it the name of the pipeline and the args.object

var pipelineargs = new LogWriterPipelineArgs();
pipelineargs.LogMessage = "Requested item is not a News Article";
CorePipeline.Run("logWriter", pipelineargs);

Conclusion

Thank you for reading and I hope that this short introduction to Sitecore Pipelines has shown the power of pipelines to customise Sitecore and help to build maintainable code.  However we should always check if there is a way we can implement something using existing Sitecore functionality rather than going for a pipeline as a first resort. As with any customisation does each pipeline or processor added have a potential to increase the challenge of Sitecore upgrades?