Sitecore Geo-Spatial Results with Azure Search

This is the second blog post in this series. In the first blog post, I spoke about setting up Azure Search.

In this post, I am going to talk about using a Geo-Spatial lookup query that Azure Search has built in and the associated Index data type called Edm.GeographyPoint

The Azure Search overview can be found here and the configuration document here.

If you’re simply looking for a working example jump over to our GitHub repository that contains the full working helix site.

https://github.com/Aceik/Sitecore-Azure-Search-Spatial

You can also find the latest version of the Sitecore package here:

https://github.com/Aceik/Sitecore-Azure-Search-Spatial/tree/master/lib/Package


Notice on the configuration document that it has a list of EDM data types and it also references the document from Microsoft which has the same list.

A notable exception is the that EDM.GeographyPoint is mentioned in the Microsoft document but not the Sitecore EDM list. So if you need to implement a geo-spatial lookup and get back results based on a latitude and longitude that isn’t possible with the Sitecore Azure Search provider out of the box.

If you dig a little deeper you can see that EDM.GeographyPoint is not in the search code. Have a look at the source code in Sitecore.ContentSearch.Azure.Schema.dll CloudSearchTypeMapper.GetNativeType()

We asked Sitecore support when and if EDM.GeographyPoint would be supported by the Sitecore provider and the answer was no plans in the immediate future.

The solution we came up with to solve this problem essentially involved two steps:

  • Getting data of type EDM.GeographyPoint into the Azure Search Index.
  • Performing a Spatial Query on the Azure Search Index (using geo.distance) in a timely manner.

To support both of the above operations we had to do a little digging into the core Sitecore Azure search code. Read on to find out about the solution we came up with or jump on over to the GitHub repository to look over the code.

Getting data into the index

As noted above the EDM.GeographyPoint data type simply isn’t coded into the Sitecore DLLs.  You can see this by looking at CloudSearchTypeMapper.GetNativeType() inside the DLL Sitecore.ContentSearch.Azure.Schema.dll.

If you’re still not convinced have a look in the Sitecore.ContentSearch.Azure.Http.dll

EdmTypes

The next thing we tried was adding Edm.GeographyPoint as a field converter.

Once again we hit a brick wall with this solution as that doesn’t change the fact that Edm.GeographyPoint is not actually a known cloud type in the core DLLs.

If you look at the error message you get when attempting to add Edm.GeographyPoint as a Converter it tells us that CloudIndexFieldStorageValueFormatter.cs is attempting to look up the native EDM type.

Exception: System.NotSupportedException
Message: Not supported type: ‘Edm.GeographyPoint’
Source: Sitecore.ContentSearch.Azure at Sitecore.ContentSearch.Azure.Schema.CloudSearchTypeMapper.GetNativeType(String edmTypeName) at Sitecore.ContentSearch.Azure.Converters.CloudIndexFieldStorageValueFormatter.FormatValueForIndexStorage(Object value, String fieldName) at Sitecore.ContentSearch.Azure.CloudSearchDocumentBuilder.AddField(String fieldName, Object fieldValue, Boolean append)
….

Working around this limitation took a bit of trial and error but eventually, we nailed it down to the following steps:

  1. Created a new search configuration called spatialAzureIndexConfiguration.
  2. Create a new index configuration based on the new configuration in step 1.
  3. In the search configuration from step 1.
    • Add the following computed field.
    • <fields hint="raw:AddComputedIndexField">
       <field fieldName="geo_location" type="Aceik.Foundation.CloudSpatialSearch.IndexWrite.Computed.GeoLocationField, Aceik.Foundation.CloudSpatialSearch" />
       </fields>
    • Add a new cloud type mapper to map to EDM.GeographyPoint
    • <cloudTypeMapper ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/cloudTypeMapper">
       <maps hint="raw:AddMap">
       <map type="Aceik.Foundation.CloudSpatialSearch.Models.GeoJson, Aceik.Foundation.CloudSpatialSearch" cloudType="Edm.GeographyPoint"/>
       </maps>
       </cloudTypeMapper>
    • Replace the standard index field storage value formatter with the following the following class.
    • <indexFieldStorageValueFormatter type="Aceik.Foundation.CloudSpatialSearch.IndexWrite.Formatter.GeoCloudIndexFieldStorageValueFormatter, Aceik.Foundation.CloudSpatialSearch">
       <converters hint="raw:AddConverter">
         ...
       </converters>
       </indexFieldStorageValueFormatter>
  4. Our new formatter GeoCloudIndexFieldStorageValueFormatter.cs  inherits from the formatter in the core dlls and overrides the method FormatValueForIndexStorage. It detects if the field name contains “_geo” and basically prevents CloudSearchTypeMapper.GetNativeType from ever running and falling over.

 

The computed field GeoLocationField returns a GeoJson POCO which is serialised to the GeoJson format (http://geojson.org/) which is required for storage in Azure Search Index.

Getting data out of the index

The GitHub solution demonstrates two solutions.

Both solutions above at the end of the day allow you to perform an OData Expression query against your Azure Search Indexes. Using the “geo.distance” filter allows us to perform spatial queries using the computing power of the cloud.

The only downside is that search results returned don’t actually include the distance as a value. They are correctly filtered and ordered by the distance yet the actual distance value itself is not returned. We suggest voting for this change on this ticket :). For now, we suggest using the System.Device.Location.GeoCoordinate to figure out the distance between two coordinates.


References:

In researching for possible solutions already out there we had a look at the following implementations for Lucene and SOLR:

  1. Sitecore Solr Spatial: https://github.com/ehabelgindy/sitecore-solr-spatial
  2. Sitecore Lucene Spatial: https://github.com/aokour/Sitecore.ContentSearch.Spatial

Special Thanks:

  • Ed Khoze –  For the research, he did and help with the initial prototype. Plus google address lookup advice.
  • Jose Dominguez – For getting the OData filtering query working.

 

Aceik’s Jason Horne Wins Sitecore “Most Valuable Professional” Award

Technology2016Elite distinction awarded for exceptional contributions to the Sitecore community

Melbourne, Victoria, Australia — February 22, 2016 — Aceik, today announced that Jason Horne, Founder and Technical Lead has been named a “Most Valuable Professional (MVP)” by Sitecore, the global leader in experience management software. Jason Horne was one of only 177 people worldwide to be named a Sitecore Technology MVP this year. There are more than 10,000 certified developers in Sitecore’s global network.

Now in its tenth year, Sitecore’s MVP program recognizes individual technology, digital strategy, and commerce advocates who share their Sitecore passion and expertise to offer positive customer experiences that drive business results. The Sitecore Technology MVP Award recognizes the most active Sitecore experts from around the world who participate in online and offline communities to share their knowledge with other Sitecore partners and customers.

Aceik provides a complete range of Sitecore services; architecture, development, integration, support and maintenance, intranets, expert help, existing site audits and upgrades. As Sitecore specialists, Aceik strives to provide excellence in every Sitecore project we  undertake.

Aceik contributes to the Sitecore community through thought leadership content on our blog, the Sitecore community blog and stack overflow. We co-founded the Sitecore Melbourne meetup group, founded the Sitecore ANZ certified developers group and continue to be involved in the ongoing management of these groups with the goal to share knowledge, learn and build the Sitecore community within Australia and New Zealand.

“We are grateful for the leadership, expertise, and ongoing contributions that Jason Horne has made to the Sitecore community,” said Pieter Brinkman, Director of Developer and Platform Evangelism, Sitecore. “Jason Horne has demonstrated a mastery of our technology and exemplifies the spirit of Sitecore.”

Sitecore’s experience platform combines web content management, omnichannel digital delivery, customer insight and engagement, and strategic digital marketing tools into a single, unified platform. The platform is incredibly easy to use, capturing every minute interaction—and intention—that customers and prospects have with a brand, both on a website and across other digital channels. The end-to-end experience technology works behind the scenes to deliver context marketing to individual customers, so that they engage in relevant brand experiences that earn loyalty and achieve results.

Aceik is a boutique web development company specialising in the technical implementation and support of Sitecore solutions. 100% of our work is Sitecore related and we pride ourselves on being experts in our field.

Jason Horne, Founder and Technical Lead
Aceik
jasonhorne@aceik.com.au
0426971867

Automate friendly field descriptions

58845601

As you may well know when creating and naming fields in Sitecore we also need to create a friendly title for the content editors. By default Sitecore doesn’t make this as easy as it could be and therefore there are a number of naming conventions and solutions for this simple but fundamental issue.

I have always taken the approach of camel casing my field names. It makes sense from a coding perspective. It is a good practice as most developers understand this convention and its benefits. Also when integrating with ORMs such as glass a camel case field name will map directly to the property name of the class without additional mapping attributes.

The obvious issue with using camel case field names is that the default value the content editors see is the field name. Camel case is great for developers but not as good for content editors, so we need to manually write a friendly display description for the editors. This leads to double entry of field name and field description and then keeping this consistent.

Solution:

Auto generate a friendly field description based on the field name by splitting the camel case and adding spaces. For additional information sometimes required we use the help text option.

Example:

Field Name:

PageTitle

Generated Field Description:

PageTitle1

Addition information:

PageTitle2

Implementation:

Tap into the item on saving event and add some basic code to auto generate a display title based on the field name.

namespace Aceik.Framework.SC.Extensions.Events
{
    using System;
    using System.Text.RegularExpressions;

    using Sitecore.Data;
    using Sitecore.Data.Items;
    using Sitecore.Events;

    /// <summary>The item on saving events.</summary>
    public class ItemOnSavingEvents
    {
        /// <summary>The field title name.</summary>
        private const string FieldNameTitle = "Title";

        /// <summary>The field template id.</summary>
        private readonly ID fieldTemplateId = new ID("{455A3E98-A627-4B40-8035-E683A0331AC7}");

        /// <summary>The on item save.</summary>
        /// <param name="sender">The sender.</param>
        /// <param name="args">The args.</param>
        public void OnItemSaving(object sender, EventArgs args)
        {
            var contextItem = Event.ExtractParameter(args, 0) as Item;

            if (contextItem == null)
            {
                return;
            }

            if (contextItem.TemplateID == this.fieldTemplateId && contextItem.Fields[FieldNameTitle] != null)
            {
                contextItem.Fields[FieldNameTitle].SetValue(SplitCamelCase(contextItem.Name), false);
            }
        }

        /// <summary>The split camel case.</summary>
        /// <param name="input">The input.</param>
        /// <returns>The <see cref="string"/>.</returns>
        private static string SplitCamelCase(string input)
        {
            return Regex.Replace(input, "([A-Z])", " $1", RegexOptions.Compiled).Trim();
        }
    }
}

Patch in the following config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
  <events timingLevel="custom">
      <event name="item:saving">
        <handler patch:instead="*[@type='Sitecore.Tasks.ItemEventHandler, Sitecore.Kernel']" type="Aceik.Framework.SC.Extensions.Events.ItemOnSavingEvents, Aceik.Framework.SC.Extensions" method="OnItemSaving"/>
      </event>
    </events>
  </sitecore>
</configuration>

As the description is being auto generated it restricts your ability to add a custom description, however you can add additional information to the display title by adding help text. This blog explains it well: http://firebreaksice.com/sitecore-item-and-field-names/