This is the second blog post in this series. In the first blog post, I spoke about setting up Azure Search.
In this post, I am going to talk about using a Geo-Spatial lookup query that Azure Search has built in and the associated Index data type called Edm.GeographyPoint
If you’re simply looking for a working example jump over to our GitHub repository that contains the full working helix site.
You can also find the latest version of the Sitecore package here:
Notice on the configuration document that it has a list of EDM data types and it also references the document from Microsoft which has the same list.
A notable exception is the that EDM.GeographyPoint is mentioned in the Microsoft document but not the Sitecore EDM list. So if you need to implement a geo-spatial lookup and get back results based on a latitude and longitude that isn’t possible with the Sitecore Azure Search provider out of the box.
If you dig a little deeper you can see that EDM.GeographyPoint is not in the search code. Have a look at the source code in Sitecore.ContentSearch.Azure.Schema.dll CloudSearchTypeMapper.GetNativeType()
We asked Sitecore support when and if EDM.GeographyPoint would be supported by the Sitecore provider and the answer was no plans in the immediate future.
The solution we came up with to solve this problem essentially involved two steps:
- Getting data of type EDM.GeographyPoint into the Azure Search Index.
- Performing a Spatial Query on the Azure Search Index (using geo.distance) in a timely manner.
To support both of the above operations we had to do a little digging into the core Sitecore Azure search code. Read on to find out about the solution we came up with or jump on over to the GitHub repository to look over the code.
Getting data into the index
As noted above the EDM.GeographyPoint data type simply isn’t coded into the Sitecore DLLs. You can see this by looking at CloudSearchTypeMapper.GetNativeType() inside the DLL Sitecore.ContentSearch.Azure.Schema.dll.
If you’re still not convinced have a look in the Sitecore.ContentSearch.Azure.Http.dll
The next thing we tried was adding Edm.GeographyPoint as a field converter.
Once again we hit a brick wall with this solution as that doesn’t change the fact that Edm.GeographyPoint is not actually a known cloud type in the core DLLs.
If you look at the error message you get when attempting to add Edm.GeographyPoint as a Converter it tells us that CloudIndexFieldStorageValueFormatter.cs is attempting to look up the native EDM type.
Message: Not supported type: ‘Edm.GeographyPoint’
Source: Sitecore.ContentSearch.Azure at Sitecore.ContentSearch.Azure.Schema.CloudSearchTypeMapper.GetNativeType(String edmTypeName) at Sitecore.ContentSearch.Azure.Converters.CloudIndexFieldStorageValueFormatter.FormatValueForIndexStorage(Object value, String fieldName) at Sitecore.ContentSearch.Azure.CloudSearchDocumentBuilder.AddField(String fieldName, Object fieldValue, Boolean append)
Working around this limitation took a bit of trial and error but eventually, we nailed it down to the following steps:
- Created a new search configuration called spatialAzureIndexConfiguration.
- Create a new index configuration based on the new configuration in step 1.
- In the search configuration from step 1.
- Add the following computed field.
<fields hint="raw:AddComputedIndexField"> <field fieldName="geo_location" type="Aceik.Foundation.CloudSpatialSearch.IndexWrite.Computed.GeoLocationField, Aceik.Foundation.CloudSpatialSearch" /> </fields>
- Add a new cloud type mapper to map to EDM.GeographyPoint
<cloudTypeMapper ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/cloudTypeMapper"> <maps hint="raw:AddMap"> <map type="Aceik.Foundation.CloudSpatialSearch.Models.GeoJson, Aceik.Foundation.CloudSpatialSearch" cloudType="Edm.GeographyPoint"/> </maps> </cloudTypeMapper>
- Replace the standard index field storage value formatter with the following the following class.
<indexFieldStorageValueFormatter type="Aceik.Foundation.CloudSpatialSearch.IndexWrite.Formatter.GeoCloudIndexFieldStorageValueFormatter, Aceik.Foundation.CloudSpatialSearch"> <converters hint="raw:AddConverter"> ... </converters> </indexFieldStorageValueFormatter>
- Our new formatter GeoCloudIndexFieldStorageValueFormatter.cs inherits from the formatter in the core dlls and overrides the method FormatValueForIndexStorage. It detects if the field name contains “_geo” and basically prevents CloudSearchTypeMapper.GetNativeType from ever running and falling over.
The computed field GeoLocationField returns a GeoJson POCO which is serialised to the GeoJson format (http://geojson.org/) which is required for storage in Azure Search Index.
Getting data out of the index
The GitHub solution demonstrates two solutions.
- A direct Azure Search query solution using the query syntax:
"geo.distance(geo_location, geography'POINT(long lat)') le radius"
- A LINQ provider that extends the core SearchContext.
Both solutions above at the end of the day allow you to perform an OData Expression query against your Azure Search Indexes. Using the “geo.distance” filter allows us to perform spatial queries using the computing power of the cloud.
The only downside is that search results returned don’t actually include the distance as a value. They are correctly filtered and ordered by the distance yet the actual distance value itself is not returned. We suggest voting for this change on this ticket :). For now, we suggest using the System.Device.Location.GeoCoordinate to figure out the distance between two coordinates.
In researching for possible solutions already out there we had a look at the following implementations for Lucene and SOLR:
- Sitecore Solr Spatial: https://github.com/ehabelgindy/sitecore-solr-spatial
- Sitecore Lucene Spatial: https://github.com/aokour/Sitecore.ContentSearch.Spatial
- Ed Khoze – For the research, he did and help with the initial prototype. Plus google address lookup advice.
- Jose Dominguez – For getting the OData filtering query working.