Serializing a #Sitecore Computed Index Field (When You Need to Pass an Immense Amount of Data)

Recently I worked on a Data Exchange Framework process that involved extracting several records. Each record would eventually be inserted into an external table as they were basically rows. The DEF process was slow creating the records so I created a computed index field to handle creating the records and the DEF process would just need to read and insert the records accordingly. Problem was most computed index fields usually return a small value. In this case it could be several rows of data. The solution was simple though. Serialize a list of objects then deserialize it and let the DEF processor take it from there.

In my example the computed index field is created using an Image field class which contains a list of ImageInfo objects. You can fill in and create the index however you want. See below.

public class
 Image
 {
  [IndexField("imageinformation")]
  publicList<ImageInfo> ImageInformation { get; set; }
  [IndexField("images")]
  public string Imagelist { get; set; }
  [IndexField("_templatename")]
   public string TemplateName { get; set; }
 }

public class ImageInfo
 {
  public string EntityId { get; set; }
  public string caption1 { get; set; }
  public string caption2 { get; set; }
  public string freeformcaptions { get; set; }
  public string url { get; set; }
  }

As stated above, Image contains a list of ImageInfo objects. I used Newtonsoft to Serialize it.

return JsonConvert.SerializeObject(image.ImageInformation); 

This is an example of what the serialized data looks like:

“[{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape3.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape1.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Exterior.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape3.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape4.ashx\”},{\”EntityId\”:\”4aa575d5-af18-466f-8e56-a06494ca6d16\”,\”caption1\”:\”NoCaption\”,\”caption2\”:null,\”freeformcaptions\”:null,\”url\”:\”https://acme.com/-/media/Images/Acme/Products/location/area51/subspecies/images/Streetscape5.ashx\”}]”

This is the computed index in the index configuration.

<fields hint="raw:AddComputedIndexField">
<!-- Custom Computed Fields -->
<!-- Default SC Fields -->
<field fieldName="images">Acme.Feature.Website.ComputedFields.CustomReporting.ImagesComputedField, Acme.Your.Website</field>
 </fields> 

In the process method of the DEF you can then deserialize the field and do your processing.

var imageinfo = JsonConvert.DeserializeObject<List<ImagesComputedField.ImageInfo>>(image.Imagelist);
foreach (var imagedet in imageinfo)
{
 UpdateDataTable(new Guid(imagedet.EntityId), imagedet.caption1, 
 imagedet.caption2,imagedet.freeformcaptions, imagedet.url); 
}

So that is it. Not too bad and will solve a lot of issues where you need to move a lot of data. The DEF process went from 15 plus minutes to less than a minute once this index was created.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s