I did a blog series on DEF and I did a few more follow up blogs. You can find the previous blog on DEF here. After I updated my Reddit Feed example to the latest DEF and doing a presentation on it I decided to do an updated blog.

You can find the my Sitecore Market Place module here. You can find the code in GitHub here.

So, let’s get started.

Overview (what is covered in this blog):

• What is Data Exchange Framework?
• How Can it be Used?
• Sitecore Setup
• Backend Setup
• How to Run
• Scheduling Data Exchange Framework Batch Jobs
• Final Thoughts

From Sitecore:

Sitecore Data Exchange Framework is designed to facilitate the transfer of data between systems. It allows you to define the logic needed to read data from a source system, transform that data into a format that is compatible with a target system, and write the transformed data into a target system. Developers can build connectors that allow 3rd party systems to serve as source and target systems.

How can it be used?

• Importing Data from One Source into Sitecore (example rss feed, reddit feed etc…).
• Exporting Data from Sitecore into Another System (external SQL tables)
• Reading and Processing xDB contacts.
• Anything data related. The possibilities are endless.

Sitecore Setup

Below is the overview on how the Data Exchange tenant was setup.

Expanding the Pipeline Batch, you will see the following:

So, let’s start dissecting this stuff.

Sitecore Setup – Value Accessor Sets

• Used to map field values from the source (Reddit).
• Used to map source values from Sitecore.

Each Accessor set has a field. I made these fields the same names I was getting from the feed. It made things a little easier to remember.

For the Sitecore field you will use the fields of a template you create. In this case it will be a Reddit item template.

The field value is going to point to the item template of the Reddit feed item. I kept the field naming consistent with the feed.

To bring those together we will use a value mapping set. This will make the connection from the Reddit feed field to the Sitecore field. In this example Title of the Value Accessor Set for the Reddit feed chosen.

In this example created a template that contains settings needed for the field. Template inherits from the endpoint base template. For the Sitecore endpoint I created and used the default values.

The converter type points to a created method in the code behind. The method is used to retrieve the blog path.

Converter Type is defaulted.

Backend Setup – Overview

This is a snapshot of each folder that was created and the corresponding class file.

RedditFeedFieldValueAccessorConverter.cs

The Supported Ids above, the template id ({68BD9AAD-635F-40F3-9ACD-711662C59EEC}) being set refers to the following template. This template inherits from the Value Accessor template installed by the DEF package.

So how this corresponds to Sitecore is simple. A Value Accessor Set item is created and the items underneath it uses this template. In the RedditFeedFieldValueAccessorConverter class it is setup to get the value of that field using the combination of the template and the field name defined in the RedditFeedFieldValueValueAccesorItemModel. You will see how this is tied into other parts of the DEF process as we go along.

In the code you will notice a ValueReader and a ValueWriter are set. The ValueReader is used to read in the feed and match the fields for each Reddit blog item. The field definition is contained in the stringValue. The ValueWriter defines a new PropertyValueWriter class. That class is part of the DEF and used to convert the field values. (This code was changed in 2.1 versus previous versions so you might see this done differently if using an earlier version).

RedditFeedValueReader.cs

This class will process the Reddit Post item and match field name with value. Title, AuthorName, SelfText etc… are field names in the Reddit feed. I believe you can name these differently, but I prefer to keep the same name as it is in the Reddit post item.

RedditEndPoint.cs

The Reddit End Point will have the information for the Blog Path in Reddit. The TemplateId defined points to the RedditEndPoint template. Settings are returned and added to the plugin. The plugin is required to be implemented by the pipeline converter and processor.

This is the model plug in code:

This is values of the Reddit Endpoint. You can see that the Converter Type points to the RedditEndPoint class.

More about this later, but as you can see in the Reddit Pipeline Step the Reddit Endpoint is selected. Now you can see how this all ties together.

Looking in Sitecore you can see where the Converter Type points to the Endpoint Converter and the Processor Type which will get called and receive the end point information and do the Reddit Feed processing.

RedditEndPointConverter.cs

The converter is one of the most crucial parts of the DEF. This is what ties together the pipeline step of the DEF. The SupportedIds value is the template id of the Reddit Pipeline step. As you can see the endpoint settings are passed into the pipeline step.

RedditItemsProcessor.cs

In Sitecore under Pipelines\Reddit Pipeline\Reddit Pipeline Step you will see under Processor Type the RedditItemsProcessor defined.

The next images shows the code for the processor. You will see that the Required EndpointPlugin is defined for RedditSettings.

Endpoint, PipelineStep and PipelineContext should all have values at this point. As you will notice the ILogger parameter is used to set up any custom logging.

If you look at the method RedditFeed, what that does is create a new Reddit class using RedditSharp. It then uses the blogpath setting from the endpoint to retrieve the blog feed.

A collection of reddit feed blogs is then created. It will use fieldnames we defined earlier to match the Reddit feed fields to the fields defined for the new Reddit Sitecore item.

How to Run

Below you will find a Pipeline Batch that was created to run all the necessary steps to process and import the data into Sitecore.

Below are the steps that get run under Pipelines. You can also see above that the Reddit Pipeline is selected.

One of the things I found difficult with DEF now that I have used it quite a bit is getting the correct settings right on each step. I got some help from an expert on Slack. So hopefully this will help someone else.

The first setting Iterable Data Location is important because that is your data that was read in. The Data Location is usually set to Pipeline Context Source.

The Identifier Value Accessor is set to your unique key for your data. Identifier Object Location is usually set to Pipeline Context Source.

Resolved Object Location will usually be set to Pipeline Context Target.

See setting below for mapping. Source and Target are common.

Finally you have the Item Location that will be used to map to the Sitecore Endpoint.

When you first click on the Pipeline Batch you will notice a group of buttons on the ribbon. If something is not correctly setup these buttons will be disabled. To run the Pipeline Batch just click on the Run Pipeline Batch button. While it is running the button will grey out and you will see the Stop Pipeline Batch button enabled. After it is finished running the Run Pipeline Batch will be enabled again.

Results

If you need information on scheduling the DEF process check out my previous blog here.

Final Thoughts

• The Data Exchange Framework should always be considered when importing/exporting data in Sitecore.
• It does not completely replace tools such as RAZL as the process would be more manually intensive.
• The possibilities are endless.

I hope this guide helps those who need to get started with the Data Exchange Framework.