Configure the SDK and Dynamics 365 Connector

 

Updated: November 1, 2016

Applies To: Dynamics Marketing

System_CAPS_importantImportant

Microsoft Dynamics Marketing is scheduled to be retired on May 15, 2018. After that date the service will no longer be available. Please plan accordingly. For details, see the blog post Microsoft Dynamics Marketing service will be discontinued, and learn what’s coming next.

Microsoft Dynamics Marketing provides both a connector for integrating Microsoft Dynamics 365 with Dynamics Marketing and a software development kit (SDK) for managing the database remotely and integrating with other third-party systems. This topic describes the settings provided by Dynamics Marketing for configuring these two types of integrations. In addition to the settings described here, you must also collect information and configure the external systems you’re integrating with.

  • To fully implement the Microsoft Dynamics Marketing Connector for Microsoft Dynamics 365, you must download and install the connector package and configure both Dynamics Marketing and Dynamics 365 to enable the communication via an Microsoft Azure service bus. For complete details about how to download the connector package and make the other required configuration settings, seeConnect Microsoft Dynamics Marketing to Dynamics 365.

  • To make use of the SDK, you must set up your development environment to use it, learn about the various commands provided by the API, and configure Dynamics Marketing to enable it. For complete details about how to download and work with the SDK, visit the Dynamics Marketing Developer Center.

You must be a Dynamics Marketing administrator to work with the SDK and Microsoft Dynamics Marketing Connector integration settings. More information:Work with user accounts and staff contacts

Both the Microsoft Dynamics Marketing Connector and the SDK use Microsoft Azure to facilitate communication to and from Dynamics Marketing. Depending on your scenario, you may be able to use built-in managed queues (which are included with your Dynamics Marketing subscription), or you may need to purchase your own Azure subscription. Requirements are summarized in the following table.

Integration type

Azure subscription

Description

Microsoft Dynamics 365 (online)

Not required

For this scenario, Dynamics Marketing supports both custom and managed queues. We recommend that you use the managed queues, which do not require an Azure subscription and allow the Azure service bus to be managed by Microsoft with minimal intervention by you. However, you could instead use your own Azure subscription, which will give you full administrative ownership and improve your ability to monitor the queues.

Microsoft Dynamics 365 (online) Government

Optional

As with Microsoft Dynamics 365 (online), Dynamics Marketing supports both custom and managed queues for integrating with Microsoft Dynamics 365 (online) Government. However, Dynamics Marketing is not available in the government cloud, and the connector only works with the standard Azure cloud, so the integration will be a hybrid government/non-government cloud solution. Both managed and subscribed (non-government) Azure queues work correctly when connecting Dynamics Marketing to Microsoft Dynamics 365 (online) Government. Azure Government queues can’t currently be used with Dynamics Marketing.

Dynamics 365 (on-premises)

Required

To connect to Dynamics 365 (on-premises), you must provide a service bus from your own Microsoft Azure subscription. The Azure subscription that you use does not necessarily need to be in the same Microsoft Office 365 tenant as Dynamics Marketing.

SDK service

Required

To use the SDK, you must provide a service bus from your own Microsoft Azure subscription. The Azure subscription that you use does not necessarily need to be in the same Microsoft Office 365 tenant as Dynamics Marketing.

Using OData as an API (unofficial mechanic)

Required

It is possible, but not officially supported, to use OData feeds as an API. To do this, you must have an Azure subscription in the same Microsoft Office 365 tenant as Dynamics Marketing. This lets you to register an application in Windows Azure Active Directory (WAAD) that has permissions to communicate with Dynamics Marketing.

System_CAPS_noteNote

You need to create your Azure Service Bus differently depending on which authentication method you use to connect to them.

To work with the integration settings, go to Settings > Administrator > Integration Options. This brings you to the Integration Options page, which provides the settings described in the following sections.

The Administration section displays the current status of the SDK and the connector and enables you clear all integration settings.

If you would like to reset your integration settings, thus disabling the SDK and the connector, click the Clear all integration settings button Reset to default mapping button at the top of the Administration section. All settings (other than Dynamics 365 / Dynamics CRM Endpoint settings) will be deleted and reset to default values (your Dynamics 365 / Dynamics CRM Endpoint settings will be remembered but will have no effect until you reactivate the services). There is no undo for this operation. On choosing this button, you will be shown a warning and prompted to confirm the action.

System_CAPS_warningWarning

If you reset the integration settings, change tracking will also be cleared for the connector. This means that you must do an initial sync if you later re-enable the connector.

Use the Services section of the Integration Options page to enable or disable the SDK or connector services. You can also view their current status here.

Here you see the current status of the following services:

  • MDM Listener: receives incoming records from Microsoft Dynamics 365

  • MDM Publisher: sends records to Microsoft Dynamics 365

  • Dynamics 365 Listener: receives incoming records from Dynamics Marketing

  • SDK Service: enables external system to communicate with Dynamics Marketing via its API

Use the buttons at the upper right of the Services section to refresh the display and change the status of these services.

Button

Description

Refresh button
Refresh

Updates the status display to reflect changes that may have occurred while the Integration Settings page was open.

Restart button
Restart

Restarts the connector and SDK services.

Disable service button
Disable Dynamics 365 Connector services

This button only appears when the connector services are running. Select this button to disable the connection to Microsoft Dynamics 365. You will be prompted to confirm the operation.

Start service button
Enable Dynamics 365 Connector services

This button only appears when the connector services aren’t running. Click this button to enable the connection to Microsoft Dynamics 365.

Disable service button
Disable SDK service

This button only appears when the SDK service is running. Click this button to disable the SDK service. You will be prompted to confirm the operation.

Start service button
Enable SDK service

This button only appears when the SDK service is not running. Click this button to enable SDK the service.

Use the Dynamics 365 / Dynamics CRM Endpoint settings to enable Dynamics Marketing to find and sign into your Dynamics 365 instance via the connector. The required settings vary depending on whether you are connecting to Dynamics 365 (on-premises) or Dynamics 365 (online). For complete details about how to use these settings, see Enable and configure the connector.

Use the settings in the Service Bus section to view and configure how the connector will find, authenticate, and use its Azure Service Bus message queues, which temporarily store data as they are transferred between Microsoft Dynamics 365 and Dynamics Marketing.

For complete details about how to work with the settings here, see Configure Azure Service Bus settings

Use the Health Check section to view the current health of your connector setup and to initiate a new health check after making configuration changes. A health check is also run automatically when the service starts up. The health check tests whether:

  • the administrative service is running

  • the Dynamics Marketing Publisher can send messages

  • the Dynamics Marketing Listener can read messages

  • the Dynamics Marketing Listener can access database

  • the Dynamics 365 Publisher can request metadata

  • the Dynamics 365 Listener can read from the queue

  • the Dynamics 365 Listener can access Dynamics 365 via SDK and has the proper permissions

  • plug-ins can access the queue

  • the Dynamics 365endpoint has been configured correctly

To run or rerun the health check, click the Run button Run button at the top of the Health Check section. All the errors encountered during the health check will be captured in a downloadable log file. To download the logs click the Download button Download button next to the Logs label in the Health Check section.

System_CAPS_noteNote

The Status field shows the status of the last health check performed on the system. After you modify the settings, you should rerun the health check to ensure that your new settings are working as expected.

Use the Initial Synchronization section to read the current sync status and to initiate a new initial sync.

The initial synchronization process resets the sync system and will typically take significant time and resources due to the large volume of data likely to need transfer. Usually you will do this just one time immediately after setting up the connector. Once the system is up and running, the sync operation is automatic and incremental so you will not usually need to start it manually or reinitialize the system.

System_CAPS_importantImportant

The initial sync operation demands significant time and resources. Avoid running this operation during peak usage times for your Dynamics Marketing and/or Dynamics 365 instances. Because of this, the initial sync is not run automatically after you set up the system; instead, it waits for you run the initial sync manually.

To run an initial sync, make all of the required settings in the Services, Dynamics 365 / Dynamics CRM Endpoint, Service Bus and Mappings sections and then click the Run button Run button at the top of the Initial Synchronization section.

Use the Mapping section to map data fields between the Microsoft Dynamics 365 and Dynamics Marketing databases on an entity by entity basis.

Use the buttons at the upper right of the Mapping section to work with the mappings as described in the following table.

Button

Description

Edit button
Edit

Opens the Interoperation Mapping page. Use this page to map fields between the two systems and to fine-tune a few other aspects of the integration.

Download button
Download field configuration

Downloads the current mapping settings as an XML file.

Upload button
Upload field configuration

Lets you select and upload a local XML mappings file. Typically this will be an edited version of a previously downloaded mapping file, thus helping to ensure that the XML format is correct.

Reset to default mapping button
Reset to default mapping

Returns all mapping settings to their default values.

Click the Edit button Edit button at the top of the Mapping section to open the Interoperation Mappings page. Use this page to map fields between the two systems and to fine-tune a few other aspects of the integration. The settings provided here are described in the following table.

Setting

Description

MDM Entity drop-down list

Use this drop-down list to choose the Dynamics Marketing entity you want to map. You can think of each entity as a simple database table that supports a given feature area, but the underlying data structure is actually more complex. The Interoperation Mappings page reloads to show the mappings and other settings for your selected entity (all of the other settings on this page apply to the entity you select here).

Remote Entity value

This read-only field shows the Dynamics 365 entity that maps to the currently selected MDM Entity. This is selected automatically and can’t be changed.

Threshold Field drop-down list and Threshold Value field

Use these settings to establish a condition that must be met for a record to be synced from Dynamics Marketing to Dynamics 365.

For example, the default mapping uses this feature to ensure that only sales-ready leads are synced back to Dynamics 365. To accomplish this, the LeadManagement entity from Dynamics Marketing is configured with a Threshold Field of “SALESREADY” and a Threshold Value of “1”. This means that leads with a sales-ready value other than one will not be synced.

The Threshold Field drop-down list is hard-coded to include only those Dynamics Marketing fields that make sense for use with this feature. Often, these are fields used internally by Dynamics Marketing that would not make sense to sync to Dynamics 365, which is why the fields available in the Threshold Field drop-down list are often different from those available for mapping.

System_CAPS_warningWarning

Misconfiguring these fields can cause some or all records to fail to sync. Only use these settings selectively and with purpose. In most cases, you should leave them blank.

Sync Interval (minutes)

This setting is only provided when you have chosen ListEntity from the Microsoft Dynamics Marketing Entity drop-down list. It establishes the schedule by which the connector synchronizes your marketing lists, and must be set to allow enough time for non-list data to synchronize between list syncs. The default setting is 10 minutes, which works best in most cases. However, if you are synchronizing very large marketing lists, then you may need to increase this interval to keep your list data from clogging the connector queues. Using too short an interval won't cause data loss, but it could degrade synchronization performance of all types of records.

Enable Entity check box

Select this check box to enable mapping to the currently selected Microsoft Dynamics Marketing Entity. Clear this check box to disable the mapping. Entities for which this check box is cleared will not be synced between systems.

Status value

Displays a short status message related to the mapping, including the success/failure/result of any recent operations (such as file upload)

Dynamics Marketing Field column

This column lists each field that exists on the Dynamics Marketing side of the mapping. You can change any of these using the drop-down list for any row. You are most likely to need this setting when you add new rows to the table.

Dynamics 365 / Dynamics CRM Field column

This column lists each field that exists on the Dynamics 365 side of the mapping. You can change any of these using the drop-down list for any row. You are most likely to need this setting when you add new rows to the table.

New button
New toolbar button

Click this button to add a new row to the table for the current MDM Entity. Then use the Dynamics Marketing Field and Dynamics 365 / Dynamics CRM Field drop-down lists to map the fields for the new row.

Delete button
Delete toolbar button

To delete a row from the mapping, select the check box for the target row and then click this button.

Change Option Value Mapping button
Change option value mapping table-row button

This button appears for rows that map fields expecting one of just a few possible values (for example, ENUM fields, typically presented as drop-down lists for input). Click this button to open the Customize Option Mapping pop-up window, which shows a table that includes a row for each possible option name, as defined in Dynamics Marketing. Use this window to map these values between systems as follows:

  • Option Name: shows the display name used in Dynamics Marketing. (These are typically established using Dynamics Marketing categories; see also Create custom drop-down values and folders).

  • Map to Value: enter the analogous value used in Dynamics 365.

If you prefer to define your mappings all at once using XML rather than working on the Interoperation Mappings page, you can easily do so by downloading and examining the current mappings as an XML file, editing the download and then uploading the result.

System_CAPS_warningWarning

This operation requires that you already know how to work with XML, and can easily understand how to edit an XML file based on its existing content to produce a valid result.

To edit your mapping via XML:

  1. Click the Download button Download button at the top of the Mapping section. Your browser saves the file either in your default download folder or in a folder that you choose.

  2. Open the downloaded file in a text of XML editor and modify it as needed.

  3. Return to the Integration Settings page in Dynamics Marketing and click the Upload button Upload button at the top of the Mapping section. Your web browser opens a file browser window; use this window to locate and select your modified XML file.

Dynamics Marketing maintains a log of messages generated by the connector and SDK. You can download one or both of these logs to get more information about how your solution is working and to troubleshoot problems. Click either of the two Download buttons Download button at the top of the Logs section to download the connector or SDK log, respectively.

Use the SDK Service Settings section to set up connections to the Azure Service Bus queues for handing communications to and from the SDK and to enable and disable various aspects of the service.

To set up Azure Service Bus queues with SAS authentication for the SDK service:

  1. If you haven't already done so, use the Azure Management Portal to create the Azure Service Bus namespace where you will run message queues for communicating to and from the Dynamics Marketing SDK service. Dynamics Marketing will create the queues themselves later in this procedure.

  2. Find the policy name and primary key generated for your Azure Service Bus namespace and paste them into a temporary text file so you can use them later in this procedure. You can find these by checking your Azure Service Bus configuration in the Azure Management Portal or by looking up its connection string, which shows them as values for its SharedAccessKeyName and SharedAccessKey parameters, respectively.

  3. Sign into Dynamics Marketing as an administrator and go to Settings > Administration > Integration Options.

  4. Check the Services section to make sure the SDK is enabled and running. If it isn't running, click on the Enable SDK service button Start service button.

  5. Click the Settings button Dynamics 365 web client Settings button at the top of the SDK Service Settings section to open the Configure Windows Azure for Dynamics SDK Service pop-up window.

    Configure Windows Azure for Dynamics SDK Service by using SAS

  6. Enter the following values:

    • Authentication: If shown, choose SAS (ACS authentication is no longer supported).

    • Azure namespace: Enter the Azure Service Bus namespace where you want to run your SDK queues.

    • Key name: Paste in the Azure Service Buspolicy name (SharedAccessKeyName) that you copied at the start of this procedure.

    • Key value: Paste in the Azure Service Busprimary key (SharedAccessKey) that you copied at the start of this procedure.

    • Queue name for incoming requests and Queue name for outgoing responses: Enter a unique name for each of these two required queues. Each of these queues must be unique and not shared or used for any other purpose. As long as you choose queue names that don't already exist in your specified Azure Service Bus namespace, Dynamics Marketing will create and configure new queues with the names you specify, exactly as required. If you enter existing queue names, then Dynamics Marketing will use the existing queues instead of creating new ones (not recommended).

    System_CAPS_importantImportant

    If you want to use existing queues rather than letDynamics Marketingcreate them for you (not recommended), then keep the following important points in mind:

    • Don't use queues that are also used by the connector because that will produce many errors, both during sync and SDK operations, and can result in data loss.

    • Your queues must be configured as follows: recommended max size = 5GB; duplicate detection disabled; sessions disabled; partitioning disabled.

    • Don't set up authentication on the individual queues. Dynamics Marketing authenticates against the Azure Service Bus namespace itself and expects those credentials to be inherited by the queues it uses.

  7. Click Submit to save your settings and close the pop-up window.

System_CAPS_noteNote

The current Azure Management Portal doesn’t allow you to view the configuration of the “Require Session” flag. To see the flag, query the QueueDescription class, or use Service Bus Explorer (or similar).

After you have successfully enabled the SDK and its Azure queues, the SDK Service Settings section will display information about your Azure connection and will also include a set of check boxes for enabling or disabling read and write permissions for each of several request groups. Each of these groups maps roughly to a feature area of Dynamics Marketing. Select the check box for each type of operation you would like to expose to the SDK. You should only grant those permissions that your custom solution actually requires.

If you use the SDK to trigger the sending of marketing emails to a specified set of email addresses, Dynamics Marketing will check to see if any contacts exist for the incoming email addresses. If a match is found, the email message is linked to the found contact for purposes of email tracking and lead scoring. For addresses where no match is found, Dynamics Marketing creates a new contact whose record includes only the email address, plus associated email tracking, lead scoring, and other records. Use the Enable Contact Reconciliation check box in the SDK Service Settings section to control how new contacts created in this way will be handled by Dynamics Marketing and synchronized to Microsoft Dynamics 365.

  • When contact reconciliation is enabled: new contacts created in this way are initially invisible in Dynamics Marketing until a contact with a matching email address is synchronized from Dynamics 365, at which time the invisible contact will be merged with the newly synced contact, thus associating all emails and leads with that contact. The invisible records aren’t synced until they are matched.

  • When contact reconciliation is disabled: new contacts created in this way are visible in Dynamics Marketing and will be synced back to Dynamics 365 as new contacts. This ensures that all such contacts are visible and usable in Dynamics Marketing, but may result in the creation of duplicate contacts in Dynamics 365.

System_CAPS_noteNote

If your Dynamics Marketing or Dynamics 365 database includes multiple contacts that share a given email address, contact reconciliation will use the first matching contact it happens to find.

Show: