Scribe Online error connecting to CRM Online

Starting my first escapade into the world of Scribe Online.  In working through a training exercise I ran into an issue that took some figuring, but resolved as follows.

First the error.  When I tried to create a connection to my CRM Online Trial I got an error message thrown by the Manage Connections window that read:

Error - Preconnect failed.  ERROR: The following error has occurred in the Dynamics CRM Connector: Could not load file or assembly 'Microsoft.IdentityModel, Version=3.5.0.0, Culture-neutral, PublicKeyToken=blahblah' or one of its dependencies.  The system cannot find the file specified.

The clue here is the Microsoft.IdentityModel Version 3.5.  This refers to the Windows Identify Foundation 3.5 that was installed when I installed the Scribe Online local agent.  Turns out the install completes but you have to go into the Add Roles and Features Wizard and enable Windows Identity Foundation.

Once that is enabled restart the Scribe Online Agent and then you can create a connection to CRM Online.

SQL database table rowcounts

SELECT o.name, ddps.row_count
FROM sys.indexes AS i
INNER JOIN sys.objects AS o
  ON i.OBJECT_ID = o.OBJECT_ID
INNER JOIN sys.dm_db_partition_stats AS ddps
  ON i.OBJECT_ID = ddps.OBJECT_ID
 AND i.index_id = ddps.index_id
WHERE i.index_id < 2 
  AND o.is_ms_shipped = 0
ORDER BY o.NAME

How to write solid integrations.

If you are integrating two or more systems together how do you manage this?  What is the best practice for making integrations robust, error-proof and efficient?

In my years of writing integrations there is only one way to do this:  Staging.

Source data must be passed through staging for validation and cleansing before pass on to the target.  This can happen any number of ways from capturing the source data into a table for further work, or simply building a number of "views" that work the data to meet the expected target specification.

Of course I'm suggesting the integration use a SQL Staging database as the "middle-ware".  I don't know of any other way to do this, but I'd love to hear of any.

Staging allows capturing source data in its original form if needed.  This can provide an important audit trail that will help in troubleshooting.

Oh, did you think it was possible to avoid errors?  Users will make sure every possible error condition is tested, but it won't happen all at once.  This can only happen over time.  And it's why it's important to create an integration that is adjustable.  It can break from bad data but it should only break once.  When the bad data is identified, which should be easy with staging, an error trap can be created to catch it.  Thus the next time the error happens the integration catches it.

This points to the next important topic - notifications and alerts.  It's critical to notify users of error conditions trapped by the integration.  And there are two tiers of notifications - expected and unexpected issues.  Expected issues are designed (or caught later) and user notification is required to allow them to fix the problem so the integration can proceed.

Unexpected issues are technical in nature - errors that are sudden and expected or are environmental.  These would go to technical resources, such as the user or team responsible for maintaining the integration.

I hope to be able to expand on these concepts in later posts.