Cannot insert duplicate key row in object ‘fim.ObjectValueReference’ with unique index ‘IX_ObjectValueReference_ObjectKey_AttributeKey-Filtered_Multivalued’.

I was working on a custom workflow activity today that was using the Update Resource activity to update a multivalue field in an object. Unfortunately, the request was coming up as ‘failed’ and in the Event Viewer, I was seeing the following error:

Microsoft.ResourceManagement.WebServices.Exceptions.UnwillingToPerformException: Other —> System.Data.SqlClient.SqlException: Reraised Error 50000, Level 14, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 50000, Level 14, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 2601, Level 14, State 1, Procedure UpdateResource, Line 515, Message: Cannot insert duplicate key row in object ‘fim.ObjectValueReference’ with unique index ‘IX_ObjectValueReference_ObjectKey_AttributeKey-Filtered_Multivalued’. The duplicate key value is (9101, 87654).

When I investigated the [fim].[UpdateResource] stored procedure and [fim].[ObjectValueReference] in the FIMService database, I could see that 9101 correlated to the object I was trying to update and that 87654 correlated to the multivalue attribute I was trying to update. Next I looked at my code, and it all seemed to be as I expected:

update parameters insert

 

Lastly, I looked at my schema.. and sure enough, my attribute wasn’t flagged as multivalue – somewhere in re-creating my schema, I’d forgotten to check that box. Deleting and re-creating the attribute and its binding resolved my issue, as well as another issue I had when I was trying to read the attribute back – because clearly you can’t read a list of UniqueIdentifiers from a single-value attribute.

Silly mistake, so why do I share it here? Because someone out there may Google this error one day, and this might help them.

Posted in FIM Troubleshooting

Microsoft Releases SharePoint User Profile, Generic LDAP and PowerShell Connectors to Connect

On March 28, 2013, Microsoft put out Release Candidates for several new FIM Synchronization Service Connectors, including:

  • SharePoint User Profile Connector
  • Generic LDAP Connector for Open LDAP
  • PowerShell Connector

The Release Candidate downloads are available via Microsoft Connect, along with the necessary upgrades to the FIM Sync Service, a technical reference for the SharePoint/LDAP connectors and some sample scripts (primarily SharePoint-related) for PowerShell

These three connectors are probably three of the most-demanded connectors, and candidates for custom connectors, so it’s good to see Microsoft taking a proactive approach in developing these.

Note that the connectors are still in Release Candidate only, so you shouldn’t be installing them in production environment yet.

Posted in ECMA2, FIM Connectors, FIM Hotfixes, Service Packs & Updates, FIM Release Candidates, FIM Resources, FIM xMA

OCG Releases Splunk for FIM

If you’re not familiar with Splunk (I wasn’t), it’s essentially an Enterprise monitoring and reporting tool that produces reports based on data generated by IT systems. According to the Splunk website, “it’s the easy, fast and resilient way to collect, analyze and secure the massive streams of machine data generated by all your IT systems and technology infrastructure.” If you can see how this might be used in a FIM context, then you’re not alone, because Oxford Computer Group have just released the Splunk for FIM App.

Announced on LinkedIn over the Easter Weekend by Sjef Van Leeuwen (Lead Software Developer at OCG), the Splunk for FIM app not only provides a dashboard for monitoring FIM operations, but also includes historic user and group data – allowing you to look at a group or user at a given point in time

I haven’t installed Splunk yet, but plan to do so in the coming weeks and will add my review to the appropriate page in my FIM Tools section once I’ve had a chance to take a real look at it.

Posted in FIM Resources, FIM Tools

More Redmond Identity 2014 Details

Oxford Computer Group sent out an e-mail this past Friday, informing alumni that their early-bird alumni special had been extended from March 31 until April 15. In addition, there was a bit more information in the e-mail about the conference:

The Summit is a multi-year journey to understand both emerging identity technologies and implementation best practices. For 2014, we’ll have a two-track conference for the technical and business minded people.

The technical track, “Window into Microsoft Identity & Access Engineering,” will present deep dives into Microsoft’s Identity stack and discuss new and upcoming product releases. We’ll answer the question “where is Microsoft Identity Management going?”

Using customer case studies, the business-focused track, “Identity Management Solutions,” will explore how scenarios and use cases are being addressed through technologies from Microsoft and their partner ecosystem. From on-premise to cloud. So, bring your manager.

Topics in both tracks will cover Azure AD, FIM 2010 R2, Role-Based Access Control (BHOLD Suite), AD FS, RMS and partner solutions.

We’re anticipating another roster full of excellent and respected speakers from Microsoft, partners, and customers, and we’re pulling together a panel of industry luminaries.

More information about the event can be found on the event website. I’ll definitely be going, so you should too. The Redmond Identity Access & Knowledge Summit 2014 will take place at Microsoft Headquarters in Seattle, from January 7-9 2014.

Posted in Redmond Identity, Redmond Identity 2014, Redmond Identity Access and Directory Knowledge Summit 2014

New FIM Tool: FIMDelta

While reinstalling FIM Portal in our development environment, one of my colleagues was working on some XSLTs to parse the changes.xml file produced during FIM Portal migration and produce a summary that would allow us to cull out any schema elements and policies that we didn’t want to reintroduce to our ‘clean’ development environment.

A few days into it, we saw a post by Carol Wapshere on her MissMIIS blog about a new FIM Tool that had been released by Alexey Skalozub, called FIMDelta.

FIMDelta basically allows you to do exactly what we were trying to achieve. It parses changes.xml, presents the changes in a summary and allows you to selectively include and exclude changes to create a new changes.xml.

Incredibly handy, we included a review in our FIM Tools section that details the application and several uses you could put it to.

Fortunately, we’ll still be able to use the XSLTs my colleague developed, as the output is something that is nice for documentation (something FIMDelta doesn’t provide).

Posted in FIM Resources, FIM Tools

I’m going to Redmond Identity, Access, and Directory Knowledge Summit 2014!

A few weeks back, I blogged that OCG had announced the dates for Redmond Identity, Access and Directory Summit 2014 as Jan7-9 again next year. Well, today I bought my ticket to go.

It looks like OCG have taken on board feedback that some of the attendees last year provided, as rumours tell that there may be two separate streams with next year’s conference – one, a technical side for those people who want all the raw, juicy detail about implementing FIM (complete with lots of tips and tricks, no doubt!); the second stream taking more of a business approach – how your organisation can use FIM; case studies to show real world scenarios and of course the strategic direction of Identity and the Microsoft FIM product suite. Yet to be confirmed, this is all rumour at this point!

Of course, I look at it from a strictly FIM point of view – RedmondIdentity2014 will also add Directory into the mix, so it will be interesting to see how big things become given the demise of TEC.

Posted in Redmond Identity 2014

FIM 2010 R2 SP1: What happened to [debug].[DeleteOrphanedRulesByType]?

After recently upgrading a development environment from FIM 2010 to FIM 2010 R2 SP1, I decided to clear all objects out of the FIM Portal to give myself a clean development environment.

So first, I ran various PowerShell scripts to clear out the users and groups (making sure not to delete my administrator users). Then, I did a Full Import on the FIM MA – at which point I realised I had a whole heap of orphaned EREs.

“Not to worry”, I said, “I’ll just go and run [debug].[DeleteOrphanedRulesByType]”. This handy SQL Stored Procedure appeared on the FIMService database in FIM 2010 build 4.0.3594.2:

 

Issue 4

Some ExpectedRuleEntry objects and DetectedRuleEntry objects in FIM 2010 can become “orphaned” over time. When aDetectedRuleEntry object is not referenced in the DetectedRulesList of any object in the system, that object is determined to be orphaned. Similarly, when an ExpectedRuleEntry object is not referenced in the ExpectedRulesList of any object in the system, that object is also determined to be orphaned.

These orphaned objects have no functional impact on FIM. However, over time, these orphaned objects can cause a decrease in performance for both FIM operations and Sync operations that are related to FIM, such as import or export by using the FIM MA.

A pruning stored procedure, [debug].[DeleteOrphanedRulesByType], was added to the [debug] namespace of the FimService database. This stored procedure must be run separately for the DetectedRuleEntry object and the ExpectedRuleEntry object. The stored procedure also has a “reportOnly” mode, and this mode can be used to determine the presence and number of orphaned DetectedRuleEntry and ExpectedRuleEntry objects in the system.

The @ruleType parameter expects one of the following well-known values:

  • N’Detected’ for DetectedRuleEntry objects
  • N’Expected’ for ExpectedRuleEntry objects

To determine the number of orphaned objects in the system, run the stored procedure in “reportOnly” mode as follows.

    DECLARE
    @deletedRulesFound BIT;
    EXEC [debug].[DeleteOrphanedRulesByType] @ruleType=N'CHANGE_ME', @reportOnly=1, @deletedRulesFound=@deletedRulesFound OUTPUT;

To loop through and actually delete orphaned objects in the system, run the stored procedure as follows. @deletionLimit=1000 instructs the procedure to stop when it has deleted 1,000 objects. If there are more than 1,000 orphaned objects in the system, either run the procedure multiple times (recommended) or increase the deletionLimit value.

    DECLARE 
       @deletedRulesFound  BIT,
       @startDateTime      DATETIME,
       @endDateTime        DATETIME;
    SELECT @deletedRulesFound = -1;         
    WHILE @deletedRulesFound <> 0
    BEGIN
        SELECT @startDateTime  = CURRENT_TIMESTAMP;
        EXEC [debug].[DeleteOrphanedRulesByType] @ruleType=N'CHANGE_ME', @deletionLimit=1000, @reportOnly=0, @deletedRulesFound=@deletedRulesFound OUTPUT;
        SELECT @endDateTime    = CURRENT_TIMESTAMP;
        SELECT @startDateTime AS [StartTime], @endDateTime AS [EndTime], @deletedRulesFound AS [WereDeletedRulesFound];
    END

 

But now it’s gone, along with some of the other stored procedures which it referenced. So, where did it go?

In the meantime, there are a few options available to me:

  1. Create a set called “Orphaned EREs” and attach a “Delete object” custom workflow activity to a transition-in MPR related to that set. This is a solution originally put forward on the FIM Technet Forums by Bob Bradley  (a colleague of mine at the time) back in 2010, prior to the release of FIM 2010 build 4.0.3594.2.
  2. Use the [debug].[PurgeObjectsOfType] stored procedure to delete all the EREs in the system. As this is a development environment, and I’ve already cleared out the objects, this is viable. I would not use this solution in a production environment.

So, away we went with option number 2… of course, now I’m getting “Stopped-Server” when I try to do a Full Import on the FIM MA from the FIM Sync Service. Note to self… never go against one of Bob’s suggestions! Next time I will definitely be going the Set/MPR/WF route.

Posted in FIM 2010 R2 SP1

FIM Hotfix Rollup 4.0.3684.2 Available

On March 19, 2013, Microsoft released FIM Hotfix Rollup 4.0.3684.2 for Microsoft Forefront Identity Manager (FIM) 2010. A minor hotfix, this patch has two specific features:

  • New version (4.0.2.0) of Microsoft.MetadirectoryServicesEx.dll. It is not clear what has changed in the new version of this dll, however it mentions you may need to fix references in your Rules Extension and ECMA projects.
  • Issue fix for FIM Synchronization Service, whereby Exchange configuration options weren’t previously available on the AD MA if FIM doesn’t detect Exchange on the domain

Paul Williams notes on his blog that the Exchange issue fix is the same issue resolved in FIM R2 build 4.1.3419.0, but applied to FIM 2010.

This hotfix will install on any version of FIM higher than 4.0.2592.0. Note, this is a FIM 2010 hotfix, not a FIM 2010 R2 hotfix.

For further information on the changes in this hotfix, refer to KB2819338, where you can also download the hotfix.

 

Posted in FIM Hotfixes, Service Packs & Updates

CustomizedObjects.aspx: Unable to process your request

Having recently performed a FIM 2010 R2 SP1 Upgrade, I was then required to apply some changes made in an earlier release using the FIM 2010 Migration scripts. In hindsight, I probably should have done this in reverse order (apply changes, then upgrade), as the changes.xml file contained a bunch of deletions related to specific R2 SP1 functionality. Anyway, I cleaned up the XML, ensuring it contained only changes relevant to the development work, but after deploying, I was receiving this error when trying to access  ~/identitymanagement/aspx/customized/CustomizedObjects.aspx?type=CustomType&display=CustomType from my FIM Navigation Bar:

Customized Objects Error

Interestingly, this error only seemed to present for one type of custom resource type – I had one which worked fine, and one which did not. After checking all the MPR permissions around my custom object type, I went into Administration->All Resources and tried to view the resources through there. Same error presented. Not surprising, since I think this page uses CustomizedObjects.aspx also, but this made it obvious that there was something else going on here. I could even create a test custom object and CustomizedObjects.aspx worked fine for it.

The surprising part, to me, was that nothing of use was showing in Event Viewer – just an error saying something was wrong – and even when I enabled verbose logging, nothing turned up. Finally, I went into Administration->Schema Management and compared my initial two custom resource types. I noticed pretty quickly that there was some vital  schema elements missing from my resource type that wasn’t working:

  • Created Time
  • Creator
  • Deleted Time
  • Description
  • Detected Rules
  • Display Name
  • Expected Rules
  • Expiration Time
  • Locale
  • MV Resource ID
  • Resource Time
  • Resource Type

Now, looking at the attributes missing, it’s pretty obvious why CustomizedObjects.aspx wasn’t working for my custom resource type – these are the base attributes used by the system! The only thing that I can imagine happened is that when I was modifying changes.xml for my schema, I must have removed any references to these attributes by accident… though looking back, I don’t recall modifying the schema changes much at all.

Regardless, once I added these attribute bindings back in through Schema Managemet, my CustomizedObjects.aspx started working again and my issue was resolved. This is a pretty obscure one, so I doubt anyone will encounter it in the future, but listing it here just in case.

Posted in FIM 2010 R2 SP1, FIM Troubleshooting

FIM R2 SP1: FIM Service and Portal Setup Wizard ended prematurely

Recently, I had to perform an upgrade to FIM 2010 R2 SP1 from FIM 2010 (pre-R2) in a development environment. After a few issues performing the upgrade, I was installing  the FIM 2010 R2 SP1 Service and Portal on SharePoint Foundation 2013 and I kept running into the error: Forefront Identity Manager Service and Portal Setup Wizard ended prematurely.

Forefront Identity Manager Service and Portal Setup Wizard ended prematurely

Background

Because I was upgrading directly from FIM 2010, I couldn’t use the R2 to R2 SP1 Update from KB2772429, which requires you to already be running FIM 2010 R2 build 4.1.2273.0 or later, so I had to perform an upgrade using the full installation media. My FIM Sync Service installation detected it was an upgrade and went through fine. Very pain free! However, when I installed the FIM Service and Portal, it didn’t seem to detect my existing installation. When I told it to proceed anyway, and told it to use my existing database, it didn’t install the SP1 Portal, it just uninstalled my existing Portal instead.

This was a bit weird, but since my Portal had already been uninstalled anyway, I decided I’d give it another try, but this time take it as an opportunity to upgrade to SharePoint Foundation 2013.

Installing FIM 2010 R2 SP1 on SharePoint Foundation 2013

I referred to the Microsoft documentation regarding Installing FIM 2010 R2 on SharePoint Foundation 2013 and found it to be pretty pitiful. In fact, there really doesn’t seem to be any benefit to using SPF2013 with FIM unless you’re running Windows Server 2012 (no SPF2010 support until SP2), and several reasons not to (I’ll get to that in a later post) but I was committed by this stage.

The main hurdle was that Classic Authentication has been deprecated in SPF 2013 and removed from the GUI options for creating a web application. Since FIM 2010 doesn’t support Claims-Based authentication, you have to create the web application using the SharePoint 2013 Management Shell. Of course, the “Standalone” installation method with SPF2013 pre-installs your application with Claims Based, so you first need to delete  that web application, then create another one via the Management Shell… anyway, all things for a later post.

FIM R2 SP1: FIM Service and Portal Setup Wizard ended prematurely

In true FIM fashion, this error is about as undescriptive as they come. So, in order to delve a little deeper, I executed the installation exceutable with verbose logging switched on:

msiexec /i “Service and Portal.msi /L*v “c:\temp\file.log

This time, when the installer ran, I was able to glean a bit more information:

MSI (s) (54:1C) [14:26:22:593]: Invoking remote custom action. DLL: C:\Windows\Installer\MSI2922.tmp, Entrypoint: AddServiceToPerformanceMonitors
SFXCA: Extracting custom action to temporary directory: C:\Windows\Installer\MSI2922.tmp-\
SFXCA: Binding to CLR version v2.0.50727
Calling custom action Microsoft.IdentityManagement.ServerCustomActions!Microsoft.IdentityManagement.ServerCustomActions.CustomActions.AddServiceToPerformanceMonitors
Adding FIMService account to 'Performance Monitor Users' group
Property name = 'ServiceAccount', value = 'ourDomain\FIMService'.
DomainName='ourDomain'
AccountName='FIMService'
Domain AD found
Exception thrown by custom action:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Runtime.InteropServices.COMException (0x800706BA): The RPC server is unavailable.

   at System.DirectoryServices.DirectoryEntries.Find(String name, String schemaClassName)
   at Microsoft.IdentityManagement.ServerCustomActions.CustomActions.ChangeUserMembershipInGroup(Session session, Boolean addUser)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object parameters, CultureInfo culture)
   at Microsoft.Deployment.WindowsInstaller.CustomActionProxy.InvokeCustomAction(Int32 sessionHandle, String entryPoint, IntPtr remotingDelegatePtr)
CustomAction AddServiceToPerformanceMonitors returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
Action ended 14:26:25: InstallExecute. Return value 3.

Note the three bolded bits: While trying to add the FIMService account to the Performance Monitors group, the ChangeUserMembershipInGroup method throws “The RCP Server is unavailable”. I put this information up on a post I started on Microsoft Technet and was also able to find another post by Robin Gaal over at Traxion who had been having a similar issue when upgrading to FIM 2010 R2, which I posted on to see if he could provide more detail.

Between the two posts, the three bits of advice that came back were:

  1. Ensure you’re using a domain user and a local admin
    Well, the account I was using was a domain user, a domain admin, a local admin and domain admins were also in the local admins group. I was also running the installer as administrator. I ruled this one out pretty quickly, but was good to do a sanity check.

  2. Check for AD Security customisations 
    This one came from Glenn Zuckerman at Microsoft in reply to my Technet post, who had debugged a similar issue and found that it was caused by some AD changes made in order to lock down security in the organisation. The result of these changes was that “the ‘Authenticated Users’ group had been removed from the Pre-Windows 2000 Compatible group” – so, another permissions issue.  I had a look at our AD, but couldn’t see anything obvious that would cause any issues with performing the necessary tasks – I mean, I was able to add the user manually to that group, using the same user account performing the installation.
  3. DNS Suffix Incorrectly Configured – SUCCESS!
    Robin Gaal had previously reported in his post that this was the issue behind their installation issue and that he was able to resolve it, so I asked him for some more detail. It turns out, his situation was very similar to mine – he was installing the FIM Service and Portal into a Test/Acceptance environment, where the default server configuration was registering the server to the  production domain. The solution was as easy as going into the advanced settings for the server’s network adapter, and changing the DNS settings:
    FIM 2010 R2 SP1 DNS Settings

Of course, when I tried this, the FIM Service and Portal installed perfectly first try, and I was even able to use and upgrade my existing database (which took almost no time at all to upgrade). Success!!

Overall, a very frustrating installation experience – I’d planned the upgrade to take two days, which I thought was pretty generous, and it ended up taking me about 7. The good news is, I learnt a lot about installing and configuring SharePoint Foundation 2013, as well as some new tricks for debugging FIM installations. Thanks to the folks on the Technet forums for their help, particularly Robin Gaal, Glenn Zuckerman and Varun Kohli.

FIM Service and Portal Installation Completed

Posted in FIM 2010 R2 SP1, FIM Hotfixes, Service Packs & Updates, Installing FIM, SharePoint Foundation 2013