Scaling to 10,000 unique permissions - Part 2 - The Solution

This follows on from my previous post; Part 1 - The Problem

The main requirement was:

• One SharePoint 2010 site
• 10,000+ uniquely permissioned objects each with a different user account

In this post we will be discussing the solution which involves programmatically creating unique permissions in a way which will scale for (what should be) well over 10,000 uniquely permissioned items...

Introducing yet another little known SharePoint API call ...

This is only possible because of one of the new SharePoint 2010 API calls;

SPRoleAssignmentCollection.AddToCurrentScopeOnly()

http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.sproleassignmentcollection.addtocurrentscopeonly.aspx

This basically adds the specified SPRoleAssignment but does not create any of the Limited Access scopes on the parent objects.

This is pretty straight forward and works in exactly the same way to normal Role Assignments in SharePoint 2010, we simply use the AddToCurrentScopeOnly() method instead of using the Add() method, for example:

 

   1: // fetch the Principal object which we are granting access to

   2: SPUser user = web.EnsureUser("Domain\\UserAccount");

   3:  

   4: // create a Role Assignment binding

   5: SPRoleAssignment roleAssignment = new SPRoleAssignment(user);

   6:  

   7: // apply contribute permissions

   8: roleAssignment.RoleDefinitionBindings.Add(

   9:     web.RoleDefinitions["Contribute"]);

  10:  

  11: // grant permissions to the list item using the CURRENT SCOPE ONLY

  12: // this ensures that Limited Access scopes are NOT created

  13: // for parent objects (we're going to have to do that bit ourselves!)

  14: item.RoleAssignments.AddToCurrentScopeOnly(roleAssignment)

 

It is very important to understand that you still need to grant "Limited Access" (it wasn't put in just for laughs, it does have a purpose). Granting "Limited Access" means that the object has access to core information on parent objects to enable construction of things like the breadcrumb, and retrieval of core files needed to render the interface.

This then means it is up to us (the developers) to go back and create each of those in a more efficient way. The problem is .. you can't assign "Limited Access" programmatically...

What do you mean .. I can't assign Limited Access??

Well, I don't really know why they did this, but if you try and assign it programmatically (Limited Access is actually a "Permission Level" in SharePoint) you will get errors (admittedly you can't do this through the user interface either!).

So, the workaround (again) is to create your own permission level which includes exactly the same permissions that "Limited Access" would have granted. This is:

• View Application Pages
  
• Browse User Information
  
• Use Remote Interfaces
  
• Use Client Integration Features
  
• Open

You can call this anything you like (I called mine "SP Limited Access") as long as you know what it means.

The code to do this is as follows:

 

   1: internal SPRoleDefinition GetLimitedAccessRole(SPWeb web)

   2:         {

   3:             string strRoleDefinition = "SP Limited Access";

   4:  

   5:             // only exists in webs with unique role definitions

   6:             if (web.HasUniqueRoleDefinitions)

   7:             {

   8:                 try

   9:                 {

  10:                     // try to retrieve the role definition

  11:                     return web.RoleDefinitions[strRoleDefinition];

  12:                 }

  13:                 catch (SPException)

  14:                 {

  15:                     // SPException means it does not exist

  16:  

  17:                     // create our custom limited access role

  18:                     SPRoleDefinition roleDef = new SPRoleDefinition();

  19:  

  20:                     // give it a name and description

  21:                     roleDef.Name = "SP Limited Access";

  22:                     roleDef.Description = "Identical to standard " + 

  23:                         "Limited Access rights. " + 

  24:                         "Used to provide access to parent objects of " + 

  25:                         "uniquely permissioned content";

  26:  

  27:                     // apply the base permissions required

  28:                     roleDef.BasePermissions = SPBasePermissions.ViewFormPages 

  29:                         | SPBasePermissions.Open 

  30:                         | SPBasePermissions.BrowseUserInfo 

  31:                         | SPBasePermissions.UseClientIntegration 

  32:                         | SPBasePermissions.UseRemoteAPIs;

  33:  

  34:                     // add it to the web

  35:                     web.RoleDefinitions.Add(roleDef);

  36:                 }

  37:  

  38:                 return web.RoleDefinitions[strRoleDefinition];

  39:             }

  40:             else

  41:             {

  42:                 // try the parent web

  43:                 return GetLimitedAccessRole(web.ParentWeb);

  44:             }

  45:         }

I've created my new Limited Access Permission Level .. now what?

One thing does need to be made clear, there is absolutely no point you just creating all of the Security Scopes that SharePoint would have created (you'll end up with the same mess we were trying to avoid in the first place).

The solution is to create a group for all of the "Limited Access" users for that List or Web. It really is up to you whether you use Active Directory Security Groups or SharePoint Groups. I decided to use AD security groups; mainly because I didn't want to clog up the Site Collection "groups" functionality, and didn't want idiot Site Collection admins from removing the group members (or worse .. the groups themselves!) and breaking the site collection.

Note - I haven't included the code to create and modify Active Directory Security Groups here, if nothing else because there are thousands of resources out there showing you how to modify AD groups programmatically, and Code Project has a particularly good reference: Howto: (Almost) Everything In Active Directory via C#

You will need to create a group for each parent object which has unique permissions although in my example it is only really the SPWeb (web site) that we are worried about as the libraries and folders are well within the security scope threshold.

So we have our 20 libraries and our root web site. So in our example we would have to create 21 different AD security groups:

• One group to store all Limited Access users for the root web site


• 20 groups to store all Limited Access for the libraries (one for each library)

Then, following this example you can then use the following code to grant “Limited Access” to one of the libraries (and just rinse and repeat for the other libraries and the root web site);

 

   1: // fetch the "SP Limited Access" role definition

   2: SPRoleDefinition limitedAccessRole = GetLimitedAccessRole(web);

   3:  

   4: // get SPPrincipal object for the AD Group we created

   5: SPUser adGroup = web.EnsureUser("My Custom AD Group Name");

   6:  

   7: // set the role assignments for this group

   8: SPRoleAssignment roleAssignment = new SPRoleAssignment(adGroup);

   9: roleAssignment.RoleDefinitionBindings.Add(limitedAccessRole);

  10:  

  11: // grant "Limited Access" to the AD Group for this list

  12: // we only have to do this once! After this we simply 

  13: // need to add members to this AD Group every time we 

  14: // add users to one of the parent objects!

  15: list.RoleAssignments.AddToCurrentScopeOnly(roleAssignment)

So having done this for all of the parent objects we now have our 21 custom Active Directory groups, each one of which has been granted “Limited Access” to one of the required “parent” objects for our folders.

From here on in it should be smooth sailing. You simply need to make sure that every time you programmatically add a new user to one of the folders you also make sure they get added to the relevant AD Groups (so that the “Limited Access” chain is not broken).

The following diagram really explains what we have done:

I have tested this model for over 16,000 unique AD accounts across hundreds of folders in hundreds of document libraries and I cannot notice any discernable drop off in performance (nothing that can’t be explained by simply having a really large number of libraries and folders anyway!) so initial tests show that this is working very well indeed :)

What I also ended up doing (to make this slightly more robust) is to build my own application page which users can use to Grant Permissions through the UI (so we don’t need to write custom code every time a new “Limited Access” scope is needed).

I then wrote an HttpModule to auto-redirect any requests to the out-of-the-box page (_layouts/AclInv.aspx) to the custom page so that if anyone tried to use the native user interface it would ALWAYS be executing my own custom code (which creates all of the AD Groups and SP Limited Access scopes programmatically, without the user having to worry about it!)

The great thing about this solution is that it doesn't matter how many users or groups you are adding to your SharePoint site .. you only ever have 1 Limited Access security scope for each List / Web!

Thanks for sticking with me through these two posts .. if you made it this far then thanks for reading and I would love to hear your comments! :)

Scaling to 10,000 unique permissions - Part 1 - The Problem

This post was borne out of a client requirement which popped up on my radar. I'm currently working for a leading global Business Intelligence provider in London, and they were looking to implement a particular third party piece of software. This software relies on SharePoint for file storage and my client wanted to roll this out to their customers "extranet" style with each customer having uniquely secured content (files and folders).

Now .. first off their customers include in excess of 10,000 different companies (i.e. > 10,000 users) so early warning bells immediately started ringing in terms of scalability.

Secondly, to make this worse, the software required all content to be stored in a single SharePoint site .. so now my early warning system had gone into full meltdown and a state of high alert was reached.
So to boil this down ...

• One SharePoint 2010 site
• 10,000+ uniquely permissioned objects each with a different user account

A Library with 10,000 uniquely permissioned folders?? Possible? My first instincts said no... so it was time to get my problem solving hat on and do some digging ..

Investigating the Limits of SharePoint 2010

I would like to think that any SharePoint {Consultant | Developer | Architect | <insert profession>} worth their salt would have read the Software and Capacity Planning guidelines (or at least be aware of it!) .. so that was my first pit-stop.

Note - I also stumbled across a great blog post by SharePoint infrastructure veteran Joel Oleson and his Best Practices for Enterprise User Scalability in SharePoint. This goes into detail about the specific size of an ACL (and the reason why this is limited, specifically in Windows) which although a good read wasn't really relevant to my problem.

The Microsoft TechNet article SharePoint Server 2010 capacity management: Software boundaries and limits (http://technet.microsoft.com/en-us/library/cc262787.aspx) is a great resource and contains one absolutely key entry:

Security Scope - 1,000 per list (threshold)
The maximum number of unique security scopes set for a list should not exceed 1,000. 

A scope is the security boundary for a securable object and any of its children that do not have a separate security boundary defined.  

A scope contains an Access Control List (ACL), but unlike NTFS ACLs, a scope can include security principals that are specific to SharePoint Server. The members of an ACL for a scope can include Windows users, user accounts other than Windows users (such as forms-based accounts), Active Directory groups, or SharePoint groups.

So what is a Security Scope then? Ok I admit it does tend to get a bit bogged down in terminology.
To put it simply ... each time you grant access to a new principal (user account or group) then you are creating a new Security Scope.

The other thing to consider pickup is that this is not just limited to lists! Any list that inherits permissions will pick up their permissions from the parent web (site) so you also need to adhere to this at the web level too!

This means that you should not have more than 1000 security scopes at EITHER the Site or List level.

Ignoring this limit can do real damage to your farm ...

There is even a Microsoft Knowledgebase article explaining why; SharePoint performance degradation with a large number of unique security scopes in lists (http://support.microsoft.com/kb/2420771)

This is really explained in far more detail in two most excellent blog posts:

The first post describes the problem of trying to create more than 1000 security scopes, and what happens when you do this: http://wbblog.datapolis.com/2011/03/setting-item-permissions-with-workflow.html

The second post is by James Love (a.k.a. @jimmywim) and goes into real "deep dive" detail looking into the root cause of the problem (SQL Server and ACL GUIDs) and how this problem can actually bring down your ENTIRE FARM and not just the list / site you are working on!
http://e-junkie-chronicles.blogspot.com/2011/03/sharepoint-2010-performance-with-item_23.html

A quote from the second post is as follows:

"When you load up a huge list with lots of item level permissions, a single operation gets every single GUID associated with the ACL for that item and passes that back to the data access layer of SharePoint. When the database retrieves the actual list item data, it will pass in all of the ACL Guids back in as one long string, all concatenated together. The query to get the data creates a table variable re-assembles the the item level ACL Guid associated with each item. How the rest of the query deals with this is anyone's guess at the moment - this table variable might just be passed back to the calling COM object (though I thought they couldn't be used this way....) for the COM object to then sort out which item should be visible to which "scope" (or ACL).

So, what can we take away form this? Passing 640k of data about the place, for a SQL Query to do some substring math and converting to Guids will soon bring your database server to its knees. This is one request and it takes 2000ms to work. Imagine if you have 5 requests per second or more hitting this list!"

Both are excellent appendums to this post and well worth looking at for another angle and a bit more detail!

Why does this become my problem?

Now .. looking back to my original problem some of you may be thinking, OK no problem; you can just create yourself 20 different lists / libraries .. and have 500 unique permissions in each list??

Well .. so you might think .. and here I introduce the juggernaut that is Limited Access Scopes!

Anyone who has spent any time around SharePoint security will have noticed the odd "Limited Access" permission popping up in their site from time to time. "Limited Access" is automatically allocated to a parent Folder, List or Web whenever a child object has a unique permission allocated to it.

You can easily see these being created if you break permission inheritence to a list and just add a few accounts to that list. The parent Web will not have a "Limited Access" scope created for each user account you have added.

Now hopefully the bright will already have spotted the problem .. it doesn't matter how many lists or libraries you create .. every single user or group that you add will end up in the parent Web site with "Limited Access" (and every single Parent Web heading upwards).

The following diagram explains why.

You simply cannot get away from this fact. If are adding 10,000 unique permissions with different user accounts then you will end up with 10,000 security scopes at the root web!

Note - It should be noted that the number of "Limited Access" scopes created is limited to the number of Security Principals you are adding.

If you are adding from a pool of 50 users then you will only ever be adding a maximum of 50 new "Limited Access" scopes (one for each user account).

For this reason it is a good idea to use Groups when adding permissions as this limits the number of "Limited Access" scopes which are created .. but this won't solve your problem if you have over 1000 different security principals!

So that was the crux of my problem .. on investigation this does look to be a major major problem (and an "impossible fix") but it would seem not! There IS a workaround (one which I have tested to over 15,000 unique user accounts and works very very well indeed)...

The solution, workaround, and code samples are all included in Part 2 ...