Code Solution - Import AD Photos into SharePoint User Profiles

[Update: Code download files updated 18/03/2011 - see below]

This is in relation to a previous post I made last week;

Active Directory Images are not imported by the SP2010 User Profile Import

So, the source code is finally ready and uploaded for your enjoyment :)

Now - I must first off give credit to the sources of inspiration. A lot of the code in this solution is copied / borrowed / inspired by the following posts:

• MSDN Blog - Background information on why you cannot import the property from AD directly
• Blog Post - How to upload a user profile photo programmatically
• Technet Blog - Programmatically using the jpegPhoto attribute in Active Directory

All I have done is brought their code and samples together and packaged it into a WSP that runs from SharePoint Timer jobs, so you have the convenience of a SharePoint 2010 farm solution :)

Also it should be understood that both AD and the User Profile database are quite critical parts of anyone's SharePoint farm, so sorry, but first I need to make a ....

... Disclaimer - All code and solutions are provided "as is" and should be used at your own risk! It is highly recommended that you test these in an isolated environment, and I confer no responsibility for any loss or damage from using the code, advice or solutions provided on this blog, or any related content.

Ok, now that is out of the way we can get on with business :)

[Update - 18/03/2011 - I have updated both sets of files so that it now uses the "distinguishedName" attribute to identify users in AD .. as this is a more reliable method and was in response to a reported bug]

I have uploaded the files to my Sky Drive including:

• Full Visual Studio 2010 Solution
• Installable WSP Package

When you roll out the WSP you will find that it includes the following functionality:

Farm Scoped Feature
The WSP package includes a farm scoped feature called:

Hatch Solutions - Import Photos from AD (Timer Job)

When activated this will automatically identify the default MySite host application and create a custom Timer Job (see below) attached to that web application.

My Site Timer Job
The Timer Job (installed by the Farm Feature) is designed to run on the My Site Host web application, and is pre-configured to run once-per hour. It is called:

Hatch Solutions - Import Photos from AD

This will do the following:

• Automatically identify all AD accounts in the current User Profile Database
• If the AD account has a "jpegPhoto" attribute, then this is extracted
• The photo is converted to three thumbnail images, and uploaded to the My Site Host profile photo asset library
• The photo for that user profile is updated to point at their newly uploaded photo

Hope you enjoy, the source code is there for all to see, and good luck!

Tip - CAML Query Retrieve tasks assigned to user (including both AD and SP Groups)

This is something I have seen so many people struggle with, but it really is very easy, with the help of the "<Membership>" element.

The Membership element allows you to basically check to see if the AssignedTo field is assigned to any group which the current user is a member of.

Of course, you still need to use it in conjunction with a standard FieldRef check against the user's ID (which you can get using the <UserID/> element.

Below is the CAML query to return All Tasks Assigned to the Current User, including specific assignments, and where the task is assigned to a group that contains the current user (both AD Groups and SharePoint Groups).

I suppose theoretically this should also work with groups in custom Membership Providers .. but haven't tried it.

<Where>
  <Or>
    <Eq>
      <FieldRef ID='Assigned To' />
      <Value Type='Integer'><UserID/></Value>
    </Eq>
    <Membership Type='CurrentUserGroups'>
        <FieldRef Name='AssignedTo' />
    </Membership>";
  </Or>
</Where>

Active Directory Images are not imported by the SP2010 User Profile Import

This is something that I found when starting a new SharePoint 2010 (SP2010) Intranet project. You see, my client has quite a simple requirement, at least it seems that way at first.

They have an HR database which they use to store everything from employee details to security card information. For this reason they manage all of the employee photos here, and they are pushed into Active Directory (using the LDAP "jpegPhoto" attribute) to make them available in applications like Outlook.

So ... to put it quite simply .. they don't want to use SharePoint to upload their photos. In fact, they already have their photos in AD .. they just want to pull them into SharePoint 2010.

Can't we just map the Profile Property to AD and Import the value?
So here is where we hit the roadblock.. The Active Directory "jpegPhoto" attribute is of type "Binary Data" and the SharePoint 2010 User Profile property is of type HyperLink (as it typically links to an image in an Asset Library in the My Site Host site collection).

As a result, you cannot import it using SharePoint 2010 functionality (although it may be possible if you have also purchased the more sophisticated "ForeFront Identity Manager Synchronisation Server" product).

Options ??
Well .. SharePoint IS used as a development platform so there are some options. Obviously there is a full API for reading and writing in and out of the User Profile database (some more accessible than others).

There are a few good blog articles that you can follow if you want to build your own import function. If you are happy to wait a while longer then I already have my own solution which I will be posting up with the following features:

UPDATE - Full Source code and WSP now published

* WSP Package
* Farm scoped feature, which installs a Timer Job attached to the MySite web application
* Iterates through all User Profiles, and finds and extracts Binary image data from Active Directory
* Automatically creates thumbnailed images in the My Site Host Asset Library
* Automatically updates User Profiles to point to those new images

During my research Glyn Clough also pointed me in the direction of another solution which allows you to do the same using a Console Application... personally I prefer the more "SharePoint" route of Timer Job and WSP package :)

In the meantime, you can feel free to avail yourselves of these posts:

• MSDN Blog - Background information on why you cannot import the property from AD directly
• Blog Post - How to upload a user profile photo programmatically
• Technet Blog - Programmatically using the jpegPhoto attribute in Active Directory

SUGUK London, 14th October Sandbox Development and jQuery

Well it was a great night last night. We had a really good turn out (around the 90 mark I think in the end) and it was a cracking dev-focused session.

Firstly, hats off to Jaap Vossers (@jvossers / blog.vossers.com) who stepped up for his first community event and gave a really good presentation on jQuery and SharePoint.

My personal favourites were the Developer Dashboard Visualiser (which created a graphical time-line of how long each page component took to load, which you could drill-into) and the Site Settings snap-in (which allowed you to drop in the "Site Settings" page for the current site by pressing Ctrl-S from any page!). Lots of content in there, including using Web Services and utilising some of the Client Object Model.

Finally Jaap ran a prize draw giving away 5 free copies of a jQuery development book! Good stuff!

My turn came next, presenting on SharePoint 2010 Sandbox Development. For those of you who follow me on Twitter (@MartinHatch) you will know that I accidentally deleted my virtual machines last week so had to rebuild my farm and re-code all my examples.

Well .. not one to try making things easier for myself I decided to code all of them from scratch (Sandbox Web Part, Full Trust Proxy and a Solution Validator).

The only one which didn't work was the Full Trust Proxy (I then found out I had a note in my slide deck on the "demo" slide which read - "make sure you re-start the Sandbox Service after you install the full-trust proxy" .. I have updated my slide deck now to include that VERY important line!) but overall I think it went very well and muchos thanks to all the feedback on Twitter (make sure you use the #SUGUK hashtag!)

My own slide decks and code samples can all be found on my Sky Drive account here:
http://cid-60f12a60288e5607.skydrive.live.com/redir.aspx?resid=60F12A60288E5607!621

Finally we had an impromtu "Ask the Experts Session" .. with the awesome team of Eric Shupps, Chris O'Brien, Andrew Woodward, Steve Smith, Jaap Vossers and myself!

The evening ended up with a SharePint (although I had to head off to get the last train out into the sussex wilderness I call home!).

If you were there .. hope you enjoyed it (all feedback welcome!). If not ... I hope to see you at the next SUGUK meeting! :)

Presenting at SUGUK London - October 14th

Yep, the next SharePoint User Group UK (SUGUK) London session has been announced, and I'm going to be presenting on Sandbox Development. (yay!)

The event is on October 14th 2010 at at Mostyn Hotel (Marble Arch, Bryanston Street, London, W1H 7BY) and the evening starts at 18:00.

I have the second slot and will be talking about Sandbox Development, a subject which I've been keenly interested in what with Content and Code's products and the expectation of SharePoint 2010 suposedly coming to BPOS (Microsoft's cloud services).

"This development focused session will walk through the new SharePoint 2010 Sandbox Solutions framework, including the architecture, configuration and development of Sandboxed solutions. Although covering off a few administrative functions do not be fooled .. we will be opening Visual Studio 2010, stepping through code and debugging processes!"

The session kicks off with Jaap Vossers talking about SharePoint and JQuery. I've been working with Jaap for several months now and some of the JQuery stuff he's done is fantastic, so this is well worth coming along for that session alone!

"This session will cover everything you need to know about harnessing the power of jQuery in your SharePoint sites. The introduction will look at getting started, the syntax and the plugins before we look at jQuery in a SharePoint context; the benefits, examples, calling web services, loading scripts and deployment. We will then cover the various frameworks and utilities (SPServices, jQueryLoader) before rounding off with a look at further integration opportunities in SP2010 and specifically the Client OM and REST. This is a technical session."

If you want to register then please sign up at the SUGUK Forum Post (http://suguk.org/forums/thread/25091.aspx).

I'm really looking forward to it, so please come along, and hopefully I'll see you there!

Forays into SharePoint 2010 Performance Testing with Visual Studio 2010

Over the past six months I have increasingly become an evangelist of Performance Testing. It has always previously been an area that I was aware of but I never really got massively involved in, but recently I've seen it as an increasingly important part of my work, especially on the larger scale projects with load balanced web front ends (for performance, not just redundancy) and you start hitting I/O limits on SQL. I suppose this may have been triggered by the SharePoint Conference 2009, and one of my follow up blog posts "Load Testing SharePoint 2010 with Visual Studio Team Test".

So in this post I firstly wanted to look at why you should do Performance Testing?

It sounds like a bit of a stupid question (with an obvious answer) but it really is surprising how many people don't do it. How many of you have ever asked the following questions on a project?

"How many users can the production system support?"
"What would be the impact of doubling the number of users?"
"What impact with backups have on performance?"
"How fast will the solution perform during peak hours?"
"What is the most cost-effective way of improving performance?"

All of these are questions that you absolutely HAVE to be able to answer. The client (whether it is your organisation, or another organisation who you are running a project for) deserves to know the answers to these, and without them how can you have any idea whether your solution is going to be fit for purpose?

Sure, you can read up on Estimating Performance and Capacity Planning in SharePoint, but all that gives you is some rough guidelines.. we need to be able to apply some science to the process!

The last question is probable the most compelling. Re-configuring farms and buying new hardware is an expensive process, the consultancy alone can cost thousands of pounds, and you don't want to have your client coming back asking why they just spent tens of thousands of pounds on a new state of the art iSCSI SAN array, to have zero impact on performance ("hey .. we thought it would help .. but we didn't really know!") because the bottleneck was actually the CPU on the Web Front End (WFE).

The story often gets even worse when things do start going wrong. If you have ever been in the unfortunate position where you are troubleshooting a system that is performing badly, these kinds of questions are quite common:

"What is causing the poor performance?"
"How can we fix this?"
"Why did you not notice this during development?"

Again, the last two questions is the killer.. if you don't do any Performance Testing then you won't know that you have a problem until it is too late. The earlier you can get some metrics on this, the faster you will be able to react to performance issues (in some cases finding them and fixing them before the client even knows about it!)

Equally, without performance testing you won't know WHY the problems are occuring. If you don't know why then you can't know HOW the best way is to fix them!

So the key messages are this:

• Early Warning .. catch problems early on and they will be easier to fix. There is no point waiting until users are hitting the system to find out the solution can't cope with the load!
• Knowledge ... what is causing the problems, and how do you fix them?
• Confidence ... not just that you know what you are doing, but you can prove it. This instils confidence in your sales, confidence in your delivery, and confidence from your clients too!

Performance Testing with Visual Studio 2010
I've been using Visual Studio 2010 Ultimate edition. It is the only "2010" product that incorporates Web Performance Tests and Load Tests, the two critical pieces that you will use to test the performance on SharePoint 2010 (or any other web based system). It also integrates tightly with Team Foundation Server and provides "Lab Management" capability, but that is out of the scope of this blog post.

In order to do comprehensive testing you really need 4 different software packages:

1. Visual Studio 2010 Ultimate: This is where you create your tests and control the execution of them.
2. Visual Studio 2010 Test Controller: Part of the Visual Studio Agents 2010 ISO, this allows you to co-ordinate tests executed by several "agents", as well as collecting results and storing all of the test results (and performance counters) in a database. The license for this is included in Visual Studio 2010 Ultimate.
3. Visual Studio 2010 Test Agent: Part of the Visual Studio Agents 2010 ISO, this can be installed on machines that will simulate load and execute tests. They are connected to a "Controller" which gives them instructions. The license for this is included in Visual Studio 2010 Ultimate.
4. Visual Studio 2010 Virtual User Pack: This is a license that allows you to increase the number of virtual "users" you can simulate by 1,000 (for each pack that you purchase). This is a separate license that must be purchased separately (there is no trial version!)

If you need any help installing these and getting them running then there is a great MSDN article which you should read: Installing and Configuring Visual Studio Agents and Test and Build Controllers or the equally awesome article from Visual Studio Magazine: Load Testing with Visual Studio 2010.

So what about actually creating the tests?

Well, the interface is pretty simple. You can create your "Web Performance Tests" using a simple Browser Recorder (literally using a Web Browser which records all of your actions, and then click "stop" when you are finished). This works great, but there are a few caveats:

• You might want to use the "Generate Code" option if you are adding documents or list items. This converts your recorded web test into a code file, allowing you to programmatically change document names, or field values .. useful to make sure you are not just overwriting the same document over and over again
• Web Service tests require a bit more "knowledge" of how they work, needing the SOAP envelope (in XML) and the SOAPAction header.

It is worth noting that there is an excellent Code Plex project available: "SharePoint Performance Tests". Although this was written for Visual Studio 2008 (you can convert it to 2010 if you want) it contains a number of configurable tests (via XML) that allow you to dynamically create tests for generic SharePoint platforms .. well worth a look!

You can then very easily create a "Load Test" which allows you to pick'n'mix tests, and a distribution of which tests you want to run.

My personal favourite is the "Tests Per User Per Hour". For this you would sit down with your client and work out "what would a typical user do in an hour of using the system.." one such activity resulted in this kind of activity distribution:

• Hit the site home page 50 times
• Execute 10 searches
• Upload 5 documents
• Respond to 20 workflow tasks

This kind of valuable information allows you to build your tests and then distribute them using the Load Test. All you do then is plug in how many users you want to simulate and away you go!

Counting the Counters?
All of this so far is great stuff, but without the performance counters you really aren't going to get much legs from Visual Studio. You might get the WHAT is going on (i.e. do the tests complete very quickly?) but you certainly won't get the WHY information which is oh-so important (i.e. is it the CPU, RAM or Disk?)

For this you need to add Performance Counters... thankfully this is rediculously simple. You have something called "Counter Sets" which you can configure to collect from the computers that operate in your farm.
There are a bunch of pre-defined counter-sets you can choose from:

• Application
• ASP.Net (I pick this for my WFE Servers)
• .Net Application (I pick this for my Application Servers)
• IIS
• SQL (I pick this for my SQL Servers)

I won't go into any more detail than that. A step-by-step walkthrough of the options (including screenshots) can be found at the Load Testing with Visual Studio 2010 article at Visual Studio Magazine.

What about the Results?
Well, there isn't a really simple answer to this. You really need to have a good understanding on how the different hardware components interact, and what limits you should be looking for.

The big hardware counters (CPU usage, Available Memory) are the obvious ones. Any server which exceeds 80% CPU usage for any sustained period is going to be in trouble and is close to a bottleneck. Equally any server which starts to run out of memory (or more importantly .. slowly loses memory, suggesting a memory leak!) should be identified.

But it's the deeper, more granular analysis that proves most useful. On a recent client project I was looking at a Proof of Concept environment. We knew that we had a bottleneck in our WFE (CPU was averaging around 90%) and it was extremely workflow heavy, but the page performance was far too bad to put down to just the CPU.

On closer inspection we found a direct correlation between ther Page Response Time and the Disk Queue Length in SQL Server:

The top-left corner is the Disk Queue Length in SQL Server, and the Top Right is the Page Response Time for the Document Upload operation (bottom right is the overall Test Response time), clearly the spikes happened at the same time.

This is the true power of using Visual Studio. All of the tests and performance counters are time-stamped, allowing you to drill into any specific instance and see exactly what was happening at that moment in time!

Looking closer at the SQL Disk usage, the Write Time (%) and Read Time (%) show us even more interesting results:

The top of the graph shows the Disk Write Usage (%) and the bottom half shows the Disk Read Usage (%). Clearly, the disk is very busy writing (often being at 100%) while it does very little reading. This fits perfectly with our test results as most of the "read" operations (like viewing the home page, or executing a search result) were extremely fast ... but most of the "write" operations (like uploading a document) were much slower.

So the WHAT is slow write performance (uploading of documents).
The WHY is now very simple, the disks on the SQL Server need looking at (possibly upgrading to faster disks, or some optimisation in the configuration of the databases).

Conclusion
To be honest I could talk about this subject all day, but hopefully this gives you some indication of just how crucial Performance Testing is .. and how powerful Visual Studio can be as a testing tool.

The ease of creating test scripts, the vast flexibility and power of the enormous performance counters available, and the ability to drill into a single second of activity and see (simultaneously) what was going on in all of the other servers .. its an awesome combination.

I'll probably be posting more blog posts on this in the future, but for now good luck, and hope you get as much of a kick out of VS2010 as I have :)

HTC Twitter "Peep" and OAuth

[UPDATE - 27/09/2010]
HTC Peep on my HTC HD2 is now working!
I found that I needed to login using my email address instead of my username, but it is now working again!
(hurrah)
[/UPDATE]

If you are using a Windows Mobile device from HTC (like me .. I have an HTC) then you've probably run into the same Twitter issue that I have.

Twitter recently shutdown their Basic Authentication method for Twitter and this hosed a whole range of Twitter applications which were not appropriately using the "OAuth" method that Twitter preferred.

One of those applications is the HTC "Peep" application. Now, I'm a quite avid consumer of Twitter, and although Twitter did recently release an announcement that it was working  it seems that HTC Windows Mobile clients (such as the HD2) are still not working.

Well, I submitted a question to HTC Support and they very kindly sent me a response back (in under 1 hour, very impressive). Their email response was as follows:

"We are currently investigating an issue with our Peep/Twitter/Friend Stream client that has stopped working and hope to have a solution soon. Please monitor the support pages for updates, or if you prefer we can record your details and contact you again once a solution is available"

So hopefully it will all be working again soon. Fingers crossed ... It's not exactly a mission-critical application for me, but I do hate it when things just "stop working" that I was a user of.

How to: Achieve Count(*) on a large SharePoint list

This has been a mission of mine for a while now (before I went on holiday and took a 2 week hiatus from all things SharePoint :)).

One of the clients I've been working with has been trying to replicate a pretty simple operation (by normal development standards). They have a SharePoint list with a LOT of items in it (we are talking 200,000 list items and above) and includes some Choice fields.

They want to return a count of how often each choice value is being used. Now, if you were using SQL Server you would simply do the following pseudo-SQL:

select count(*) from myList group by myChoiceField

At first look in SharePoint this is not possible:

• There is no "count" operation in CAML, nor any other kind of aggregation function
• SharePoint Search "full text query" does not support the count(*) operator (or anything similar)
• The only reference to aggregations is in the SPView.Aggregations property .. this is only used by the rendered HTML and the values are not returned in the result set.

Now .. I know that you can get count values on a list, if you create a View with a Group By then it shows you the number of items in each group, so it MUST be possible! So my mission started

List view with groups

We want to replicate this behaviour,

but programmatically!

First.. we need a test environment
The first thing I did was create a really big list. We are talking about 200,000 list items, so you can't just pull all the items out in an SPQuery (as it would be far too slow!).

I generated a simple custom list. I add a choice field (with optional values of 1-20) and then generated 200,000 list items with a randomly assigned choice value (and a bunch of them without any choice value at all .. just for laughs).

Now I could play with my code

Attempt Number 1 - Retrieve all list and programmatically calculate the counts (fail)
I kinda knew this wouldn't work .. but I needed a sounding board to know HOW bad it really was. There are 200,000 items after all, so this was never going to be fast.

• Use SPQuery to retrieve 2 fields (the ID, and my "choice" field).
• Retrieve the result set, and iterate through them, incremementing an integer value to get each "group" count value

This was a definite #fail.To retrieve all 200,000 list items in a single SPQuery took about 25 seconds to execute ... FAR too slow.

Attempt Number 2 - Execute separate query for each "group" (fail)
I was a little more positive with this one ... smaller queries execute much faster so this had some legs (and this is certainly a viable option if you only want the count for a SINGLE group).

• Create an SPQuery for each of the "choice" values we want to group by (there are 20 of them!)
• Execute each query, and use SPListItemCollection.Count to get the value

Unfortunately this was another spectacular #fail. Each query executed in around 2 seconds .. which would be fine if we didn't have to do it 20 times! :( (i.e. 40 second page load!!)

Attempt Number 3 - Use the SPView object (success!)
Ok .. so I know that the SPView can render extremely fast. With my sample list, and creating a streamlined "group by" view it was rendering in about 2 seconds (and thats on my laptop VM! I'm sure a production box would be much much quicker).

The main problem is ... how do you get these values programmatically?

The SPView class contains a "RenderAsHtml" method which returns the full HTML output of the List View (including all of the group values, javascript functions, the lot). My main question was how did it actually work? (and how on earth did it get those values so quickly!)

I started off poking into the SPView object using Reflector (tsk tsk). The chain I ended up following was this:

• SPView.RenderAsHtml() -->


• SPList.RenderAsHtml() (obfuscated ... arghhhh)

So that was a dead end .. I did some more poking around and found out that SPContext also has a view render method ...

• SPContext.RenderViewAsHtml() -->


• SPContextInternalClass.RenderViewAsHtml() -->


• COM object ! (arghhhh)

Now .. the fact that we just hit a COM object suggests that we are starting to wander towards the SQL queries that get executed to retrieve the view data .. I didn't want to go anywhere NEAR that one, so I decided to leave it there and perhaps try using the output HTML instead (nasty .. but not much of a choice left!).

using (SPSite site = new SPSite(http://myspsite))
{
SPList list = site.RootWeb.Lists["TestList"];
string strViewHtml = list.Views["GroupedView"].RenderAsHtml();
}

Having done this we now have the HTML output of our view (and this code takes about 2-3 seconds to execute ... fast enough for my laptop .. we can always cache the value if needed).
 
Looking through the DOM output in the browser, it was possible to identify the "group" element by their attributes. It is a TBody node with both an ID attribute and a "groupString" attribute (the GroupString is the important one, as it tells us the view is configured to "Group By").
 
What I needed next was a way of getting the actual values out of the HTML. For this I used the extremely awesome "HTML Agility Pack" from Codeplex. This is a set of libraries that allow you to parse DOM elements, including both plain "poorly formed" HTML as well as XHTML, and then use XPath queries to extract any values you want (much in the same way you would normally use the XML namespace for XHTML).
 
This gave me the TBODY node, and from there I could use string manipulation on the "InnerText" to pull out the group name and the count value :)

// Using HTML Agility Pack - Codeplex
// load the HTML into the HtmlDocument object

HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(strViewHtml);

// retrieve all TBODY elements which have both
// an ID and groupString attribute
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//tbody[@id][@groupstring]");

if (nodes != null)
{
foreach (HtmlNode node in nodes)
{
// extract the Group Name
string strGroupName = node.InnerText.Substring(node.InnerText.LastIndexOf("&nbsp;")+6);
strGroupName = strGroupName.Substring(0, strGroupName.IndexOf("&#")-1);
Console.Write ("Group: " + strGroupName + ", ");

// extract the number of items
string strValueText = node.InnerText.Substring(node.InnerText.LastIndexOf("(") + 1);
Console.WriteLine("Number of Items: " + strValueText.Substring(0, strValueText.Length - 1));
}
}

As you can see I'm doing some rather nasty SubString statements.. there may well be a quicker and cleaner way to do this using Regex .. this was more a proof of concept than anything else :)

Result!

Console output, showing group names and counts.

3 seconds isn't bad, running on a "single server" laptop VM image :)

The end result was 2-3 second bit of code, retreiving Group By, Count values for a list with 200,000 list items.

Not bad for an afternoons work :)

Attempt 4 - Do the same thing in JQuery (kudos to Jaap Vossers)
This was actually the original solution, I asked Jaap if he could look at this if he had spare time, as I knew he had a lot of JQuery experience (and he blew me away by having it all working in under 30 minutes!).

Basically it uses pretty standard JQuery to go off and retrieve the HTML content from another page, scraping the HTML and pulling back the values. Same as the C# it grabs the group TBody, then walks down the DOM to retrieve the text value that it outputs.

The speed is roughly the same as the actual view itself. I'm sure some more JQuery could be employed to pull out the specific values and do more with them, but the concept appears to be sound:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>

<script type="text/javascript">
$(document).ready(function(){

// you will need to change this URL
var url = http://myspsite/Lists/MyList/GroupedView.aspx;

var groupings = [];

$.get(url, function(data) {
$(data).find("tbody[id^=titl][groupString] > tr > td").each(

function(index, value){
groupings.push($(this).text());
}
);

$("#placeholder").append("<ul></ul>");

$.each(groupings, function(index, value){

$("#placeholder ul").append("<li>" + value + "</li>")
});
});
});

</script>
<div id="placeholder"></div>

tada .. (thanks Jaap!)

Result from JQuery output,

dropped into a Content Editor Web Part

Summary
Well .. I know doing HTML scraping isn't pretty, but seeing as the code is MUCH faster than anything else I've seen (and is stuck in the middle of a COM object) there didn't seem to be much choice.

By all means, feel free to let me know if you have any alternatives to this.

RCWP Part 3 - Edit Web Part using a Ribbon modal dialog

Also check out the other part of the series:

• RCWP Part 1 – Extending the Search Core Results Web Part
• RCWP Part 2 – Web Part with Ribbon Contextual-Tab
• RCWP Part 3 – Edit Web Part using a Ribbon modal dialog

This follows on from Part 1 (where we created a “Related Content Web Part”) and Part 2 (where we added a contextual tab to the Ribbon).

This post summarised the final part of this Web Part (which we completed in the final session of the day of SPRetreat last Saturday).

We wanted to provide a pop-up window, accessed through our new Contextual Tab in the Ribbon, which allowed us to easily modify some web part properties.

The basis of this was quite straight-forward, and it certainly starts off quite easy.
We created a new Application Page (RCWP_SetFieldValue.aspx) which would contain the code to update our Web Part Properties.

In this file we added a simple ASP.Net Label, Drop Down List and button.

<asp:Content ID="Main" ContentPlaceHolderID="PlaceHolderMain" runat="server">
<p>
    This allows you to set the field value for the <strong>Related Content Web Part</strong>
</p>
<asp:Label runat="server" ID="lblChoice" Text="Select Field:" AssociatedControlID="ddlFields" /><br />
<asp:DropDownList runat="server" ID="ddlFields" /><br />
<asp:Button runat="server" ID="btnNike" Text="Just Do It!" />
</asp:Content>

Back in Part 2 we created a JavaScript file which was use for the “Command” events for our Buttons (yes .. I told you we’d be looking at that again!).

Here we are going to modify one of the Buttons so that it throws up a SharePoint Modal Dialog with our Application Page in it.

The code below is modified from the original MS Blog Article I referenced in Part 2 (called “How to create a Web Part with a Contextual Tab”).

if (commandId === 'CustomContextualTab.GoodbyeWorldCommand') {
            //alert('Good-bye, world!');
            var options = {
                url: '/_layouts/SPR3/RCWP_SetFieldValue.aspx,
                title: 'Set Field',
                allowMaximize: false,
                showClose: true,
                width: 800,
                height: 600
            };
            SP.UI.ModalDialog.showModalDialog(options);
        }

I have basically changed the JavaScript for the “GoodbyeWorldCommand” button so that it does something different.

I am using the new SP.UI.ModalDialog namespace in the SharePoint ECMAScript to pop up a modal dialog window.

(Note - I also changed the display text to “Set Field” .. and deleted the other button to clean up the ribbon a bit)

But don’t forget that our Application Page is running from _layouts … it’s in a completely different place to our Web Part so this really isn’t enough for our page to work. In order to do anything else our Layouts page would need the following information:

• The Page that the web part is on (URL)
• Which Web Part to update on that page (Web Part ID)

The URL of the current page is easy enough using JavaScript (location.href) but the Web Part ID … this represented a new challenge.

How do you get the server-side Web Part ID through JavaScript?

This problem took the entire final hour of the day (Session 5) and took quite a bit of research and web searching. Eventually (after a few suggestions) we hit upon the answer:

Back in Part 2 we created a JavaScript file which was used to register our Contextual Tab. The JavaScript file that registers the Contextual Tab contains a reference to a “PageComponentId”.

getId: function ContextualTabWebPart_CustomPageComponent$getId() {
    return this._webPartPageComponentId;
}

Thes pecific instance of our Web Part had a “PageComponentID” of “WebPartWPQ2” and after some digging we found it in the Source of the page!

<div WebPartID="866ef42d-6626-45e0-af9c-a00467ed2666" WebPartID2="1ad9529a-5e86-4e7c-9d4d-022a1fa6e6c0" HasPers="false" id="WebPartWPQ2" width="100%" class="ms-WPBody noindex ms-wpContentDivSpace" allowRemove="false" allowDelete="false" style="" >

The attribute that REALLY stands out though is the WebPartID:

WebPartID="866ef42d-6626-45e0-af9c-a00467ed2666"

This is clearly a GUID value, referring to the server-side Web Part ID for that instance of the Web Part.
So .. how do we get this to our dialog.. well, good old trusty document.GetElementById() (we could have used JQuery, but I didn’t want to have to install the framework .. and don’t forget .. I only had 1 hour to get this working at SPRetreat!!)

Using this information, I could modify my JavaScript to retrieve these values, and pass them through to my Modal Dialog.

// get the Web Part DIV element
            var element = document.getElementById(this._webPartPageComponentId);
            // extract the Web Part ID (as a GUID object)
            var wpID = element.attributes["WebPartId"];
            // pass through the URL and Web Part ID
            var options = {
                url: '/_layouts/SPR3/RCWP_SetFieldValue.aspx?wpID=' + wpID.nodeValue + '&url=' + location.href,
                title: 'Set Field',
                allowMaximize: false,
                showClose: true,
                width: 800,
                height: 600
            };
            SP.UI.ModalDialog.showModalDialog(options);

Note – as the Web Part ID is of type HTML Attribute, we need to use the “NodeValue” property instead of toString();

So .. first off, in our Application Page we can use the URL to retrieve the fields from the page’s Content Type and populate our Drop Down List.

protected void Page_Load(object sender, EventArgs e)
        {
            TargetUrl = Request.QueryString["url"];
            // remove any query strings
            if (TargetUrl.IndexOf("?") != -1)
            {
                TargetUrl = TargetUrl.Substring(0, TargetUrl.IndexOf("?"));
            }
            if (!Page.IsPostBack)
            {
                ddlFields.Items.Clear();
                SPFile file = this.Web.GetFile(TargetUrl);
                foreach (SPField field in file.Item.Fields)
                {
                    if (!field.Hidden)
                    {
                        ListItem item = new ListItem(field.Title, field.StaticName);
                        ddlFields.Items.Add(item);
                    }
                }
            }
            btnNike.Click += new EventHandler(btnNike_Click);
        }

I did a bit of string manipulation on the URL to make sure we trim out any URL query strings, and then use that to retrieve an SPFile object.

We then just iterate through the SPListItem.Fields collection, adding any fields that are not hidden.

Note – we are using an ASP.Net ListItem object in the Drop Down List, so that we can use the Display Name in the drop-down, but store the Static Name as the value .. it’s the Static Name we need to save to our Web Part!

The next bit is under our Click event. We can now use the URL to get the SPLimitedWebPartManager for the page, and pass through the Web Part ID property, and it would retrieve the instance of my Web Part (allowing me to set the field value).

protected void btnNike_Click(object sender, EventArgs e)
       {
           // get Web Part ID
           wpID = Request.QueryString["wpID"];
           // retrieve the Web Part Panager for the URL
           SPFile file = this.Web.GetFile(TargetUrl);
           SPLimitedWebPartManager wpm = file.GetLimitedWebPartManager(PersonalizationScope.Shared);
           // get the safely-casted web part object
           RelatedContentWebPart.RelatedContentWebPart wp =
               wpm.WebParts[new Guid(wpID)] as RelatedContentWebPart.RelatedContentWebPart;
           if (wp != null)
           {
               // set the web part property, and save settings
               wp.FieldName = ddlFields.SelectedValue;
               wpm.SaveChanges(wp);
           }
           // close the modal dialog
           this.Context.Response.Write("<script type='text/javascript'>window.frameElement.commitPopup();</script>");
           this.Context.Response.End();
       }

So .. we should be done…

Build / Deploy / Test

So .. a long journey but worth it, five different 1 hour sessions and a great day at #SPRetreat .. but definitely worthwhile, and a new “Related Content Web Part” to boot!

A massive thanks to Andrew Woodward (21Apps) and Ben Robb (CScape) for organising the event, the venue and the food! (great food!!!)

Source Code

Sorry it took so long for me to get it all online, I was very busy then went on holiday. You can find all of the source code downloadable from my SkyDrive here:

http://cid-60f12a60288e5607.office.live.com/self.aspx/SPRetreat/SPR3.zip

RCWP Part 2 - Web Part with Ribbon Contextual-Tab

Also check out the other part of the series:

• RCWP Part 1 – Extending the Search Core Results Web Part
• RCWP Part 2 – Web Part with Ribbon Contextual-Tab
• RCWP Part 3 – Edit Web Part using a Ribbon modal dialog

This follows on from Part 1, where we created a “Related Content Web Part” (RCWP) which would dynamically return search results based on the value of a field on the current page.

This post summarises the second-phase of that Web Part (which we started in session 4 of SPRetreat) to provide an easier method for the Content Editors to cherry-pick which field they wanted to use for the keyword search value. Seeing as me and my session 4 partner hadn’t done anything with Contextual Tabs before, we decided to give that a crack.

I’m not going to cover every single details of this. There are 2 fantastic articles:
Chris O’Brien (MVP) has a great 4-Part series titled “SharePoint 2010 Ribbon Customisation series” which is a must-read if you want to understand more about Ribbon development work.

There is also an excellent step-by-step guide on the Microsoft SharePoint Developer Documentation Blog called “How to create a Web Part with a contextual tab”.

In my example, I used the Microsoft blog’s example code, but there are plenty of good tutorials for this on the web. I’m not going to copy the same information in the blog article here, but the summarised core points are as follows:

• Your Web Part must implement the Interface IWebPartPageComponentProvider(Microsoft.SharePoint.WebControls).
• The Interface involves a WebPartContextualInfo property, which you use to register you “contextual” event
• You will need a JS file to stored your “actions”
• You will need XML configuration to register the definition of your “tab” and it’s controls.

So first off, we need to modify our web part to implement the Interface IWebPartPageComponentProvider:

public class RelatedContentWebPart : CoreResultsWebPart,
    IWebPartPageComponentProvider
{
    // omitted for clarity
}

Now, before we implement the Interface we need to setup a few methods and properties.
First up is the XML schema for both the Tab Template and the Tab itself. It’s a lot of XML so I’m not going to post it here, but I basically copied the XML provided on the MS Blog post, so you should find it easy enough to go and fetch it from there or just use my Full Source Code (in Part 3).

A snippet of the first part is below;

private string contextualTab = @"
   <ContextualGroup Color=""Magenta""
     Command=""CustomContextualTab.EnableContextualGroup""
     Id=""Ribbon.CustomContextualTabGroup""
     Title=""Related Content Web Part""
     Sequence=""502""
     ContextualGroupId=""CustomContextualTabGroup"">
          <Tab
              Id=""Ribbon.CustomTabExample""
              Title=""Filtering Settings""
              Description=""This holds my custom commands!""
// remainder omitted for clarity

You can see that we have created a string called “contextualTab” that references a “ContextualGroup”. This has a colour, as well as a unique ID that we need to create. The Title will appear in the ribbon as the “contextual” group, which will span across multiple tabs (if you have multiple tabs in the same “contextual group” of course!).

Note - I have changed the title and description of my Tab and the Contextual Group, but everything else is the same as the MS Blog!

We also have a standard Ribbon “Tab” declaration, with an Id value which we will need later) as well as the Tab Title and Description.

Further down in the XML is a reference to the "Buttons” that we are going to have on our tab.
(Note – the code below is an exact copy of the "MS Blog Article” code .. we will be changing this later on!)

<Button     Id=""Ribbon.CustomTabExample.
    CustomGroupExample.GoodbyeWorld""
    Command=""CustomContextualTab.GoodbyeWorldCommand""
    Sequence=""15""
    Description=""Show the Goodbye World text""
    LabelText=""Goodbye World!""
    TemplateAlias=""cust1""/>

Make a note of where these button controls are, because we’re going to need to modify this in Part 3 of this post!

There is also another string created called “contextualTabTemplate” which you can use for the layout configuration.

private string contextualTabTemplate = @"
          <GroupTemplate Id=""Ribbon.Templates.CustomTemplateExample"">
            <Layout
              Title=""OneLargeTwoMedium"" LayoutTitle=""OneLargeTwoMedium"">
              <Section Alignment=""Top"" Type=""OneRow"">
                <Row>
                  <ControlRef DisplayMode=""Large"" TemplateAlias=""cust1"" />
                </Row>
              </Section>
// remainder omitted for clarity

So … now we have all of our XML defined we need to create a JavaScript file. The MS Blog article includes a JS file called “CustomContextualTabPageComponent.js” which deploys directly to the 14\Template\Layouts\ folder.
This contains a whole load of JavaScript for the Tab, Group and Button items in your XML file.

Important Note – The Command and Enable scripts use in your XML are in this JS file. If you start changing names you will need to manually keep these files in check.. they are XML and JS files so no compile errors or warnings! … except this one ;)

Probably the most important section in this file is that which handles the “Command” event for your buttons:

handleCommand: function ContextualTabWebPart_CustomPageComponent
    $handleCommand(commandId, properties, sequence)
{
    if (commandId ===
        'CustomContextualTab.HelloWorldCommand) {
            alert('Hello, world!');
    }
    if (commandId ===
        'CustomContextualTab.GoodbyeWorldCommand') {
            alert('Good-bye, world!');
    }
}

Again, this code has been copied from the MS Blog, but keep a note of it .. we will be modifying this too in Part 3!

So .. we have our XML schema, and we have our JS file created, but we haven’t actually DONE anything with them yet.

So lets create a method to register our new Contextual Tab:

private void AddContextualTab()
        {
            //Gets the current instance of the ribbon on the page.
            Microsoft.Web.CommandUI.Ribbon ribbon = SPRibbon.GetCurrent(this.Page);
            //Prepares an XmlDocument object used to load the ribbon extensions.
            XmlDocument ribbonExtensions = new XmlDocument();
            //Load the contextual tab XML and register the ribbon extension.
            ribbonExtensions.LoadXml(this.contextualTab);
            ribbon.RegisterDataExtension(ribbonExtensions.FirstChild, "Ribbon.ContextualTabs._children");
            //Load the custom templates and register the ribbon extension.
            ribbonExtensions.LoadXml(this.contextualTabTemplate);
            ribbon.RegisterDataExtension(ribbonExtensions.FirstChild, "Ribbon.Templates._children");
        }

So .. this method is basically using the SPRibbon object to get a reference to the Ribbon on the current page. It is then loading in our XML schema to register a new contextual tab).
(Eventually we will end up calling this method from the PreRender() event).

Now we are going to need to register our JavaScript, and strangely enough we are going to do that with yet more JavaScript! (yes .. when you are working with the Ribbon JavaScript really is word of the day!)

public string DelayScript
        {
            get
            {
            string webPartPageComponentId = SPRibbon.GetWebPartPageComponentId(this);
            return @"
            <script type=""text/javascript"">
            //<![CDATA[
            function _addCustomPageComponent()
            {
                var _customPageComponent = new ContextualTabWebPart.CustomPageComponent('" + webPartPageComponentId + @"');
                SP.Ribbon.PageManager.get_instance().addPageComponent(_customPageComponent);
            }
            function _registerCustomPageComponent()
            {
                SP.SOD.registerSod(""CustomContextualTabPageComponent.js"", ""\/_layouts\/CustomContextualTabPageComponent.js"");
                SP.SOD.executeFunc(""CustomContextualTabPageComponent.js"", ""ContextualWebPart.CustomPageComponent"", _addCustomPageComponent);
            }
            SP.SOD.executeOrDelayUntilScriptLoaded(_registerCustomPageComponent, ""sp.ribbon.js"");
            //]]>
            </script>";
            }
        }

The really important part of this is the first line:

SPRibbon.GetWebPartPageComponentId(this);

This gets a unique reference for the instance of your Web Part, on the current page, in the context of the Ribbon. This is used to effectively identify your Web Part when someone clicks on it!
The other really important bit is the last line:

SP.SOD.executeOrDelayUntilScriptLoaded(_registerCustomPageComponent, ""sp.ribbon.js"");

This is the new “Script On Demand” (SOD) method which allows us to tell SharePoint not to try loading our JavaScript until the SP.Ribbon.JS has already been processed!
Once that has been done we need to implement our WebPartContextualInfo method:

public WebPartContextualInfo WebPartContextualInfo
        {
            get
            {
                // create objects for the contextual web part tab
                WebPartContextualInfo info = new WebPartContextualInfo();
                WebPartRibbonContextualGroup contextualGroup = new WebPartRibbonContextualGroup();
                WebPartRibbonTab ribbonTab = new WebPartRibbonTab();
                //Create the contextual group object and initialize its values.
                contextualGroup.Id = "Ribbon.CustomContextualTabGroup";
                contextualGroup.Command = "CustomContextualTab.EnableContextualGroup";
                contextualGroup.VisibilityContext = "CustomContextualTab.CustomVisibilityContext";
                //Create the tab object and initialize its values.
                ribbonTab.Id = "Ribbon.CustomTabExample";
                ribbonTab.VisibilityContext = "CustomContextualTab.CustomVisibilityContext";
                //Add the contextual group and tab to the WebPartContextualInfo.
                info.ContextualGroups.Add(contextualGroup);
                info.Tabs.Add(ribbonTab);
                // fetch dynamic component info for the current page's Ribbon control
                info.PageComponentId = SPRibbon.GetWebPartPageComponentId(this);
                return info;
            }
        }

Now lets take a quick look through some of this code first. The first thing you should notice is that there are 3 new classes provided in the OOB API specifically for providing contextual tabs (from Web Parts):

• WebPartContextualInfo
• WebPartRibbonContextualGroup
• WebPartRibbonTab

These objects contains all of the core functionality that you will need to handle the interaction between selecting your Web Part in edit mode, and the Ribbon dynamically popping up your Tab when that happens.
The RibbonContextualGroup and RibbonTab objects then get given both an ID property and a VisibilityContext. These values refer to the same pointers in the XML schema definition for the Tab, Group and Controls that we are going to register (which we created earlier).
The final part of the puzzle is to add these to the Web Part’s load stack (in this case, the Pre-Render method):

protected override void OnPreRender(EventArgs e)
{
            base.OnPreRender(e);
            this.AddContextualTab();
            ClientScriptManager clientScript = this.Page.ClientScript;
            clientScript.RegisterClientScriptBlock(this.GetType(), "ContextualTabWebPart", this.DelayScript);
}

So, we are registering our contextual tab (using the AddContextualTab method we created earlier).
We then dynamically add a “lazy load” reference to our “DelayScript” string, which will register all of the JavaScript functions that we are using.

Build / Deploy / Test
There you have it … a contextual tab in edit mode!

Make sure you check back for:

RCWP Part 3 – Edit Web Part using a Ribbon modal dialog

(Full source code for the solution will be published in Part 3)

RCWP Part 1 – SPRetreat and the Related Content Web Part

I’m sat on the train after a great day of SPRetreat (followed by SharePint of course!), superbly organised by Andrew Woodward (21Apps) and Ben Robb (CScape). It was a really good day of innovative ideas, problem solving, chewing the fat (and the occasional dirty joke... you know who you are!).

The core challenge thrown down for the day involved trying to provide cross-site (collection) “related information”, effectively a “cross-pollination” function, using SharePoint 2010. There were some great ideas and a lot of top effort into involving the Managed Metadata Service, Custom Search API work and some cracking Scrum / Agile design processes.

We had 5 sessions of 1 hour each, and my efforts for the day mostly revolved around delivering a “Related Content Web Part”, which could use Search to show other content from any data source which is in some way related to the information on the current page.

In this post I’m going to walk through how my efforts of the day culminated in the “Related Content Web Part” but ended up being a generic “Dynamic Field Driven Search” web part, basically allowing a set of search results to be displayed based on the value of a field that is stored in the publishing page (i.e. content type) that contains the web part (which derived from the Search CoreResultsWebPart).

In the final sessions we though about how to improve this, and ended up building a custom contextual ribbon interface to surface SP Modal Dialogs to allow easy updating of core web part properties.
Core Functionality

The functionality is really split into 3 major sections:

• RCWP Part 1 – Extending the Search Core Results Web Part
• RCWP Part 2 – Web Part with Ribbon Contextual-Tab
• RCWP Part 3 – Edit Web Part using a Ribbon modal dialog

(Full source code for the solution will be published in Part 3)

RCWP Part 1 - Extending the Core Results Web Part
This is one of the nicest new “features” of SharePoint 2010. They have stopped sealing all of the Web Parts (and Web Part Connections)that are used for the results pages in SharePoint Search solutions. This means it is now much easier for you to extend and add-value to these web parts without having to throw out all of the OOB functionality.
The reason for using the CoreResultsWebPart was simple;

• Using search is fast, efficient, cross farm and highly configurable.
• The core results web part using XSLT for rendering, so easy to design the output.
• Leveraging an OOB web part means we get loads of added functionality for free! (like specifying which scope we want to use).

In this solution, we extended the CoreResultsWebPart to create our own Web Part. Simple create a new Visual Studio 2010 “Web Part” item and set it to inherit from “CoreResultsWebPart” (you will need to add a reference to Microsoft.Office.Server.Search.dll).

[ToolboxItemAttribute(false)]
public class RelatedContentWebPart : CoreResultsWebPart
{
}

In terms of functionality we need to do 1 thing:

Override the Query programmatically, based on the metadata of the current page.

This involved a few steps. First off, we need to identify which field we want to be “targeting”. For this we created a simple string field, exposed as a Web Part Property.

private string fieldName = "Title";
///<summary>
/// The field on the current page we want to use for filtering
///</summary>
[WebBrowsable(true)]
[Personalizable(PersonalizationScope.Shared)]
[WebDisplayName("Field Name")]
[SPWebCategoryName("Filter SPR")]
[WebPartStorage(Storage.Shared)]
[Description("Which field on the current page do you want to use for filtering?")]
public string FieldName
{ get { return fieldName; } set { fieldName = value; } }

Then we needed to override the query itself. This is simply done by setting the “FixedQuery” property of the CoreResultsWebPart. The trick here is you need to set this in the “OnInit” event, otherwise it won’t take effect (if you try to place it in OnLoad, CreateChildControls or any other later method then it won’t have any effect!).

protected override void OnInit(EventArgs e)
{
    this.FixedQuery = “keyword string value”;
    base.OnInit(e);
}

Finally, we need to make sure we are pulling out the field value of the current list item, based on our custom field. For this we used a simple set of SPContext combined with some defensive programming to make sure we don’t get any NullReferenceException errors. So change the “OnInit” event to the following:

protected override void OnInit(EventArgs e)
{
    if (!string.IsNullOrEmpty(fieldName))
    {
        if(SPContext.Current.ListItem.Fields.
            ContainsFieldWithStaticName(fieldName))
        {
            this.FixedQuery =
                SPContext.Current.ListItem[fieldName].ToString();
        }
    }
    base.OnInit(e);
}

After that … Build / Deploy and the web part was working!

The code could obviously be re-factored a little bit, but on the whole it’s all working :)
Make sure you check back for:

RCWP Part 2 – Web Part with Ribbon Contextual-Tab

RCWP Part 3 – Edit Web Part using a Ribbon modal dialog

(Full source code for the solution will be published in Part 3)

SP 2010 Websites and JavaScript "on demand" bug

We are currently working on building the new Content and Code website on SharePoint 2010. This is a “brochure-ware” anonymous publishing site, and as such it is fairly lightweight (with most of the normal “heavy” JS files for things like the SP Ribbon, dialogs framework and the like not required).

 

You can imagine my surprise when our landing page was showing up as over 650KB in size! On further inspection, the vast majority of this was OOB JavaScript files (including the Ribbon, dialog and core JS files!).

 

I thought we’d made a mistake so I checked an OOB Publishing Portal and even the public “sharepoint.microsoft.com” website (both had the same problem) so it appears this is a more fundamental problem! Importantly, ALL of these tests are as an anonymous user!

 

Lazy Load vs On Demand

The first thing to mention is that these files are on “lazy load” (i.e. they load AFTER the UI has been rendered). So end users don’t really notice .. the page is displayed fairly quickly, and the remaining JS files download in the background.

 

But lets be clear, this is NOT “on demand” JS. Files are clearly being loaded that should NEVER be required by anonymous users (like the ribbon??)

The “On Demand” was one of the major infrastructure improvements to promote fast and efficient web-based systems, and for custom development works a charm (my own visual web part with on-demand JS worked perfectly .. the custom JS file didn’t get downloaded until I called a specific JS function).

Caching

 

The second thing to mention is that it appears these files are only downloaded once. Doing repeated page requests (or even “Ctrl-F5” full load requests) do not bring back these files unless you do a full clear of your browser cache (or try using a different browser) but it does mean that the “first hit” page size is rather large to say the least.

 

So how many files are we talking about?

 

I decided to test this on a public facing SharePoint 2010 Website, and the easiest one that sprang to mind was http://sharepoint.microsoft.com (a newly launched Microsoft site powered by SharePoint 2010 specifically for the launch).

 

I was using Fiddler for the session information (as I could use it for multiple browsers, and knew it was independent to the actual page-source).

I found that, on the first page hit (subsequent pages do not load them) the following JavaScript references were being downloaded (size in Bytes shown in brackets):

• core.js (237,096)
• sp.core.js (12,349)
• cui.js (351,361)
• sp.ui.dialog.js (34,243)
• sp.runtime.js (68,784)
• sp.js (389,372)
• inplview.js (38,836)

 

 

You can see the actual page statistics below, where the 9 JavaScript references represent 1,362,477 Bytes of data! (that’s over 1MB of JavaScript!)

 

 

  Is there a solution?

 

Well, I haven’t looked too deeply into a solution yet (still peeling back layers looking for what is causing the “on demand” to load on first hit).

 

One solution is not to use the <SharePoint:ScriptLink> control, and manually refer to whichever JS files you need in your Master Page (using Edit Mode Panels and Security Trimmed Controls to make sure the appropriate JS is available for editing experience and the like).

 

This however is a complicated approach and requires quite a deep understanding of the JS files, what they do and when they are required (not for the faint hearted).

 

I would certainly be very keen to hear about your run in with this particular issue. Do you experience this issue on your environments? Have you managed to get around it yet?

 

Meanwhile I’ll continue my on-going love-hate relationship with SharePoint and try to work out how to stop my website having a 1MB page payload.

Content and Code wins Microsoft Partner of the Year Awards

Its that time of year again, and the Microsoft WPC (Worldwide Partner Conference) is almost upon us (Washington D.C., USA, July 11th-15th).

But before the conference kicks off Microsoft has announced the award finalists and winners, and Content and Code has won two categories:

• Microsoft Worldwide Partner of the Year, Information Worker Solutions, Enterprise Content Management (link)
• Microsoft Country Partner of the Year, UK (link)

I am incredibly proud of Content and Code for winning these awards, not least because the award was attained through the RNIB project which I worked on (along with a fantastic team at both Content and Code and the RNIB!). This is a great achievement, and I'm both proud and privileged to have worked on that project, and for Content and Code themselves!

To top this off, Content and Code won Microsoft Worldwide Partner of the Year, Information Worker Solutions, Portals and Collaboration for 2009, so this now makes 2 years running that Content and Code have won a Worldwide Partner of the Year recognition!

The official announcement can be found on the Worldwide Partner Conference site.

SharePoint 2010 Exams Announced

A few days ago Microsoft announced several new SharePoint 2010 exams that will be coming soon to a testing centre near you...

IT Pro Track

• TS: Microsoft SharePoint 2010, Configuring
• PRO: Microsoft SharePoint 2010, Administrator

Passing both of those exams will earn you the new certification:

MCITP: SharePoint Administrator 2010
(Microsoft Certified IT Professional)

Developer Track

• TS: Microsoft SharePoint 2010, Application Development
• PRO: Designing and Developing Microsoft SharePoint 2010 Applications

Passing both of those exams will earn you the new certification:

MCPD: SharePoint 2010
(Microsoft Certified Professional Developer)

Unfortunately there are no upgrade paths from SharePoint 2007 (which makes sense, as the structure has changed).

Of course, the exclusive Microsoft Certified Master will continue to run including SharePoint 2010.