Load Testing SharePoint 2010 with Visual Studio Team Test

 

So exactly what do we mean by "load testing" when it comes to SharePoint 2010? There are lots of methods that people tend to point towards, and I've heard "hits/visits per day" and "throughput" bandied about, but at the end of the day it comes down to 2 things:

 

1. Requests Per Second

The requests per second literally means how many requests for information each server is capable of responding to per second. Each page may consist of dozens of artifacts, and for each artifact the browser needs to make a "request", therefore the more of these  "requests" it can serve the better.

 

1. Server Response Time.

The response time represents any processing on the server side (or TTLB - Time to Last Byte). This doesn't factor in network latency or bandwidth though!

 

So the first thing you should think about is what can influence those metrics? And you end up with 5 different elements of your SharePoint 2010 farm:

• WFE
• Storage
• Network
• App Servers
• SQL

 

This, as I'm sure you can imagine, can involve a LOT of testing. Simply testing the WFE on their own is going to be struggle for your average developer, and if you don't have any industry testing experience you are going to have a hard time, but this is where the new SharePoint 2010 wave continues to make it's presence felt. ..

 

SharePoint 2010 Load Testing Toolkit

This is a new set of tools being released with the SharePoint 2010 Administration Toolkit and represents the easiest possible way of load testing your SharePoint environment. The main objective here is to:

 

• Standardise and simplify the cost of load testing.
• Simulate common SharePoint operations
• Be used as reference to create other custom tests (for custom code, for example!)

 

The whole thing relies on the IIS analysis logs. These logs give pointers on where users are going, what kinds of requests they are doing (GET / PUT) as well as the types of files they are typically accessing (ASPX / CSS / JS / JPEG / DOCX / etc...)

 

The Load Testing Toolkit will analyse your IIS logs and automatically generate a set of loads tests to appropriately match your environment, producing automated scripts that can be run in Visual Studio (either Team System or Team Test Edition).

 

How hard can it be?

It is really quite simple (well, according to the ridiculously simple explanation at the SharePoint 2009 conference!). You literally point the tool at your IIS logs, and it spits out an entire suite of tests, for WFE, SQL, Storage, etc .. Including all the metrics you would want (from CPU, RAM, Network, Disk I/O and even SQL , ASP.Net and .Net Framework specific performance counters).

 

Then you just run it and analyse the results!

 

Analyse That!

The analysis couldn't be simpler. With "Requests Per Second" and "Response Times" two of the metrics generated by the Visual Studio test reports, you really can't go far wrong.

 

If you do find a problem, then you can delve into the new SharePoint 2010 "Usage Database" (which now runs on SQL Server) in order to identify exactly what was causing your dip in performance (say when someone deletes a large list?).

 

Tips and Tricks

There are a few gotchas, one thing is to be careful of "Validation Rules" in Visual Studio. Typically it will be happy with pages that return "200" codes. This of course includes Error and Access Denied pages (which SharePoint will handle, and returns a perfectly valid page (hence the 200 code!)).

 

It is also recommended that you let your test "Warm up" for around an hour before you start taking the results seriously.  This allows all of the operations, timers and back-end mechanics of SharePoint to properly settle down, and means you are getting a realistic experience of what the environment will react like once it is bedded into it's production environment.

 

Finally, the SharePoint Usage Logging Database is a great location to grab information out of, so why not leverage other great aspects of the Office 2010 family. You could pull through the Usage DB information into Excel 2010 (perhaps using PowerPivot?) so that you can spin out charts and pivot tables to easily drill down into your data.

 

Typically load testing tells you WHEN bottlenecks are occurring, but the Usage Database can tell you WHAT is causing the bottlenecks!

SharePoint 2010: Architecture Guidance - things everyone should know!

Well, the final day of the conference came and with it some of the most useful sessions (from my perspective). One of which was the "Architecture Guidance for SharePoint 2010". This hopefully distils some of that information. It's not a be all and end all, but hopefully points you in the right direction so that you can focus your research a little better!

 

[UPDATED: 27/10/2009 16:09]

 

UI Design

• Entire interface in SharePoint 2010 to be W3C XHTML compliant
• SharePoint 2010 "more accessible mode" to be WCAG 2.0 AA compliant
• New ribbon interface replaces toolbars and menus (and considerations for old "CustomAction" commands which may no longer work!)

• Wiki content allows web parts to be dropped in (removing over-reliance on web part zones)

 

Lists

There are a whole load of new List capabilities (in addition to the "External List" that BSC brings to the plate!).

• Lookup to Multiple

This means that when you create a new lookup column, you can now pull down additional fields from the lookup list item and use them for filtering.

• CAML support for Joins!

You can now perform "JOIN" operations in your CAML queries for linking lists together.

• Enforced List Relationships

You can now enforce specific relationships for lookup columns with two options:

• Restrict Delete - cannot delete parent if child items exist.
• Cascade Delete - If you delete the parent, all child items are automatically deleted (recycle bin aware with "restore" options!)

• Store-level enforcement

This is code level "required fields", so now you can enforce the requirements even through code !

• Unique Fields

Specify a unique field, so that no two values can match (e.g. Email addresses in contacts list)

• Compound Indices

If you want to query by 2 fields, you can now index both at once as a compound index.

• <In> clause for reverse lookups

This allows a CAML query to do a reverse lookup to get all child items that are associated with the parent!

• Formula based validation

e.g. Don't allow Field2 to be lower than Field1.

 

Workflows

• Out of the box SharePoint 2010 workflows can now be extended in SharePoint Designer 2010.
• SharePoint Designer 2010 can be used to create "re-usable" workflows
• Site Workflows - to manage processes across an entire site.
• You can now import a SharePoint Designer 2010 workflow into Visual Studio 2010!
• Import/Export workflow using Visio 2010 for visual workflow modelling.

 

Content & Document Management

• "Document Sets" allow you to treat a group of documents as a single item (with 1 version history, group executed workflow and policy, and a "download as zip" option).
• Managed Metadata Service  allows cross-farm Content Type management and a pre-defined enterprise taxonomy structure! This is a killer-app, bringing true enterprise content management to SharePoint 2010.
• Enterprise Wiki's allow more rapid "in edit" content, as well as Web Parts deployed directly into the rich text editor (no more web part zones?).
• Spelling check and broken link check when you "check-in" WCM pages.

 

Event Handlers

Three new event handlers added (at last!!)

• WebAdded - Fired every time a child site is created in the web.
• ListAdded - Fired every time a list is created in the web.
• Feature Upgrading  - Fired when a feature has it's "upgrade" method called (more on this in a future blog post).

 

Security

• Editing of ASPX pages now required "Designer" permissions (instead of contribute).
• XSS (Cross Site Scripting) protection for pages and web parts.
• HTML pages will now "force download" by default. This stops people from uploading HTML files with malicious scripts, so if you click on an HTML file in a document library you will get a download dialog instead of the file opening in the browser!
• There are still no field level permissions (it was estimated that this would add a 30% overhead to performance! Maybe in a future release)

 

BI and Connectivity

• New Business Connectivity Services (BCS) allows no-code connections of databases and LOB systems to content types and lists with two-way synchronisation of data  and full CRUD support.
• BCS interactivity from within Office clients, allowing LOB system data to be edited directly from desktop applications (such as Outlook and Word).
• PowerPivot for Excel allows upwards of 100 million rows into an excel workbook with phenominal performance.

 

Office Application Support

• New web level services for applications (Excel / Visio with JavaScript events!)
• SharePoint Workspace to replace "Groove" for offline file support and editing.
• Office Web Applications to allow for direct opening and editing of documents from within the browser!
• InfoPath 2010 can now be used to edit the List forms out of the box!

 

Databases

• Still a 100GB "limit" for content databases.
• Still cannot have site collections spanning multiple databases.
• New support for "Failover" databases, SharePoint 2010 is now SQL mirror aware!
• All "Service Applications" have their own SQL database, along with many other new databases (e.g. Feed Activity, Social Data, Usage Logs).
• New "read only content databases" open the door for simple content deployment (utilising SQL log shipping or database replication).

 

Content Deployment

• All execution now in Timer Jobs.
• Performance (and memory usage) improved.
• Export routine now creates database snapshot to improve data integrity!

 

Sandboxed Solutions

• Ability to upload WSPs directly into the content database to execute in minimal permissions using "virtual files" (no impact on the file system!)
• Resource throttling, code performance checking and "bad routine" blocking
• Provides new best practice for code development and deployment!

 

Search

• New FAST search with thumbnail views (and navigation!) for office documents
• Improved relevancy and non-query searching
• 2 new search products (FAST based)
• New refinement panel for advanced sorting and filtering "on the fly"
• Multi-lingual support with over 80 languages built-in.

 

Social Networking

• New My Sites structure
• Activity Feeds to provide updates on user activity with an extensible architecture!
• "Social Feedback" functions akin to Delicious and Digg allowing tagging of any URL based content, and subsequent discussions around items that have been "tagged".
• Ratings mechanism distributed throughout the product.

 

I'm sure there are many other things, so please let me know if there's anything else you think should "make the grade" and I'll see if I can add it in :)

100 million rows in Excel? PowerPivot.. a first look from the SharePoint Conference 2009

 

"Project Gemini" has been batted around for a while now but it was unveiled at the conference that it is now known as SQL PowerPivot for Excel 2010 and SQL PowerPivot for SharePoint 2010.

 

What does it do?

In short, PowerPivot allows you to pull data into an Excel workbook from almost any data source. This can be SQL databases, Analysis Services Cubes, or any ODBC data source.

 

This is all handled via the import wizard, which contains a nice interface to setup which tables and filters you want to apply (the wizard then generates the necessary query).

 

You then have access to a whole raft of Excel Formulas (and a bunch of new aggregation and time intelligence formulas) that you can use to add new columns to the data. You can even bring in your own Excel worksheets as tables of data that can be linked up to the other data sources (say to provide foreign key tables where the lookups are stored and managed in Excel!)

 

Ok ... So what's so special about this?

Well, the main thing that is impressive is that they demonstrated an example system running with over 100,000,000 rows of data! Now remember that this is running from Microsoft Excel!

 

You could then add your own extension columns (using simple Excel style formulas) and the whole  data set refreshes in seconds.

 

So the performance is good huh?

The performance is quite simply jaw-dropping.

 

One of the demo sessions the presenter imported over 3.5 million rows of data from a SQL Analysis Services cube and it imported in just under 2 minutes.

 

He then created a pivot table of the total sales data, split into rows by country.

He then added "slices" so that you can flick between sales figures for different years or product categories.

 

With all of these calculations the pivot table was refreshing it's data in under 2 seconds!

 

Not even SQL Reporting Services can execute that fast, and this is in EXCEL so the user has full control over the pivots and can filter / query / change the results as much as they like.

 

How does it actually work then?

The main thing that PowerPivot does is that the database columns are separated out and compressed individually. Foreign key values can then be separately indexed and this makes the compression levels fantastic.

 

Take an example of a foreign currency field for Europe. Regardless of how many rows of data you have that column is only ever going to contain a small number of different values (£, €, etc). You could have one thousand rows or one billion rows and it would still have the same variation in the values. This makes it extremely compressible so you can get extremely large data sets down to a very small footprint.

 

When you then query the data set it loads those columns into memory for execution, so you end up with a column based querying model running directly from memory (which is the reason it is so incredibly extremely fast).

 

Now before you start wondering if this will only work on beefy 64-bit workstations with RAM in double figures I have been assured by the presenter that this works fine on a 2GB netbook! Although he was running the demo on a quad core laptop (presumably with about 8GB of RAM).

 

What about SharePoint 2010 then?

Well, SharePoint 2010 has support for Excel Services, and with SQL PowerPivot for SharePoint 2010 you can publish Excel Workbooks containing PowerPivot data sets directly to SharePoint!

 

This allows you the flexibility to share and present your workbooks with colleagues and other users of the SharePoint platform directly from the browser!

 

Even better than this, if you save an Excel Workbook containing PowerPivot data to a document library, then you can import that into another PowerPivot workbook!

This means that your PowerPivot workbook has actually become a data source in it's own right, paving the way for true BI applications being built with this technology!

PerformancePoint Services 2010 new features

Some very nice new features for PerformancePoint Services 2010 for creating SharePoint 2010 dashboards.

 

The KPI web parts and filters now execute Asynchronously, so you can expect your web parts to refresh and update without page refreshes (hurrah!)

 

There was also some very nice cool stuff around Time Intelligent Filtering. If you are using SQL Analysis Services then you can use small formula functions like "month" or "year" and it will automatically calculate the query that needs to be called.

 

So for an example, you can create queries to pull through data for:

• Sales this month ("month")
• Sales last month ("month-1")
• Sales this month last year ("(year-1).month")
• Sales last month, last year ("(year-1).month-1")

 

All without any code and without going into SQL, very impressive.

 

There is also improved SharePoint connection settings so that you can associate SharePoint list data with your OLAP based KPIs. This allows you to use SharePoint lists to configure your scorecard information. But better than that, you can also configure your web parts to allow in-place editing of that scorecard information, so now the editing of the scorecard data can take place for within the dashboard itself!

 

Probably the best feature (and certainly got the most applause from the audience at SharePoint Conference 2009) is single-click deployment to SharePoint from the Dashboard Designer application.

 

You can now setup SharePoint connections to configure your dashboards, and from a single click of the button it will compile and deploy all of your dashboards into your SharePoint environment!

Social Feedback and Activity in SharePoint 2010 - Ratings, Tags and Notes

The social functionality in SharePoint 2010 has been massively improved from the previous versions of SharePoint, and one of the areas is around the concept of Social Feedback.

 

Question: How many times have you found a useful link somewhere on the internet, but had no way to usefull record that and get feedback from your colleagues?

 

Well, SharePoint 2010 social feedback can help with this, you can now "tag" any source on the internet (or intranet) which has a URL. This is stored in your "tags" section on your My Site, and also appears in your "Activity Feed" (which is one of the new areas in the SharePoint 2010 My Site).

 

Other users can also post "notes" relating to your tag, which effectively creates a discussion board around the "tagging" activity, allowing conversations around something that has been tagged.

 

Now, one of the key points is Security Trimming. Lets take this example: what happens if you Tag a document that someone else doesn't have access to?

 

The good news is that social tagging uses the Search Index to provide security trimming on content that is stored in SharePoint.

 

This provides the capability for senior managers to tag confidential documents (and hold conversations about that using notes) but those tags (and notes) are not visible to anyone who doesn't have read-access to the document!

 

On top of this is included a Ratings feature, where you can rate content within SharePoint lists (finally, the death of third party "rate my content" web parts).

 

This means that SharePoint 2010 now has similar social feedback functionality as other products like Digg or Delicious, in that you can tag and rate content, and other people can interact with that "tag" creating a discussion.

 

Architecture

All of the Social Feedback information in SharePoint 2010 is stored in a separate "Social Database". This sits alongside the Profile Database.

 

There are then "Gatherers" (Timer Jobs) which will collect all of the changes to both the Social Database and the Profile Database and this is stored in another database for Activity Feeds (the Activity Feed Database) with foreign key pointers back to the Profile Database (so you know who's activity it is).

 

The performance is impressive, aiming for 2000 requests per second, and in terms of storage they are looking to support over 600,000,000 rows of data! They claim that this is sufficient for activity (including social feedback) for 400,000 users over 5 years!

 

Extensibility

You can also hook into this process yourself. You can build your own "Gatherer" jobs to collect information from any data source that you like.

 

A good example is a CRM database, so that you can show activity in CRM in the My Site Activity Feed, showing when people schedule meetings or achieve sales activites.

 

 

All in all the Social Feedback and Activity in SharePoint 2010 is shaping up very nicely. The performance is something that they are still working on, so don't expect amazing results in the Beta version, but Microsoft are already using this for all of their employees so the dogfooding will make sure that this is given all the attention that it needs!

Securing SharePoint 2010 Web Servers

 

This was one of the best topics I've seen so far at the conference. The amount of concrete information was impressive (and to be honest a bit too much to post here) but there was some great information on how to harden your Web Servers.

 

SharePoint 2010 Security Features

There are a whole load of new features and changes to the SharePoint 2010 product for security.

 

• ASPX Pages are gone for contributors. You can no longer upload ASPX pages into document libraries unless you have "Designer" permissions! The main reason this becomes possible is because the new Wiki Pages are so much more extensible than they were.
• Anonymous Users Lockdown feature  now works for Web Services and WSS (SharePoint Foundation 2010)!
• PowerShell Access - you can now delegate remote scripting rights through PowerShell, so you no longer need the Setup account to perform PowerShell commands. This can be delegated to farm administrators!
• XSS (Cross Site Scripting) protection is now in place through the headers (although you can turn it off). This can be even be locked down to individual web part properties (through development)!
• Application Page settings can now be controlled more granularly, so that you can set the master pages used and even swap out individual pages (such as the Error Page). This makes lock downs and branding of these far easier, without breaking the supported state of your environment, and without extensive development!

 

There was then whole load of recommendations for hardening your environments. It's a bit of a list so apologies for that, but a lot of information to get through:

 

Hardening your Web Application

• Place your web application directories on a non-system volume. If you have any issues with logging or file access then the I/O operations (or even disk space requirements) could damage the Operating  System!
• Change the IIS header. By default this will include the SharePoint version number (which means any attacker knows which service packs and critical patches you have installed!). Removing this reduces your public footprint

 

Hardening your Web Servers

Windows Server 2008 takes care of most of the previous recommendations for hardening automatically, but there are still some things that you should do:

 

• Restrict remote administration of the Registry (no-brainer, but a lot of people forget to do this)
• Rename Administrator account
• Delete / Disable unused accounts (again, make sure your dev and test accounts don't hang around on the web front ends)
• Use the IUSR instead of IUSR_<serverName>

The IUSR account is a "built in" account so therefore it doesn't have a password and no-one can login using that account. This makes it much more secure than the Server specific IUSR account that gets created!

 

Hardening SQL 

There's a whole load about this on the internet. The only one to mention here is change the port number! There are a lot of viruses and malware that will specifically target this port.

 

Hardening your Network

Again, none of this is SharePoint specific, but goes a long way to making sure that your network in general is secure (which is of course best practice for SharePoint systems).

 

Routers:

• Block unused protocols and ports (see ports required, below)
• Screen Traffic (e.g. ICMP)
• Intrusion Detection should be in place

 

Firewall

• Use packet filtering policies
• Log your permitted / denied traffic, and make sure those logs are checked (using alerts)
• Make sure perimeter networks are firewall secured, effectively providing end to end firewall security.

 

Switches

• Disable any unused services in the switch
• Do not overly trust VLANS. Just because your traffic is isolated to a VLAN doesn't mean you shouldn't still block off the relevant ports and protocols.

 

Ports Required for Web Servers

Note - When SharePoint is installed the communication ports are automatically opened on the Windows Firewall!

 

External:

• Http 80 / TCP
• HTTPS 443 / TCP
• SMTP 25 / TCP

 

Internal*:

• HTTP 32843/TCP
• HTTPS 32844 / TCP
• TCP 32845 / TCP
• SMB 445 /TCP|UDP

 

* note that "internal" means Web Application --> Service consumtion over WCF. It does not include SQL or inter "server" communications.

Topology Changes for SharePoint 2010 Logical Architecture

The SharePoint 2010 topology has been massively updated, allowing for greater flexibility and scalability than ever before.

 

The "Shared Service Provider" is dead, it doesn't exist in SharePoint 2010 and instead is replaced with new "Shared Service Applications". This allows core services to have their own security settings, run in their own applications and on their own databases.

 

There is even support for "cross farm" Service Applications (such as Search, User Profiles and the Managed Metadata Service) to allow distributed farm architecture like never before. Now in SharePoint 2010 you can scale up into multiple farm environments, allowing you to take advantage of more geo-distribution flexibility, and greater performance and availability from having dedicated farm hardware for important applications.

 

For the larger enterprise environments you have the benefit that different farms provide the opportunity to service different SLA requirements, and the Many - Many relationship for Web Applications to Shared Service Applications means that core enterprise level services can be shared globally, but smaller core specific services can be hosted multiple times, closer to the client environments, to service  those farms that need them.

 

If you need greater security boundaries and better utilisation of resources you can spin up department specific farms for business critical organisational boundaries (such as HR and Finance) each with their own independent services or shared services (such as an HR specific BCS, or Finance and HR sharing their own  set of Managed Metadata for payroll and accounting data, a service that is not provided to the more generalised collaboration and publishing environments).

 

All of this comes together with other administrative changes (such as the SQL failover awareness and Managed Accounts) to make SharePoint 2010 a truly industry leading platform for web applications and technology. I cannot think of any other product on the market that offers this level of flexibility across so many different technology streams.

Cross Site Scripting (XSS) protection for SharePoint 2010 Web Parts

 

Some of the new features in SharePoint 2010 offer some great new opportunities for malicious scripts to be manipulated in your system. The new SharePoint 2010 Client Object Model is a great case in point.

 

Let's take the example where a contributor adds some Client Object Model scripts through exposed web Part properties to change list data that they don't have access to. As soon as someone with admin privileges visits the page that Client OM kicks off and you've got yourself malicious script executing!

 

Well, step in the new XSS protection. The WebPartPages class now includes a new attribute that you can add to your Web Part Properties called "RequiresDesignerPermissionAttribute". There is also a new SafeControl attribute called "SafeAgainstScript".

 

These allow you to protect your assemblies and properties against contributors. The main problem is that none of your MOSS 2007 web part properties will be accessible to contributors without these added!

 

This obviously creates quite an overhead in terms of code use, but it really is required to make sure that your web parts are running in an appropriately secure state.

Web Parts on SharePoint 2010 Wiki Pages.. marriage made in heaven

 

This is something that really confused me the first time I did it (by accident actually), but you can indeed drop web parts directly into Wiki Content.

 

Let me just repeat that in case you missed it:

You can drop web parts directly into the HTML of Wiki content

 

There is no concept here of web part zones, or ordering .. You can literally seamlessly have them embedded in the HTML!

 

This of course means great things for allowing dynamic page content to truly flow, with dynamic web part content sitting seamlessly side-by-side with your Wiki content (hopefully this also means the death of over-complicated Page Layouts to accommodate hundreds of Web Part zones .. And also hopefully the death of the Content Editor web Part!)

 

To add the web parts is really easy, it uses the new SharePoint 2001 "Ribbon" interface, and you just literally just insert web parts the same way you would with tables, images, or any other type of content.

 

It actually achieves this by using a hidden web part zone (called the "WP Zone") which the Wiki uses to store the web parts (and retrieve the web part properties.)

 

Now, let me just hit you with another big one: Web Parts now support content versioning.

 

Again: Web Parts will now roll-back along with page versioning! So when you restore a version of a page, the Web Part properties in that version will also work!

You don't need extra code for that, it "just works" (very very cool!)

 

How can I do this programmatically?

There are 2 different methods you can tackle for this:

 

The "WikiEditPage" class includes a method called "InsertWebPartIntoWikiPage". This is a ron-seal method (it does what is says on the tin!).

 

Alternatively you can also "roll your own".  Web Parts are identified in the Wiki HTML through a DIV placeholder with some specific GUID references. So you can hand-crank this HTML content and drop it into your wiki page.

Improving SharePoint 2010 Administration

 

This is a big area for SharePoint 2010. Far too often in the MOSS 2007 interface was administration settings a little bit neglected (and lets face it, the public facing sections of our systems always get more attention) but in SharePoint 2010 there are a number of massive improvements.

 

Logging & Alerts

One of the key areas for reporting is centralising the reporting and alerting interface. In this the main logging and event data sources (ULS logs, Windows Events, Performance counters for SQL , .Net Framework and  hardware resources) are going to get pulled together into a single SQL Database. The best bit is that this database will have a published open schema!

 

This allows the database to be queried and reported upon, and is expected to include full SCOM integration!

 

Managed Accounts

This is a HUGE feature for managing service accounts in SharePoint 2010. You can create specified accounts that can be used by farm administrators when setting up SharePoint services (such as Search and Timers) as well as creating new applications (such as Web Applications and the new Managed Service Applications which replace the SSP).

 

The upshot of this is that you don't have to hand out domain accounts just so that someone can provision a new web application.

 

But that's not it. Managed accounts can reset the password (presumably into some rediculously long strong password) and manage that password through SharePoint 2010. If you have an AD security policy for password expiry, then Managed Accounts can automatically reset the password for you, so you never have to worry about password expiry hosing some key services in your SharePoint 2010 farm!

 

Health and Monitoring

Central Admin is set to gain a whole raft of performance monitoring reports.

 

On such example is the "slowest pages" report, literally showing you the average times for the slowest pages to render.

 

You can then use the new features of the "Developer Dashboard" to show you key performance information about that page

• Which webserver was used
• What SQL queries were run
• What web parts loaded (and how long they took)
• The call-stack of code method calls
• SPRequest allocations

 

This allows detailed analysis and rapid debugging of problems that otherwise were seen by many as a black art!

 

Failover

There is now an option for a Failover Database Server when creating Web Applications and Managed Service Applications. This effectively allows SharePoint 2010 will automatically be aware of database mirroring. If the primary SQL server dies, SharePoint will automatically re-connect to the failover database.

 

Of course, you still have to setup mirroring manually, but this is a huge boon to creating high availability systems on SharePoint 2010.

 

Restoring Data

You can now recover a list from an unattached content database. From SharePoint 2010 Central Administration you can connect directly to any content database (regardless of whether it is attached or not), and you can browse the content structure from Central Admin.

 

You can navigate the structure and export any Site or List and it will download the package straight to your computer. This can also pull in versioning and security settings!

 

There is tonnes more content to cover, and loads more sessions .. But hopefully this gives you an indication of some of the improvements that are being made in SharePoint 2010!

Standards Compliant SharePoint 2010 - Too good to be true??

 

Well it seems not. Microsoft have really pulled out all of the stops this time around, with standards compliant, cross browser compatibility and even accessibility hitting the conference headlines.

 

The first item of note is that SharePoint 2010 aims to be W3C XHTML compliant. This is a massive leap towards making SharePoint 2010 a truly robust platform, worthy of both internet facing sites and global internal systems.

 

I saw a demo of an embedded video in the Monday afternoon Web Content Management  session where a video was streamed from a Document Library (another new SharePoint 2010 feature) and it was loaded in FireFox with no plugins or downloads, full screen streaming and buffering.

 

But the biggest impact is bound to be around the statement that one of the major release goals for the SharePoint 2010 release is that the entire interface of SharePoint 2010 meets WCAG 2.0 AA compliance! This includes editing and authoring of content! This is absolutely massive, and represents a huge push from Microsoft to really respond to the community feedback that accessibility is a subject to be taken seriously.

 

Of course, there is bound to be community discussions around the ideology that "compliant" is not the same as "accessible" (which we at Content and Code know all too well, having developed several systems for the RNIB) but this is still a massive step forward and certainly shows a continuing respect of standards from Microsoft and good news for the future of compliant software.

Why I love the performance in SharePoint 2010

 

There really is no doubt about it, SharePoint 2010 is QUICK... We are talking several factors faster than SharePoint 2007. Now this is mainly from looking in the product demos and session examples, but if they are a true reflection of a typical SharePoint implementation then we're really in for a treat.

 

I watched a demonstration in the opening "Overview of SharePoint 2010 for IT Professionals" an example of a  document library with 1,000,000+ documents which was filtering, sorting and refreshing in under 3 seconds.

 

Having filtered the view down to just 4 records it was then demonstrated that images could be opened from that library in under 1 second! And this is native, out of the box ... and with over 1 million documents in a single library ... running on pre-beta code!

 

This is nothing short of astonishing, and with some of the other features such as automatic column indexing, filtering setup and hierarchical navigation we are looking at an amazing set of features to make even the largest of SharePoint 2010 deployments absolutely fly!

 

Every time I look at SharePoint 2010 I get more excited about the feature sets, and it seems that Microsoft have followed the same vein with SharePoint 2010 that they did with Windows 7, in that performance and usability are two of the biggest "killer apps".

Key notes from the Keynotes - SharePoint Conference 2009

 

The keynote speeches contained a helluva lot of content, but there were some key points that were worth condensing into a post ... so errr.. thats what I did :)

 

One of the main aesthetic changes is that  Windows SharePoint Services (WSS) is now called SharePoint Foundation 2010. Hopefully more details about capabilities will surface during the conference.

 

There was much talk about SharePoint Online. They apparently have over 100,000,000 users and the platform is updated quarterly with new functionality, so we can hope to see new SharePoint 2010 functionality in that creeping through once the RTM version of SharePoint  is released.

 

The list item storage limits has gone WAY up ... 1,000,000 items per folder/list and over 10,000,000 documents per library (more about this in my next post).

 

Another favourite was how Excel Services in SharePoint 2010 allows you to expose excel data as REST feeds (such as charts, tables, images, pivot tables). This allows you to subscribe to an image URL which is actually being dynamically generated from the Excel 2010 spread sheet. The main focus here was that you can embed this image anywhere that a normal HTML or Office Client image can be placed, but if the Excel 2010 spread sheet data is modified then the image is automatically updated to reflect those changes!

 

Other highlights included forcing spelling checkers and broken link checkers on check-in of a page.  In fact there was a lot of mention around web sites full stop, particularly 2 new products for websites:

• SharePoint Server 2010 for Internet Sites
• FAST Search Server for Internet Business

 

The new Wiki Editing features were also demonstrated with auto-complete URLs for lists, views and folders in libraries.

 

For those power users there are also over 500 new PowerShell commands for SharePoint 2010 which will be shipped with the Beta version in November! These can even be run on a Windows 7 machine and executed remotely!

 

But by far the most impressive part for me was the presentation on SQL Server PowerPivot for SharePoint 2010 and SQL Server PowerPivot for Excel 2010. This is the product formerly known as "Gemini" and allows you to pull data in from SQL databases to allow up to 100,000,000 rows of data in Excel 2010! You can then filter, sort and produce charts which refresh near instantaneously! There was a demo of this in the session causing a round of applause from pretty much everyone.

 

A very interesting session in the end, with glimpses of promise from all over the platform.. I just can't wait to get my hands on it all!

Tasty facts and figures about SharePoint Conference 2009

Wow ... thats the word that pretty much summed up the team's reaction to the Keynote speeches to open the conference. I'll post about that in a bit, but to warm you up some tasty facts and figures:

 

538 onsite labour days to setup the conference

7.5 miles of CAT5e cable used (for the Wireless Network?? ;))

7400+ attendees (up over 90% on the last SharePoint conference)

160+ partners

300+ hours of brand new content

297 speakers

240 break-out sessions

(and 2 marriages!!)

 

so .. you could say they are taking this conference VERY seriously.

 

We also had some very interesting news. The Public Beta of SharePoint 2010 and Office 2010 will be launched in November!

They are expecting an RTM release in the first half of 2010.

 

Visual Studio 2010 Beta 2 was also released today! :)

The trip to Vegas

Well, it's been a REALLY long day... Around 26 hours straight without sleep. Don't get me wrong, sometimes I love being cramped into a metal tube with no leg-room and only 1 movie I want to watch for 14 hours of flying ... but what really hit hard was when they run out of beer!

 

So .. What have we been up to so far? Well, unfortunately not a lot. I met up with the group at Heathrow around 10am Saturday morning. Everyone in good spirits (apart from one of my colleagues insistence that a cavity search was imminent going through customs!)

 

All went well, and we were in the air on-time. The flight passed surprisingly quickly. My colleagues decided to use the power of Heineken to help the flight pass quicker ... unfortunately ... the plane ran out!!! (Jason did try London Pride but that did not last long). After that there was nothing for it .. Time to start on the Vodka (you can tell when you're having an impact when you ask for a drink and the stewardess says "not you lot again!").

 

11 hours later we landed in Dallas. A bit to eat. Another 3 hours (me stuck next to two guys trying to tell me about their ATM conference and the virtues of American "Football" (shudder)) and we landed in Vegas... dark, hot, and full of bright lights!

 

So the first day is over and the conference starts tomorrow!! (can't wait!)

 

Some interesting Vegas facts:

 

• If you spent 1 night in every hotel room in Vegas you would end up staying for 288 years
• The Luxor atrium (the hotel we are staying in) is the largest in the world
• Over 60,000 people take up residence in Vegas every year, making it the fastest growing city in the USA.

 

 

Another post tomorrow... right now it's 9am and I'm heading for breakfast! (all you can eat! HELL yeh!)

Visual Studio 2010 - Automatic creation of SharePoint WSPs for deployment (joy!)

It's long been a challenge to generate WSP packages effectively.

 

Back in the old days we would manually hand-crank out DDF files and manifest.xml and use makecab utilities to generate them. There was the most short-lived of joy when everyone rushed to download VSeWSS (Visual Studio extensions for Windows SharePoint Services) ... and then made an equally mad rush to un-install it when they realised how bad it was! Finally most people settled on 3rd party tools like WSP Builder, but even then quietly grumbled about it's limitations and bugs.

 

Well .. the time is here to rejoice once more .. Visual Studio 2010 to the rescue! (queue golden lights, music and champagne).

 

So what does Visual Studio 2010 bring to the table then?

 

Well, the most interesting thing to note is that VS10 (Visual Studio 2010) has baked-in support for SharePoint 2010. This includes a whole raft of project templates; Web Parts, Site Definitions, Workflows, Business Continuity Services (a.k.a. BDC.. and very interestingly an "import workflow from SharePoint Designer" option!).

 

All of these include baked in WSP support in the form of a new "package" folder in the project structure. This is the bit responsible for turning your Visual Studio solution and re-packaging it up into a WSP (no 3rd party tools, no code, no complicated build scripts!)

 

The really cool bit is in the deploy / undeploy options:

• Create WSP file
• Reset IIS app pools
• Add / Retract solution from SharePoint
• Activate features
• Run Pre/Post build commands
• <Custom Actions>

This allows you to customise these actions (so that they execute in any order you like!) and the ability to add your own Custom Actions makes this VERY exciting!

 

You can also modify the manifest.xml template that it uses (adding your own custom markup if you want!) and specify any external assemblies that should be included, even specifying if those other assemblies should be GAC or bin deployed!

 

Finally, the last one of note is the ability to import an existing WSP and convert it to a Visual Studio Project. The implications for this are massive! Any older projects or 3rd party WSPs can now be migrated over to the new VS10 structure. Of course custom code will not be migrated in, but it is a cracking effort and goes a long way to making this more manageable (especially for "code free" CAML based features!!)

 

All of the above is covered in the online free sneak-peek sessions, so if you haven't already, go watch the SharePoint 2010 Sneak Peek video for Developers now!

http://sharepoint.microsoft.com/2010/Sneak_Peek/Pages/Developer-Video.aspx

Visual Studio 2010 and TFS connection: "The ServicePointManager does not support proxies with the https scheme."

Using the beta version of Visual Studio 2010, and trying to connect to a Team Foundation Server, you may find that you get the error message:

 

"The ServicePointManager does not support proxies with the https scheme."

 

You'll only get this of course if you are trying to connect using HTTPS / port 443. The workaround seems to be some reg fixes which tell the TFS client to bypass the proxy.

 

Aaron Block's MSDN blog seems to have the answer to this:

http://blogs.msdn.com/ablock/archive/2009/05/20/for-tfs-2010-beta-1-resolving-tf31001-the-servicepointmanager-does-not-support-proxies-with-the-https-scheme.aspx

SharePoint Tabs Made Easy

While browsing EndUserSharePoint found a good post about using a JQuery script embedded into a CEWP.

From my brief look this basically turned every web part on the page into tabs so that you can create VERY quick and easy tabbed dashboard interfaces. Extremely quick, extremely easy ... the best bit is no code, no development.

You can check out the article (and the video) here: http://www.endusersharepoint.com/2009/10/05/yes-believe-it-the-sharepoint-easy-tabs-interface-web-part/

The History of SharePoint

A really nice article posted up by the Microsoft SharePoint Team Blog:

As we are just two weeks away from disclosing SharePoint 2010 at the SharePoint Conference starting October 19^th, I wanted to write three posts to provide context on the upcoming release. This first post will cover the history of SharePoint. I hope it will provide some useful perspective behind our vision and what we have learned as well as a few fun anecdotes. The second post will cover the engineering process for SharePoint 2010 - how we design and develop SharePoint in the Office team, what new approaches we have taken during the 2010 development cycle and my take on a few frequent questions I hear from customers and partners. The third post will coincide with the opening of SharePoint Conference and cover the major feature investments. After that, our team will star blogging in depth about the new SharePoint capabilities. For folks who cannot wait, we have highlighted a few of the new features on the SharePoint 2010 Preview Site as commented on several Office 2010 client capabilities including a few points of SharePoint integration on the Office Engineering Blog.

Before we jump in to this post, I want to thank everyone who has been with us on the journey so far and is supporting us in the release of SharePoint 2010 and feedback for releases beyond. While I will talk about the efforts that evolved into SharePoint, its origin dates to around March 1998 when we first started planning the projects that led to SharePoint. Since the first beta of the first release, we have been extremely fortunate to have your support and hear your feedback. It has been extremely rewarding to see all things you are doing with SharePoint. We have posters of many of your sites on the walls of Building 16 on the Microsoft Redmond Campus and these are a source of great motivation and pride for our team to take things to the next level. We always appreciated your ideas and feedback friendly or otherwise about where the product should go or how we could help you better. We think SharePoint 2010 will be another big step forward but we know there is more to be done and are thankful you care enough to keep pushing us. Finally, we want to thank the Microsoft teams around the world who has worked so hard to build and support the product with special thanks to a few dozen people who were with us from the very beginning in 1998. We are very fortunate to have a great team, customers and partners and that is what keeps us fired up to come to work every day.

to read more go to:

http://blogs.msdn.com/sharepoint/archive/2009/10/05/sharepoint-history.aspx

SAS (SharePoint Accessibility Solution) - WCAG 2.0 AAA reusable framework for SharePoint

I am proud to finally announce the official release of SAS (SharePoint Accessibility Solution) by Content and Code.

 

The release of an accessible framework for SharePoint is a massive step for both SharePoint, Content and Code any organisations looking for an accessible SharePoint platform.

 

This framework was used (and developed) for the RNIB Website (which I posted about a few weeks back: RNIB - World's first AAA website launched in SharePoint).

"Accessibility is mandatory for many organisations, allowing access to the web for all. Our SharePoint 2007 Accessibility Framework allows sites to be built rapidly without sacrificing SharePoint functionality."

- Tim Wallis, CEO, Content and Code

By all means check it out and tell me what you think :)

http://www.contentandcode.com/solutions/Pages/accessibility.aspx