Aggregating across SharePoint Site Collections
Guest Author: Brett Lonsdale
Sometimes with SharePoint you will find yourself in a Chicken and Egg situation! Many SharePoint specialists consider it a best practice to use many site collections in their environments than to have one big site collection. Each site collection can be assigned its own content database and therefore makes backup and restoration of SharePoint data more efficient than try to back up one huge database. This can however cause you other problems such as isolated content types, and difficulty in rolling up data from lists and libraries that are distributed through multiple site collections.
Fortunately Lightning Tools helps with this trade off by allowing the aggregation of List or Library data from across multiple site collections. The Content Query Web Part that ships Out-of-the-Box with MOSS is only able to aggregate list or library data from within a site collection even though it does that very well and very efficiently.
The Lightning Conductor Web Part is configured entirely through the toolpane with no requirement for any custom XSL work at all. Simply all you have to do is configure your data source, the lists or libraries you would like to aggregate from, filter the results, and define the way you would like to present the rolled up information. You can also choose columns to display simply by selecting check boxes for each column including custom columns.
Below you can see the options for selecting the data source which includes All Site Collections (aggregated data will continue to grow as new sites are created), Multiple Site Collections allow you to select which Site Collections to aggregate from, Specific Sites allowing you to select predefined sites from multiple site collections, or Specific lists allowing you to select predefined lists or libraries from across multiple site collections.

Figure 1 – The Data Source section of the Lightning Conductor Web Part
As soon as you have defined the source, you are able to select the List or Library type along with the Content Type. The List type will show all available List types within your selected site collections including custom list templates or custom list definitions. You can optionally further refine the results by choosing the Content Type group and Content Type of the list items you would like to aggregate. Again this will include any custom content types.

Figure 2 – Selecting the List Type and Content Type from the Lightning Conductor Web Part
It is still possible to filter the returned list items further by setting filters. You can filter data using filters set in the toolpane by the Site Administrator or allow users to set their own filters using the Quick Filter toolbar. The Lightning Conductor Web Part is also able to get filter information from other web parts such as the Current User Filter Web Part which is available in MOSS.

Figure 3 – Filters set from within the Web Part Tool Pane or in the Web Part itself.
The final stage of configuring the web part is to configure the way you wish to present your data. You have three options to choose from; 1. The Default View option will display the results using the exact same configuration from the List or Library default view itself, 2. Configure an SPGridView which allows you to select the columns for display, set grouping, linked fields, column order etc, or 3. Completely define your own look and feel by writing a custom XSL file. Below is the SPGridView configurator which demonstrates columns being selected for display.

Figure 4 – The SPGridView configurator allowing the columns to be selected for display
After configuring these options, you can see the results displayed below. Other properties are also available such as conditional formatting, Pagination, Caching etc.

Figure 5 – The completed Lightning Conductor Web Part
For anyone wishing to write their own custom XSL files, the below example shows some useful examples of the types of thing you can achieve with this web part. Such examples are available for download from http://www.lightningtools.com/pages/lightning-conductor-web-part.aspx

Figure 6 – The Lightning Conductor Web Part configured with a Custom XSL file
Brett Lonsdale from Lightning Tools will be hosting a free webinar on the 24th September 2009 at 11:00am Eastern. Please email [email protected] for further information. If you wish to download a free trial of the web part, you can download that from http://www.lightningtools.com/pages/lightning-conductor-web-part.aspx.
Guest Author: Brett Lonsdale
Brett Lonsdale is a SharePoint developer who specializes within the Business Data Catalog, co-owner of Lightning Tools Ltd, Co-host on The SharePoint Pod Show www.sharepointpodshow.com.. Strategically (kind of) Brett has based himself in Florida where he lives with his wife and daughter. You can read Brett’s blog on www.brettlonsdale.com, and also follow him on twitter @brettlonsdale
Hi Brett,
I like that the tool seems to give a lot of power to end users. But if necessary a dev can still come in and customise with XSLT.
Presumably querying across entire site collections can be time consuming. How does the web part handle this? Does it perform some sort of caching?
Thanks,
Alex.
most cross site aggregators I have seen get slow when you add in too many data sources that are too far away. I like the idea of being able to use XSLT to customize the layout. CorasWorks is more powerful but does not allow this type of control.
HI Brett,
I have tried your web part and i have a question.
I have 2 site collections. One Site Collection is using IT Template which comes with sharepoint. I have a List called “Service Requests” in this Site Collection. in this List i have 2 views called Open SR’s(group by resolution type and collapsed b default) and second view called Closed SR’s(group by resolution type and collapsed b default).
Now in my second site collection i am using your web part to retrieve data from my first site collection. I added 2 Lightingtools webparts and i can retrieve both open and closed SR’s. But problem comes when i want to expand the treeview of my open or closed SR’s. It just says “Loading…”.
second problem i observed was :
I made both the views expanded by default and came to the page in second site collection where i added the web parts. First was open SR and second was closed SR webpart. Here the problem starts.When i tried to collpase the second web part, it was collapsing the first web parts (open SR’s).
Do yo have thoughts on this.
I appreciate your help.
Thanks, Ram.
The fact that tools like this are a de facto requirement for using Sharepoint in an Enterprise depresses me. One of Microsoft’s marketing ploys for Sharepoint is that it allows you to leverage your documents and data across the whole enterprise. However, once you installed Sharepoint you quickly discover that if you create multiple site collections for whatever reason, you have reenforced the silos within your organisation because your data is trapped within those Site Collections which likely reflect organisational structure. The solutions. Spend another $800 on a tool that will enable you to do what you paid already for in Sharepoint. Or rebuild all your team sites under the same Site Collection – which kind of begs the question, why did Microsoft create Site Collections?
The lesson I have learnt from this is not to trust Microsoft’s marketing. Sharepoint 2010 sounds fabulous, but I now expect to repeatedly be disappointed by functionality gaps such as this one.