SharePoint: Convert Folder Structures to Metadata
Author: Kerri Abraham
Prior to Sharepoint I had never heard of a shared drive or even seen a folder structure so I took to metadata (column tagging) organization right from the start. I love a good filtering, sorting, grouping challenge and folders just don’t fit my way of thinking, so I can proudly say I’ve never used them. I have however inherited a few libraries with folder structures and have cursed them from the outset. It wasn’t until a recent automation of a process with KnowledgeLake capture that I actually had a requirement to restructure one of these libraries.
This library holds nearly 2000 documents. It had been created to store scanned insurance remits for staff to access as they worked denied claims. The folders had been organized by year, but a few other folders were labeled with payer information in addition to the year. Clearly I had two column/metadata requirements of ‘Year’ and ‘Payer’. It is important to take a thorough inventory of the information prior to getting started so that you don’t have to go back and add columns resulting in rework.

Add the columns to capture the metadata under Settings. Keep in mind that the most dependable way to get consistent grouping or filtering results is by limiting the options available with a choice column. If the folder structure is extensive, an alternative would be to keep the fields as text columns, free text in the metadata and then later change the field to a choice column, which will result in the same end-user consistency for future additions. Once the columns are added, it is a matter of opening each folder and adding the appropriate metadata. Since the documents are already grouped within the folders (i.e. will need the same metadata tagging) editing the properties can be done very quickly using Edit in Datasheet.

The Datasheet view is similar to an Excel sheet. Enter the metadata on the first line item and then highlight the entire column (Shift + left click). Right click and choose Fill >Fill Down. Fill Down options will fill the entire column with the value in the first highlighted row. This process can be used to fill multiple columns at the same time.

Once the metadata is applied to each document go back to Standard view under Actions. Be prepared to wait just bit…

Repeat the process to get all the documents within the folders assigned metadata. When all the documents are tagged (this really is a much faster process than you might expect using the above method) check out the results:

Ah, man! Still in folders?! Not to worry, now that the metadata is assigned, the fun part is figuring out how to optimally display the information. Either create a new view, or use the existing view and choose to display the items without folders.

The only thing left to do is to modify the view to filter, sort, and group the information in a way that makes the most sense. Consider the size of the files, and how to reduce load time when creating the views. ‘Group By’ mimics a folder structure, but using metadata instead of folders allows more detailed sorting and filtering. Showing the grouped view in default of ‘Collapsed’ or limiting the ‘Item Limit’ to display only the most relevant items aids in faster load times.

And the results:

All right! Our documents aren’t in folders anymore…. Or are they? Let’s take a look at that URL:

I share Laura Roger’s pet peeve of ugly URLs, and if there were no other reason to avoid using folders, that right there should be reason number 1! The folder name becomes part of every document’s URL, so when the nested folders become 2, 3, or 4 levels (shudder) those URLs start to expand until we find you on Stump the Panel asking how to increase the 255 character URL length limit. That answer is easy “Stop using folders!” Don’t fret, even this can be straightened out with very little pain, and in the process you’ll be proud to know that in the eyes of the experts, you are growing in Sharepoint maturity!
Using the Explorer view of the library, this is an easy fix. Start by right clicking the folder, and choose Explore. This is a cool feature, as it opens a window to view of the documents within the folder. Then simply highlight the documents and right click and cut ( Ctrl+X), and then click back into the Explorer view of the library to paste (Ctrl+V) them back in, metadata and all.

When you check your URL this time, that folder name is gone!

Now, what does that do to any existing links within Sharepoint? Most of the time nothing, somehow it knows how to find that document without the folder path in the URL. I find that Explorer view is fabulous for keeping URLs active even through a process like renaming. Just to be safe, if your documents are linked to multiple sources, or, as in my case the file size is prohibitive, you may have to be content just displaying them without folders, and leave the URL alone knowing that all new additions to the library will no longer have ugly URLs.
Now you have gone to the trouble of getting rid of folders, keep it that way. Use Settings to keep end-users from being able to easily create them. The below options are found under Advanced Settings.

Okay, I admit, I might have overstated in the first paragraph, I have used folders within Sharepoint. My scripting library comes to mind; but filtering in columns with metadata is far and away my preferred method. Information structuring is unique to each kind of information requirement including capture, use, and storage. If you find yourself on the fence as to what direction to take, below are a few articles that outline the pros and cons to each method, all three of excellent educational merit.
SharePoint Folders vs. Metadata by Eugene Rosenfeld
Folders and Metadata Rehashed by Eric Alexander
Folders bad, metadata good! By Bil Simser
Author: Kerri Abraham
Kerri Abraham is the Revenue Cycle Sharepoint Coordinator at Mercy Medical Center in Cedar Rapids, Iowa. Converting Sharepoint enthusiasts one information worker at a time, she is leading the charge to win end-user adoption through the creation of extensive training materials and knowledge capture. Introduced to Sharepoint in 2006, but holding a dedicated Sharepoint support role since 2008, and brand new to tools like Sharepoint Designer, she focuses entirely on out-of-the-box solutions.
Really helpful and informative article !!!
Thanks a lot for sharing.
Great work, Kerri!
I’ve not worked with folders, either, though some of my users have. It will be great to take this information back to them as a presentation for how to manage their libraries (and URLs) better with metadata.
I must confess that I’ve heard the “Folders Bad, Metadata Good” song playing in the background for some time, but have tuned it out while I worked on other things. For the first time, the chorus is starting to resonate. Thanks for penning this little ditty; it makes me want to join the dance.
Blessings,
Jim Bob
I’d have to disagree that folders are inherently bad. There are scenarios when folders become extremely useful. Security trimming docs come to mind, also controlling what you download into Outlook is extremely useful also. Folders become crucial when using explorer view to purge too. The metadata offers no help in this case.
The one major riff I have when I hear not to use folders is the fact that you can create your own content type and manage metadata tied to the folders. So in the end you get the best of both worlds. I feel inspired to write a rebuttal, but I can’t promise anything.
Matt,
pay attention that an heavy use of folders make Sharepoint like a file system on company servers.
A good trade-off between folders and metadata must be achieved.
Rosario Rizzo
That’s pretty much the same thing I was getting at. My comment is really to provoke thoughts about folders and why they are useful. By not ever using them, you miss out on a lot of potential solutions, especially when dealing with large sets of documents. An interesting comment in this thread is about metadata and how users’ only want to fill out no more than 6 columns to tag that document. Well, if you had metadata attached to your folders, e.g. 6 more columns, then essentially you have 12 columns/layers to slice and dice your data… Folders will always have a purpose in SharePoint, whether you use them or not is your preference. This article should state why you should convert folders to metadata instead of saying *Not to use them*…
Hi Matt, I don’t believe that I stated anywhere in the article that I felt folders are bad (except maybe for the ugly URLs they leave in their wake), I do state they don’t fit my way of thinking, and to this date, with the exception of my scripting library, I haven’t had a requirement to use them. Of course there are added benefits if considering security trimming that folders allow that metadata alone can’t achieve, and I also recognize there are even ways to apply metadata to folders.
However, in my experience I see folders being created for no other reason than for comfort, as Veronique can attest I’m sure. I am certain we have plenty of readers that are maturing in their Sharepoint thinking and recognizing that their first attempts at migrating shared folder information into Sharepoint was a bit messy, this article just outlines how to clean it up. If you are using Sharepoint for security, you know what you are doing – you don’t need my article!
I continually run into document libraries (named just that) with folder structures holding unrelated information, where I can’t easily create links to views that are pertinent to my end-user. I find that very frustrating, and it is seriously limiting collaboration in some instances… so in that way, folders are working against the very principles of Sharepoint thinking. For EUSP readers that encounter that same irritating structure, this articles simply outlines how to restructure, never did I say “You SHOULD…. do anything.”
I included 3 links for further education, and encouraged readers to read them, the first article listing all the benefits of folders, which I listed first for that very purpose, why reiterate what an expert has already written so well? Again, I would point readers back to those resources.
I think it is important to consider current practice when looking at using folders and understand what the benefits and drawbacks are — and I think more often than not, we have end-users jumping into folder structures in lieu of metadata because they just don’t know any better. I just want to raise awareness – and everyone’s comments here have helped do just that – so thanks!
Hi Kerri,
Overall I think this article has some sound advice for ‘migrating’ from folder structures to metadata libraries. However, the overriding theme here seems to be saying “folders are evil! Don’t use them!”. I disagree with this:
My personal opinion is that a balance between folders and metadata is the most effective way to use document libraries. In terms of flexibility, metadata is the stronger option. However, it can be difficult to encourage end users to make use of metadata when they have had decades of windows explorer folders ingrained into their way of operating.
Folders are the only option if you want to set altered security access on different items (setting it at item level is, in my experience, bad, bad practice) and they can also improve performance in large libraries (2000+ documents). The URL thing only becomes an issue about 4-5 folders deep (unless you have very long folder names) and often it can be difficult to sort out some items just based on their metadata without having to add a new field – meaning you have to apply some metadata to all the documents in the library for this new field. This could also present the issue that it could be one field too many for users adding files to the library. Six is (apparently) the maximum number of metadata fields your average person is willing to fill out when adding documents to a library like this (and a similar principle should, in an ideal world, be applied to choices within the metadata columns).
So yes… I think it is wrong to dismiss folders out of hand – they definitely have their uses!
Great article Kerri. Like others have said, folders do have their merits at times, they are key to managing item level permissions like Matt mentioned as well as for high volume libraries where you are constantly skirting the 2000 items per view boundary.
While a pure metadata structure is ideal, sometimes folders are a necessity.
A subject that comes up daily in our world.
Everyone commenting has a point about defending folders. I must admit, I’m a metadata convert and just love it. While there certainly are cases where there are thousands of docs, there are conversely thousands of projects that don’t have that many where metadata can be a dream. But breaking permissions on an item level, agree with Nathan, hell no.
It’s a looooooong change management, hand holding exercise getting some users to adopt this new way of working. Some go for it, some just refuse point blank.
Funny as it may sound, you know what we did to convince one particular team? They were also hell bent determined to use subfolders. All we did was name each metadata tag with “Folder” in the front so that when the views were displayed, it showed up as Folder : BRD’s; Folder : Project Charters; Folder : Sign Off Sheets. Then they were happy and kept the system. Sometimes you need to think creatively to get the buy-in. :-)
I published a simple How To for beginners for public use some time ago, feel free to use it :
http://www.letscollaborate.co.za/How%2520To%2520-%2520Replace%2520Subfolders%2520with%2520Metadata%2520v1.1.pdf
Veronique, I would love to see your how-to, but it appears the link is dead (at least for me) is there another you can add?
I love the ‘folder’ metadata trick, that is classic! I may have to use that in the future. Thanks!
Veronique, you have the most lovely documents, thanks for the new link! I have visited your site often, lots of great information there. I especially like your tips and tricks.
Hi Kerry,
great post!
I use metadata views to manage documents of several little projects that not justify a dedicated project site. The possibility in Sharepoint to permit the users to create own views with the filter, sorting and grouping power make the rest: a flexible and practical way to view and search documents.
My advice is to create coding view (a sort of look-up tables) that codify the metadata in one place and not permit the users to insert own values.
Ciao
Rosario Rizzo
I have used this method as well, (we do not like folders), and it works fine. Now I always set the option to “No” in display new folders in new libraries as I create them.
Kerri, I can certainly attest to what you say. I see end users every day that are coming round to metadata and they are just loving it. It’s about educating them gently, baby steps and constant advice when they need it.
I don’t know what’s wrong with that link, but here’s the link the whole page, see if that doesn’t help :
http://www.letscollaborate.co.za/downloads.html
Please will you email me, I’d like to stay in touch with you. My email address is on my website under Find Me.
Hi Kerri -
This really IS a great post! I am definitely in the metada camp but I am now being challenged by a large document library (2,523 folders) with a 3-level folder structure. (I will be using indices andf filtered views to limit the number of documents the user can work with at one time.)
I need to add two columns to this document library for metadata and would like to know if there is a way to auto-populate the metadata columns for each document based on their respective folder names? An iterative SPD workflow seems unwieldy and I probably will need to resort to .NET coding but I may be missing something?
I pinged twitter and it seems that it should be doable with Powershell if you have server access. It has the capailities to query lists and loop through results. So you should be able to script something up that could query the items, find the folder values and set a metadata value.
I pinged twitter and it seems that it should be doable with Powershell if you have server access. It has the capailities to query lists and loop through results. So you should be able to script something up that could query the items, find the folder values and set a metadata value.
Thank you so much Eric!
- Tracey
Yes! Many thanks Eric!
People considering SharePoint, something to beware – if you have lots of documents to migrate. In MOSS 2007 using the datasheet view is not 100% reliable. We’ve found that many times SharePoint doesn’t allow you to do the drag-copy approach to entering metadata. It gives you an error. It means that you have to enter metadata using the one at a time method. Has meant a lot of compromise in the metadata we introduce.
Hi Paul, I have experienced that same issue. While I can’t pinpoint the exact cause every time, most recently I have found that if one of the documents in the list is checked out, or saved to a different format (.doc vs .docx or even .pdf) that it won’t let me fill all the way down. In those cases I have still been able to fill most of the metadata though, just by working in chunks, 10-15 at a time instead of filling the complete list. I have only encountered this problem with migrating documents too, not when the documents were native to that library to start with.
Good point, Paul. If you have time, I would appreciate an article on your migration and the things that went wrong, plus how you had to end up fixing them. Would be really helpful to many people. — Mark
So, this is great if the folder structure only has a folder and a subfolder level…what does one do with a DocLib with a very deep hiearchy of folders? Like 3 and 4 deep? I have a user who wants to move their whole Shared Drive onto SharePoint (which we encourage) but, it’s a HUGE Folder monstrosity. Any good strategies for organizing that type of migration?
Forgot to hit the reply notification…
First of all, are you talking about an existing document library or a migration? Either way, I think you need to analyze your documents and see if they relate closely enough to be housed in one library. Probably they should be broken up. I imagine that will clear up some of your top level folder issues. Then it is matter of inventory. What do you have in there, how granular do you want to be able to filter/sort? Layout your columns and start applying metadata. It will probably be a big job, but your user will gain a great deal of valuable insight into filtering and sorting as they work through it. It can be done in stages too, no need to exhaust them with the effort. Hopefully your end-user is a clear thinking, organized type who will grasp the concept quickly.
I feel your pain Chris, I have a similar situation I am addressing. Unfortunately mine is probably much worse. As an example I have a group that has one folder out on the file server that contains what they call their “Top Level” folder structure which consists of 19 folders. Another has 60 “top level” folders with another 1500 folders and 18,000 files under those.
Doesn’t sound too bad until you take a closer look and realize that under those “top” 19 there are an additional 560 folder and 13,000 files. And this is one of the smaller examples.
My plan at this point takes a number of approaches:
1. Does all that content really belong in SharePoint or is it “archival” type data (any doc over x number of years old). Because of database limits set on us we do not encourgae moving everything into SharePoint.
2. How much of that data is duplication. This one will be interesteing. I’m figuring probably 30% of the content is multiple versions of documents (note that this is a very conservative estimate).
3. Is using content types to categorize, group and filter a viable solution?
Still in the very early stages of this “little” effort.
We are looking at DocAves File Share Connector to see if it is a workable solution for accessing files on the file server and treating them as if they are in SharePoint. Initial testing went fairly well but the powers that be have to deccide on whether to spend the bucks or not.