In this article you will learn how fields in SharePoint can be encoded from a display name to an internal field name. The problem with the difference between internal field names and display name is that, SharePoint won’t have any built in function that is available to use in your code. If field names are only defined in English you might not have such problem. My mother tongue is German and we have some special characters that needs to be special treated and encoded.

The basic

Lets’s create a field called “Project Number”. The internal name for this field then will be “Project_x0020_Number”. The “_x0020_” in the field name is the urlencoded representation of the space in between of both words. To be more precise it is a form of unicode url encoding. The Unicode encoding of the space is “%u0020” but for fields “_x0020_” is required. To transform the encoded field name to a proper SharePoint internal field name we need to transform the encoded space somehow.

Another point that need to be considered is that SharePoint 2010 has a maxium length for internal field names is limited by 32 characters. By encoding the string we will get a much more longer string than the display name is. SharePoint handles this by truncating the field name and if multiple field names will be produced an additional index will be added as character number 33.

The good news is that this has been improved in SharePoint 2013. Now you can use display names with a length up to 255 characters. The limitation of the internal field names doesn’t exists anymoure. It seems that the limitation of the internal name is now raised to 1024 characters. In some test cases I got fields with an internal name length of more than 600 characters.

From display name to internal field name in C#

As mentioned before somehow SharePoint doesn’t have an out of the box function to convert a field name from the display field name to an internal field name. At least as far as I know.

There is one method that works for field names in English, but it doesn’t work for non ASCII characters and punctuation marks. The method which was recommended by Hugh Wood (@HughAJWood) is to use XmlConvert.EncodeName for encoding and XmlConvert.DecodeName for decoding the field names. What I have experienced with this is that it cannot considered to be a save approach to encode the display name strings. To give you an example “Österreich”, the german name of the country where I live in, will be encoded to “Österreich” but a field with this label in SharePoint will be encoded as “_x00d6_sterreich”. This is because the letter “Ö” is valid UTF-8 character to be used in a XML but somehow it’s not save enough for SharePoint.

My approach is a little bit different, might not the fastest, but save in a manner of SharePoint. So what I do to create a valid encoding, is to encode every single character of the display name using the method HttpUtility.UrlEncodeUnicode.

I tested this helper with the Japanese word for Austria, which is written like “オーストリア”. In this case the XmlConvert. EncodeName method will fail too, but mine do the encoding right. I mean I cannot read this Japanese word but I think you will get the point here. The only thing the helper not support is the truncation of the field names but I think this won’t be that hard to do.

Decoding the field name is pretty much straight forward all that needs to be done is to replace all “_x” with “%u”, remove all remaining underscores and use HttpUtility.UrlDecode and here is the code.

From display name to internal field name using Javascript

If you use SPServices or the Javascript Client Object model, then you might need the same functionality that I previously presented in C# in Javascript. The encoding and decoding can be used in CAML Queries for example. This javascript encoder and decoder follow the sam pattern at the c# methods. I do the same, encode every single character by using the “escape” function for encoding and “unescape” for decoding. The finished code looks like this.

To use this you just need to call the functions. In the code you also should consider the length of the the field names of SharePoint.


In SharePoint you cannot prevent that end user will create field names with special characters. I also think that it is not useful to deploy any column by using code or the declarative approach. One big advantage of SharePoint is that the end user is able to customize the platform to their needs, without asking a developer.

Both approaches, JavaScript and C#, show how to handle the SharePoint encoded field names, no matter if you use SharePoint 2007, SharePoint 2010 or SharePoint 2013. I also think that the JavaScript solution will become more and more important in future implementations. One reason is the new JavaScript API available in SharePoint 2013 or one of my all time favorite SPServices, if you like to access SharePoint via JavaScript and Web Services.

At the end I like to thank Marc D. Anderson (@sympmarc), James Love (@jimmywim) and Hugh Wood (@HughAJWood) for a great discussion on twitter prior this blog post.

If you have any enhancements to the provided code or like to give me a general feedback, please feel free to add a comment.

In this article I like to show you my way to deploy lookup fields to SharePoint declarative. I need to note that I won’t do this fully declarative but found a solution that works best. This will be used by SharePoint internally and is fully supported. The lookups I’m talking are lookup fields deployed as site column.

Whenever it comes to content types and site column in my opinion the best way is to create them declarative. The reason for this is that I can handle that all deployed fields have the same GUID that I defined and every Site Collection has the same fields with the same IDs. Simple field types such as text, number, Date and Time, User Fields, Yes/No Fields can be created easily using the declarative approach but with lookup fields this can be a little bit trickier. During the last weeks I did a couple of interviews with Becky Bertram, Doug Hemminger and made a poll on SPYAM. The result of this was that most people deploy or create lookup fields using code and not using the declarative approach. Here is the result of the poll.

SPYam Poll for creating lookups

SPYam Poll on how creating lookups as web site columns

But now let’s take a look how to create a lookup field.

Creating the Lookup Field

The basic declaration of a lookup field looks simple and you can reference the List by using the URL of the web where the site column will be deployed to. This can be done according to the documentation as long as the list will be deployed in the same feature as the lookup column.

In a second module I created a basic list instance with a custom list.

Both elements will be deployed in the same feature which means that the reference of the lookup should match the list and replace the token List URL with the value for “Lists/Colors” with the GUID. The problem is that after the deployment the lookup column looks like this.

Lookup field after declarative deployment

Lookup field after declarative deployment

If you go to the definition of the lookup column the list is empty and won’t be displayed correctly and the field won’t work with lists. The empty “Get information from” is one reason that people create lookup fields using code. As always in SharePoint many ways lead to a solution and a lot of people have created their own provisioning classes to create site column lookup fields, but here comes the good news. This problem can be fixed some simple modifications of the field configuration using code.

Fixing declarative Lookup Field

From my experience the problem is that SharePoint won’t replace the List URL token with the GUID of the list and the web. When you like to use site columns as lookup fields than you have to get those GUID somehow to the schema of the fields. The URL token in Site columns cannot be used because site columns need to have a reference to the GUID of a list and a web. Then SharePoint will be able to find the matching list and web where the lookup should reference to. A sub site cannot make use of a list with the value “List/Colors”.

A quick solution would be to create the list first and the web first and add the GUID in the field definition. Works great as long as you don’t like to reuse the lookup field in another site collection. In this case you have to change the GUID when you like to deploy the solution to another web application or sites. Not really a great option.

My solution is to add the GUIDs to the schema of the field definition and use the List URL as a token to find the proper list. This can be done with a simple feature receiver. The code for this looks like this.

The code above does the following tasks:

  • Get the Field
  • Get the list that should be referenced by the lookup
  • Change SourceID and List to the GUID of the web and list.

Without using the feature event receiver the field looked this:

After the deployment with the feature event receiver the field looked like this:

SharePoint incremented the version number of the field after the changes have been applied to. The manipulation of the schema using the object model is fully supported by SharePoint but you need to take care that your modification are set with a proper xml that matches the field schema definitions. You will find this comment in the SPField.SchemaXML on the MSDN. There is not much difference in setting the schema of a field or create a field directly from XML using AddFieldAsXML.

A second look to the column settings shows that everything will be displayed correctly.

Lookup field after modified schema

Lookup field after modified schema using feature receiver

If you like to try this yourself you can download the solution N8D.Lookup.

Some other recommended Information on lookup fields I used previously was.

Since SharePoint 2010 was introduced there are much more ways to customize forms for list and document libraries. In most cases there is only the simple requirement to hide fields or have better control over them on lists and content types. SharePoint 2010 is the same way limited for hiding fields like it was in MOSS 2007. For advance control it was always necessary to code those fields. So first let’s take a look at how you can controls fields on the list using the normal settings dialog. The base setup in this test is a custom list called “List” with a new column called “NewColumn” which is of type single line of text.

There are no settings available where you can control where the fields should be appearing. The options to hide a field form certain forms are even not available in SharePoint. So the only way to remove the fields from NewFormDialog for example will be to set up a custom list form. No not at this stage.

When the management of content type is enabled to the list using some advance settings will come up to the fields. This can be done in the list settings:

  • “Advanced settings”
  • “Allow management of content types”
  • “Yes”

Behind ever list and library there is a content type assigned. For my custom lists this is the “Item” content type. After selecting this content type there is the “NewColumn” I created available too. On the column the following view options can be selected.

Required optional or hidden is good but won’t help in most cases. So what now? Create a custom list form? No, this is not needed.

SharePoint developer might find SPField Object from the server object model. So if a developer creates a custom content type the following options can be selected in CAML Definitions or as properties using object model.

  • ShowInDisplayForm
  • ShowInEditForm
  • ShowInListSettings
  • ShowInNewForm
  • ShowInVersionHistory
  • ShowInViewForms

I think the names of those properties are self-explaining. Is there coding needed to use those properties? Yes it is but not as a WSP. This field can set using PowerShell the following example demonstrates how to set those properties by a simple script which only opens the list using object model and sets the values, which are basically Boolean values. I use simply 0 and 1 values to set.

0         enables this property

1         disable this property

So for hiding our column in the new form dialog we just simply need to set the value of ShowInNewForm to false or 0.

So how does it look after executing the script:

NewColumn Hidden

NewColumn is hidden on NewForm but still avaliable on EditForm.


I actually can’t say what the reason is behind those properties are available using object model but not be available using the interface or SharePoint Designer. Those properties will give power user better control over their fields and will not require custom list forms using SharePoint Designer or InfoPath Forms for every simple field hiding. Those properties are not new they were still available in Moss 2007. Might this will be supported by SharePoint 2013 or 2014.