Programming, Policitcs, and uhhh Pineapples.
# Friday, February 26, 2010

SharePoint 2010 Mobile Rendering (And Device Emulator Fun!)

Friday, February 26, 2010 5:11:57 PM UTC

I was recently tasked with taking an exploratory look at the mobile rendering capabilities of SharePoint 2010 to get a feel for it.  The short of it is that it's sad to say that in 2010 (the year, that is), we are still doing not much better than 3-4 years ago.  Of course, this raises the question of whether it's an issue with the browser or an issue with the software (SharePoint 2010).  Based on my early experience, I have to say it's an issue with 2010 - it seems like the SharePoint team simply took the easy way out.

If you've ever used the Gmail web page from a mobile browser (like Opera), you'll see that there is a "mobile" version and a "basic HTML" version available (links on the bottom).  For all intents and purposes, the "basic HTML" version looks and works pretty much like the full web version (minus the fancy drag/drop stuff, chat, etc.) and loads fine on my nearly two year old HTC Touch Pro.  This is what we should expect from a company as big as Microsoft with vast resources at their disposal.  This is what we should expect from them given that it's 2010 and the mobile space is an increasingly bigger piece of the pie. 

What's worse is that there's no apparent way from the browser client to access the "full view".  What happens when more capable mobile browsers are deployed that can match the capabilities of the current desktop browsers?  Seems like you'll have to make configuration changes on the server to enable proper browser detection but it would be much easier to simply have the option, like on mobile Amazon.com, to load the full view in lieu of Microsoft actually offering a more usable "basic HTML" view.

Alas, instead, we get neutered functionality and weak rendering to mobile browsers for a product that's supposed to carry us to 2013 or 2014.

The following screenshots are taken using my desktop IE8, Microsoft's Windows Mobile 6.1.4 emulator, and RIM's Blackberry 9630 emulator:

Landing Page

List View

Calendar View

New Event

In analysis, I think what makes matters worse is that they've dramatically increased the capabilities of the desktop browser version, but neglected to carry forward some of the basic functionality that seems like it wouldn't be too had to implement like the ratings or tagging functionality.  This discrepancy makes the shortcomings of the mobile offering all the more apparent.

Certainly, I'm not expecting that the desktop experience to be fully emulated on a mobile device, knowing the limitations of the small screens and limited resources and capabilities of mobile browsers.  That said, I would expect a more worthy effort than this.  I would expect something similar to Gmail's "basic HTML" option and a link to the full version.  It's an embarassment, especially the new event screen; at the least, the developers could have aligned the fields (it's pretty offensive).

It's just sloppy and lazy all around and I suggest Microsoft spend some time with the mobile browsers from their competitors.  People are accustomed to a certain level of mobile browser capabilities these days and it's nowhere near as low as Microsoft would like to believe.

For comparison's sake, here's Gmail's rendering in "mobile" and "basic HTML" mode:

Gmail desktop browser, "mobile", and "basic HTML" rendering

This is much more usable and a much more congruous experience across platforms.  Come on Microsoft, stop being lazy!  As Mark Jackson would say "You're better than that!"

On a side note, working with the emulators was a bit challenging.  If you'd like to try it out for yourself, here are some resources:

Mobile IE 6

Blackberry Devices

My experience is that working with the Blackberry emulator is extremely frustrating because you'll want to click on the screen with your mouse but then you remember that the devices don't support touch so it would only be logical that the emulator doesn't either.  However, this logic leads to a hair-pulling experience when working with the emulator.

# Friday, February 05, 2010

Paging With SPListItemCollectionPosition

Friday, February 05, 2010 10:48:01 PM UTC

Let it be known that Microsoft's terrible design of list paging is inexecusable and to make matters worse, the documentation is practically non-existent.  You would think that paging with a SPQuery would be a piece of cake!  I mean, I write paged data queries all the time.

NOT!

Colby Africa has a good overview of the issues and some basics that gave me some good insight into the core issues.  Of course, the difficulty with the SPQuery and paging isn't really with going forward, it's with going backwards.  One MSDN poster suggested storing the previous pages (see the last post here), which when you think about it, isn't a really good solution given the amount of paging strings you'd have to store (you can't just store the last navigation since going "Previous" twice requires going back twice).

Since going forward is the easy part, I won't get into that.  With regards to paging to previous pages, I hacked around a bit and spent a good 2-3 hours trying to deduce how the paging query string worked before I finally figured it out.

The algorithm works like so:

  1. When paging FORWARD, capture the ID <AND SORT FIELD VALUE> of the first item on the newly retrieved result collection.
  2. When paging BACKWARDS, use the previously captured ID <AND SORT FIELD VALUE> of the first item on the current result collection to generate the paging string.
  3. Once paged BACKWARDS, capture the ID <AND SORT FIELD VALUE> of the first item on the loaded page for the next BACKWARDS operation (just leave this off of the input query if going forwards again).

By capture, I mean to store the data and persist the data somehow (perhaps in the ViewState or SessionState).

And there you have it: three simple steps :-D

As an example, consider the following data set, sorted by Title:

ID		TITLE
---------------------------PAGE 1
17		Apple
21		Banana
18		Currant
---------------------------PAGE 2
19		Durian
5		Elderberry
1		Fig
---------------------------PAGE 3
7		Guava
10		Honey Dew
12		Indian Gooseberry

Consider a scenario where we're building a web application.  After the result set for page 3 of the data is loaded, I want to capture three pieces of data if I want to be able to load the previous page:

  1. The value of the ID of the FIRST item on the page
  2. The value of the TITLE of the FIRST item on the page (or whatever field you are sorting on)
  3. The PagingInfo string after the query is executed

The PagingInfo is already set for paging foward again.  When I page backwards, I will need to generate a new query using the PagingInfo.  To do so, I will need to:

  1. Replace the p_ID and set it to p_ID=7
  2. Replace the p_Title and set it to p_Title=Guava (replace p_Title with the static name of your sort field)
  3. Add two parameters:
    1. PagedPrev=TRUE
    2. PageLastRow=6 (last index of the second page, where we're going - this is easily calculated if you know your page size and your current page (keep these in the ViewState))

I used the following two regular expression patterns:

Regex _pidPattern = new Regex("p_ID=(?'pid'\\d+)");
Regex _pTitlePattern = new Regex("p_Title=[^&]+");

To replace the values in the string like so:

pagingInfo = _pidPattern.Replace(pagingInfo, 
	string.Format("p_ID={0}", firstItemId));
pagingInfo = _pTitlePattern.Replace(pagingInfo, 
	string.Format("p_Title={0}", firstTitle));

In this case, if I'm on page 3, the string for going back to page 2 should be:

Paged=TRUE&PagedPrev=TRUE&p_ID=7&p_Title=Guava&PageLastRow=6

And once I'm on page 2, the string required to go back to page 1 should be:

Paged=TRUE&PagedPrev=TRUE&p_ID=19&p_Title=Durian&PageLastRow=3

Well, actually, you don't need the string to go back to page 1; but this is just to give you the general idea.  In summary, the trick is pretty simple: you always need to store the PageInfo string for paging SharePoint queries and when you retrieve a resultset, you want to capture the ID and sort field of the first item.  When you go forward, the PageInfo string is enough.  When you go backwards, you need to use the captured pieces of info.

Hope that this has shed some light on the otherwise nebulous paging functionality with SharePoint list queries!

# Tuesday, January 26, 2010

Simple (?) AJAX Upload For ASP.NET

Tuesday, January 26, 2010 11:59:36 PM UTC

As I was working on an AJAX upload web part for SharePoint, I looked around to see if there was anything out there that would be suitable before I rolled my own after discovering that the ASP.NET UpdatePanel doesn't play nicely with file inputs.  Since I'm using jQuery, I figured I'd start there and see if there were any plugins which would meet the need.

To my dismay, it seems as if kids these days are all using flash (flash!) to implement asynchronous upload.  This was wholly unacceptable to me for some reason (not to mention it might cause compatibility issues for downstream clients) so I ended up tackling it myself :-D

The basics of getting this to work are simple enough; the solution is made of three main components:

  1. The main page that is displaying the upload control.  Since I was using this in a SharePoint site, I wanted to write my uploader as a web part (the control) that could be placed on any page.  The main page is simply the container for your control.  It does not post back.
  2. The control that basically just renders the <iframe/>.  The control is relatively simple.  It provides a container for the <iframe/> and also the scripts which are executed when the frame is loaded and unloaded.  The control also holds the progress layer since we want this to be shown when the user starts the upload.   This also does not post back.
  3. The upload page that receives the actual file.  This page is the target for the <iframe/> and contains the logic that validates the posted file.  This is the only page that posts back.

Before we go into the details, let's look at the screens of this in action (pay particular attention to the times):

The first screen shows the general layout of the page in the default state.  In case you're wondering, I found a handy guide for hacking the file input control and used that to customize the appearance.  Again, note the time.

Once you click "Upload", a progress layer is shown over the control.  You can also see that we've got an unloaded time now as well.

And finally, you can see that the loaded time changed once the form upload completes.

The control ASCX file contains two very simple scripts:

    function handleFrameLoaded() {
        // Do animation here.
        $("#progress").hide();
        
        $("#load").html("<b>Loaded!</b> " + 
            (new Date()).toTimeString());
    }

    function handleFrameUnloaded() {
        // Do animation here.
        $("#progress").show().fadeTo("fast", .90);
        
        $("#unload").html("<b>Unloaded!</b> " + 
            (new Date()).toTimeString());
    }

The first function is called when the frame is loaded and the second function is invoked when the file upload is submitted.  Note that both functions are called from the upload page in the <iframe/>.  In this case, I've just added simple animation calls to show and hide a progress panel.  You can hook up whatever custom code you want here.

On the upload page itself, we need to wire the events to the functions above:

    $(window).load(function() {
        parent.handleFrameLoaded();
    });

    $(document).ready(function() {
        $(".upload-button").click(function() {
            parent.handleFrameUnloaded();
        });

        /* simulate hover */
        $("#fake-container").hover(
            function() {
                $(this).addClass("hover");
            },
            function() {
                $(this).removeClass("hover");
            });

        /* simulate populating the file value since 
            we can't see the file input */
        $("input.file").change(function() {
            $("#fake input").val($(this).val());
        });
    });

It's important to note that the upload page gets its own set of window events since it's loaded inside of the frame.  The upload page makes calls to functions in the control.  I've highlighted the points of interest; you'll note that I only bind the load event of the window (I don't bind the unload).  It's also possible to do this using the onbeforeunload event, but I found that this would fire the progress layer even if I was browsing away from the page (which may confuse your users).  So it made more sense to just do it simply from the upload button click.

The upload page itself is remarkably simple:

<body>
    <form id="_form" runat="server">
    <div>
        <div id="fake-container">            
            <input type="file" id="_file" runat="server" class="file"/>            
            <div id="fake">
                <input type="text" />
            </div>            
        </div>
        <asp:Button runat="server" ID="_upload" 
            Text="Upload" OnClick="HandleUploadClick" 
            CssClass="upload-button"/>
    </div>
    </form>
</body>

The control isn't much more complex either:

<div id="frame">
    This is an asynchronous upload control.  
    The control load time is <asp:Label ID="_time" runat="server"/>

    <iframe id="upload-frame" src="Upload.aspx" 
        frameborder="0" scrolling="no" height="100px">

    </iframe>  
    <div id="progress" style="display:none;"></div>
</div>
<div>
    <div id="load"></div>
    <div id="unload"></div>
</div>

There's no codebehind for the control to speak of. The only place where you need to implement custom code is in the codebehind of the upload page to receive the posted file:

using System;
using System.Threading;
using System.Web.UI;

namespace AsyncUploadControlTest
{
    /// <summary>
    /// This is the actual page that handles the upload.
    /// </summary>
    public partial class Upload : Page
    {
        /// <summary>
        /// Handles the Load event of the Page control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="System.EventArgs"/> 
        /// instance containing the event data.</param>
        protected void Page_Load(object sender, EventArgs e)
        {
            
        }

        /// <summary>
        /// Handles the upload click.
        /// </summary>
        /// <param name="sender">The sender.</param>
        /// <param name="e">The <see cref="System.EventArgs"/> 
        /// instance containing the event data.</param>
        protected void HandleUploadClick(object sender, EventArgs e)
        {
            // Just fake a long running upload for dramatic effect
            Thread.Sleep(2500);

            // Add your logic here
        }
    }
}

There you have it. Works in SharePoint 2007 just fine.  Works in IE7 and Firefox 3.5 as well.  Because the receiving upload page is just an ASPX page, you can simply output your errors or success messages to the page itself; no special hackery required.

Download the source code here: AsyncUploadControlTest.7z (17.54 KB)

# Wednesday, January 06, 2010

SharePoint 2010 Content Type Publishing Setup

Wednesday, January 06, 2010 9:46:35 PM UTC

One of the cool features of SharePoint 2010 is the simplification of content type management/synchronization.

In SharePoint 2007, site collections formed a metadata boundary meaning that content types defined in one site collection could not be used in another site collection without redeploying the content type to the other site collection.  This works well enough if you are using features to deploy your content types (and I mean, who isn't!?), but what about managing those content types and keeping them synchronized after you've deployed them?

Nightmare.

While it's relatively easy to set up, I came across an immediate error when trying to publish my first content type:

No valid proxy can be found to do this operation.

Using the correlation ID, I looked through the logs and found the following:

No proxy associated with the site http://metadata.dev.com
recognizes the site as a hub site d4d22525-2e9d-4ea4-becc-fe1d42735ee4 

A user failed to publish or unpublish a content type
Microsoft.SharePoint.Taxonomy.ContentTypeSync.ContentTypeSyndicationException:
No valid proxy can be found to do this operation.

I came across Chakkaradeep Chandran's instructions here, but his instructions seemed to have a small gap.

To access the functionality, you need to go to Central Administration -> Application Management -> Manage service applications (under Service Applications)

In this screen, select (don't click the link) Managed Metadata Service.

Again, very important: don't click the link (that takes you to the screen for managing keywords/terms/taxonomy; click somewhere next to the text and select the row to enable the Properties button at the top.

Next click Properties:

This will bring up the pane where you can set the Content Type hub setting (scroll to the bottom of the screen).  Just enter the URL of your hub application (as an example, http://metadata.dev.com)

Next, you need to select the Managed Metadata Service Connection (the indented Managed Metadata Service) and again, click Properties.  This brings up the pane that will allow you to check the Consumes content types from the... setting.

As Chakaradeep mentions, you need to run two timer jobs to get the content types to sync up.  You can access this via Central Administration -> Check Job Status (under Monitoring):

Then page through that list looking for Content Type hub and Content Type subscriber.  Clicking on the link to the job will open it up and allow you to click the Run now button to execute the job immediately.

In any case, it's a neat feature.  I wonder, though, if there are improvements for managing deployment of content types via features (i.e. an incremental feature which adds new fields to exsiting content types and pushes the changes down) as opposed to UI based management of content types (which I'm not a terribly big fan of).  My guess is "no".

Hope this helps someone!

# Tuesday, December 22, 2009

SharePoint Design Patterns: Entry 2.5

Tuesday, December 22, 2009 3:11:48 AM UTC

In the previous entry, we looked at how we can model a SharePoint list item using a more domain specific model to simplify programmatic access to the list item thus reducing otherwise error prone data access code and making the overall framework easier to use.  Again: the idea is to promote reuse and decrease complexity through domain specific code that abstracts the underlying SharePoint object models, making it easier for a team to build functionality on top of this framework.

One interesting point is that if you're already building your fields and content types using features XML, the work required to generate the domain specific wrappers can be simplified dramatically using automation.  In a sense, a content type is basically a class (this is generally how I map them in my domain design); why double your effort and write both the content types and the classes?

So how do we go about this?

(Side note: this isn't so much a "design pattern" as it is an "implementation pattern")

As an example, here is a simple XML file which defines a set of fields and content types which use those fields:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Field DisplayName="Model Code"
    Name="Model_Code"
    StaticName="Model_Code"
    ID="{F0000000-0000-0000-0000-000000000001}"
    Type="Integer"
    SourceID="http://schemas.someusedcarinventory.com"
    Group="My Custom Columns"/>
  <Field DisplayName="VIN"
    Name="VIN"
    StaticName="VIN"
    ID="{F0000000-0000-0000-0000-000000000002}"
    Type="Text"
    SourceID="http://schemas.someusedcarinventory.com"
    Group="My Custom Columns"/>
  <Field DisplayName="Make"
    Name="Make"
    StaticName="Make"
    ID="{F0000000-0000-0000-0000-00000000003}"
    Type="Text"
    SourceID="http://schemas.someusedcarinventory.com"
    Group="My Custom Columns"/>
  <ContentType Name="Vehicle"
    ID="0x0100FC000000000000000000000000000001"
    Description="Used car inventory"
    Group="My Custom Content Types" >
    <FieldRefs>
      <FieldRef ID="{c042a256-787d-4a6f-8a8a-cf6ab767f12d}" Name="ContentType" />
      <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" 
        Required="TRUE" ShowInNewForm="TRUE" ShowInEditForm="TRUE" />
      <FieldRef ID="{F0000000-0000-0000-0000-000000000001}" Name="Model_Code"/>
      <FieldRef ID="{F0000000-0000-0000-0000-000000000002}" Name="VIN"/>
      <FieldRef ID="{F0000000-0000-0000-0000-000000000003}" Name="Make"/>
    </FieldRefs>
  </ContentType>  
  <Field DisplayName="Dealership Code"
    Name="Dealership_Code"
    StaticName="Dealership_Code"
    ID="{F0000000-0000-0000-0000-00000000004}"
    Type="Integer"
    SourceID="http://schemas.someusedcarinventory.com"
    Group="My Custom Columns"/>
  <Field DisplayName="Dealership Fax Number"
    Name="Dealership_Fax_Number"
    StaticName="Dealership_Fax_Number"
    ID="{F0000000-0000-0000-0000-00000000005}"
    Type="Text"
    SourceID="http://schemas.someusedcarinventory.com"
    Group="My Custom Columns"/>
  <ContentType Name="Dealership"
    ID="0x0100FC000000000000000000000000000002"
    Description="Dealerships"
    Group="My Custom Content Types" >
    <FieldRefs>
      <FieldRef ID="{c042a256-787d-4a6f-8a8a-cf6ab767f12d}" Name="ContentType" />
      <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" 
        Required="TRUE" ShowInNewForm="TRUE" ShowInEditForm="TRUE" />
      <FieldRef ID="{F0000000-0000-0000-0000-000000000004}" Name="Dealership_Code"/>
      <FieldRef ID="{F0000000-0000-0000-0000-000000000005}" Name="Dealership_Fax_Number"/>
    </FieldRefs>
  </ContentType>    
</Elements>

When approaching this problem, I considered three ways of handling the class file generation:

  1. Use an object model and StringTemplate to create .cs files.  This invovled writing POCO classes (or generating them from the schema) which I could deserialize the XML to and then passing those objects to a template instance.  This seemed like too much work, given that I really didn't feel like maintaining all of that code as well.  Plus, while StringTemplate isn't - by any sense of the imagination - hard, it is a non-standard syntax that someone would have to learn to maintain and/or extend the conversion.
  2. Use an XDocument and CodeDom to create .cs files.  This seemed like even more work!  While it's framework supported, I feel like this solution would be hard to extend and maintain for most developers.
  3. Use an XSL transform to create .cs files.  This seemed to be the most natural solution given that the source file is already in XML format and a the target content structure is far from complex (the basic class file is fairly simple).  Plus, while XSLT isn't trivial, it's not that hard either (and the syntax is "standard").

One of the cool features of XSL 2.0 is the xsl:result-document element which allows you to create multiple documents from one source document.  Only one problem: .NET's XSLT engine doesn't implement XSLT 2.0!  What a bummer; it seemed like if I wanted to get this to work and generate multiple output files, it was going to take some work in code or find an XSLT 2.0 capable processor.

Enter Saxon, which provides an XSLT 2.0 processor for .NET.  The following code takes the XML above and uses the xsl:result-document to create two class files, one for each content type:

using System;
using System.IO;
using Saxon.Api;

namespace FeatureToClass
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Uri xmlFile = new Uri(
                @"C:\Users\Charles\Desktop\elements.xml");

            // Create a Processor instance.  
            Processor p = new Processor();

            // Load the source document.  
            XdmNode node = p.NewDocumentBuilder().Build(xmlFile);

            using (Stream stream = File.OpenRead("core-transform.xslt"))
            {
                // Create a transformer for the stylesheet.  
                XsltTransformer transformer = 
                    p.NewXsltCompiler().Compile(stream).Load();

                // Set the root node of the source document
                // to be the initial context node.  
                transformer.InitialContextNode = node;

                // BaseOutputUri is only necessary for xsl:result-document.  
                transformer.BaseOutputUri = xmlFile;

                transformer.SetParameter(
                    new QName("ct", "http://www.customtransform.com", "namespace"), 
                    new XdmAtomicValue("My.Custom.Package"));

                // Create a serializer.  
                Serializer serializer = new Serializer();
                transformer.Run(serializer);
            }
        }
    }
}

The code above is a simple console program that takes a (hardcoded) path to a source XML file (a SharePoint elements.xml file) and (hardcoded) namespace and loads an XSL file to transform the XML to C# class files.

Here's the transform (it's a bit messy for output formatting reasons, so you're best off copying it into an XML aware text editor to get a better view):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE stylesheet [
    <!ENTITY space "<xsl:text> </xsl:text>">
    <!ENTITY cr "<xsl:text>
</xsl:text>">
]>

<xsl:stylesheet
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:sp="http://schemas.microsoft.com/sharepoint/"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:functx="http://www.functx.com"
  xmlns:ct="http://www.customtransform.com"
  version="2.0">
    <xsl:param name="ct:namespace"/>
    <xsl:output method="text"/>

    <xsl:template match="/">
        <xsl:for-each select="//sp:ContentType">
            <xsl:variable name="classname" select="replace(@Name, ' ', '')"/>
            <xsl:variable name="filename" select="concat($classname,'.cs')" />
            <xsl:value-of select="$filename" />
            <!-- Creating  -->
            <xsl:result-document href="{$filename}">
using System;

    namespace <xsl:value-of select="$ct:namespace"/>
    {
        public partial class <xsl:value-of select="$classname"/>
        {
            private string _contentTypeId;
            
            public string ContentTypeId {
                get { return _contentTypeId; }
                set { _contentTypeId = value; }
            }

            public <xsl:value-of select="$classname"/>()
            {
                _contentType = "<xsl:value-of select='@Name'/>";
                _contentTypeId = "<xsl:value-of select='@ID'/>";
            }

            <xsl:apply-templates/>
    }
}
            </xsl:result-document>
        </xsl:for-each>
    </xsl:template>

    <!--///
        Templates
    ///-->
    <xsl:template match="sp:FieldRef">
        <xsl:variable name="fieldname" select="functx:lower-first(replace(@Name, '_', ''))"/>
        <xsl:variable name="fieldid" select="@ID"/>
        <xsl:variable name="dotnettype">
            <xsl:choose>
                <xsl:when test="//sp:Field[(@ID = $fieldid) and (@Type = 'Integer')]">int</xsl:when>
                <xsl:otherwise>string</xsl:otherwise>
            </xsl:choose>
        </xsl:variable>

        private <xsl:value-of select="$dotnettype"/> _<xsl:value-of select="$fieldname" />;
        <xsl:call-template name="attribute"><xsl:with-param name="fieldid" select="$fieldid"/></xsl:call-template>
        public <xsl:value-of select="$dotnettype"/>&space;<xsl:value-of select="replace(@Name, '_', '')"/>
        {
            get { return _<xsl:value-of select="$fieldname" />; }
            set { _<xsl:value-of select="$fieldname" /> = value; }
        }
    </xsl:template>

    <xsl:template name="attribute">
        <xsl:param name="fieldid"/>
    <xsl:if test="exists(//sp:Field[@ID = $fieldid]/@ID)">[Caml(Id="<xsl:value-of select='//sp:Field[@ID = $fieldid]/@ID'/>", StaticName="<xsl:value-of select='//sp:Field[@ID = $fieldid]/@StaticName'/>", Type="<xsl:value-of select='//sp:Field[@ID = $fieldid]/@Type'/>")]</xsl:if></xsl:template>


    <!--///
        Custom functions
    ///-->
    <xsl:function
        name="functx:lower-first" as="xs:string"
        xmlns:functx="http://www.functx.com" >
        <xsl:param name="arg" as="xs:string"/>
        <xsl:sequence select="concat(lower-case(substring($arg,1,1)), substring($arg,2))"/>
    </xsl:function>
</xsl:stylesheet>

I borrowed one function from the FunctX library to create the camelCased field names.  The XSL probably isn't nearly as clean or optimized as it should be (my XSL is admittedly a bit rusty), but it gets the job done.  Here's one of the two classes (and class files) which get generated at the source directory of the input XML file:

using System;

namespace My.Custom.Package 
{
    public partial class Dealership
    {
        private string _contentTypeId;
        public string ContentTypeId {
            get { return _contentTypeId; }
            set { _contentTypeId = value; }
        }   

        public Dealership()
        {
            _contentType = "Dealership";
            _contentTypeId = "0x0100FC000000000000000000000000000002";
        }
                        
        private string _contentType;
        public string ContentType
        {
            get { return _contentType; }
            set { _contentType = value; }
        }      

        private string _title;
        public string Title
        {
            get { return _title; }
            set { _title = value; }
        }
    
        private int _dealershipCode;
        [Caml(Id="{F0000000-0000-0000-0000-000000000003}", 
            StaticName="Dealership_Code", Type="Integer")]
        public int DealershipCode
        {
            get { return _dealershipCode; }
            set { _dealershipCode = value; }
        }
    
        private string _dealershipFaxNumber;
        [Caml(Id="{F0000000-0000-0000-0000-000000000004}", 
            StaticName="Dealership_Fax_Number", Type="Text")]
        public string DealershipFaxNumber
        {
            get { return _dealershipFaxNumber; }
            set { _dealershipFaxNumber = value; }
        }
    }
}

Awesome!  It's amazing how little code was required to get this basic scenario working.

Now we have a single source which defines our SharePoint artifacts and our code artifacts; I love it.  Write your fields and content types in your feature and you get class files for free!  You'll note that I've added a simple CamlAttribute where applicable.  This will prove handy when it comes time to automate construction of the object instance from a SharePoint list item, which we'll look at next time (for the time being, feel free to modify the XSL and remove the line for it or write an implementation of CamlAttribute).

Again, to reiterate: the goal is make it easy to build applications on top of a SharePoint deployment by adding a layer of domain specific APIs and objects so that a team can be productive while reducing duplication and the ramp up time required to understand the business domain. 

Other points for improvement and enhancement (look for these in a future installment):

  1. Parameterize the program.
  2. Consider making it a Visual Studio add-in or a custom tool.
  3. Make it go the other way; in other words: generate the content type and field XML from class files (which would be cool, too).

But even as it is, it's incredibly useful.  On the next installment, we'll see how we can build more intelligence into the model and make it more useful.

# Monday, December 07, 2009

Host Headers, SharePoint (2010), and Access Denied (401)

Monday, December 07, 2009 8:10:25 PM UTC

I've been working on setting up a VM to play around with SharePoint 2010 and kept running into a weird issue where I would be prompted for my credentials repeatedly and I was denied access to the site entirely, even when using an administrator account.

I had set up my SharePoint web application using a host header; for example, beta.dev.com.

Meanwhile, I was able to access the same site from my host environment without any issue.  What gives?  Turns out that there is a slight registry tweak you have to make when working with host headers and IIS.

Phil Harding has a good writeup with links to the Microsoft KBs and additional details.

# Friday, November 13, 2009

Removing a Dead SharePoint Web Part

Friday, November 13, 2009 7:48:08 PM UTC

If you've deleted the DLL or if you reinstalled a DLL which no longer contains the codebehind class for a user control in a web part, you could end up in a scenario where you have a "dead" web part on the page.  In my case, it prevented the page from loading and resulted in an error because it couldn't load the type.

Don't panic!  The solution is to add a ?Contents=1 to the URL (i.e. http://moss.dev.com/default.aspx?Contents=1), which will allow you to delete the web part from the page.

# Monday, September 28, 2009

SharePoint? Is That You?

Monday, September 28, 2009 6:46:05 PM UTC

Weird discovery of the day: Recovery.gov is SharePoint (check the source or try a search).

Cool.

# Saturday, August 29, 2009

SharePoint Design Patterns: Entry 2

Saturday, August 29, 2009 4:58:15 PM UTC

In the previous entry, we explored how to clean up our interaction with instances of SPSite, SPWeb, and SPList objects.  Logically, the next scenario that we'd want to cover is working with the SPListItem that we get back from the list.

What does this look like?  Usually something like this (we'll use some sample code from our last entry):

public string GetLastModifiedBy(int id) {
    string lastModifiedBy;

    using(ProposalsLibrary library = new ProposalsLibrary()) {
        SPListItem item = ...; // Get the instance

        lastModifiedBy = Convert.ToString(item["Modified_x0020_By"]);
    }

    return lastModifiedBy;
}

By itself, this seems innocent enough, but the problem lies in the fact that it's rarely so easy as returning simply one field value.  Even if that were the case, the issue remains that these field name strings leak out across the codebase.  Again, this is typically mitigated through the use of Constants or configuration settings, but it's still messy code, IMO.  Aside from that, it can be difficult to figure out which fields are valid for which lists.  In general, it makes it hard for developers to figure out how to work with the existing code without good documentation.

Enter the Decorator pattern.  The intent of Decorator, as described in Design Patterns, is as follows:

Attach additional responsibilities to an object dynamically.  Decorators provide a flexible alternative to sub-classing for extending functionality.

Basically, what we'd like to do is to make working with SPListItems more intuitive and more domain specific.  Since we can't subclass SPListItem (and even if we could I don't think it would make sense to do so since it would also expose all of the properties and operations on the SPListItem), we'll have to leverage the simplified version of the Decorator pattern to help us instead.  Here is an example:

using System;
using Microsoft.SharePoint;

namespace SharePointExample
{
    /// <summary>
    /// Models a sales proposal.
    /// </summary>
    public class Proposal
    {
        private SPListItem _managedItem;
        private string _title;
        private string _lastModifiedBy;
        private bool _checkedOut;
        private string _checkedOutUserLoginName;
        private string _checkedOutUserDisplayName;
        private string _status;
        private string _customerName;

        /// <summary>
        /// Gets the title.
        /// </summary>
        /// <value>The title.</value>
        public string Title
        {
            get { return _title; }
        }

        /// <summary>
        /// Gets the login name of the check out user.
        /// </summary>
        /// <remarks>
        /// Null or empty if not checked out.
        /// </remarks>
        /// <value>The login name of the checkout user.</value>
        public string CheckedOutUserLoginName
        {
            get { return _checkedOutUserLoginName; }
        }

        // Other properties omitted...

        /// <summary>
        /// Private constructor; use the <see cref="FromSPListItem"/>
        /// call to create an instance.
        /// </summary>
        private Proposal() { }

        /// <summary>
        /// Creates an instance from a SharePoint list item.
        /// </summary>
        /// <param name="item">The item.</param>
        /// <returns>An instance of <c>Proposal</c>.</returns>
        public static Proposal FromSPListItem(SPListItem item)
        {
            Proposal proposal = new Proposal();

            proposal._title = Convert.ToString(item["Title"]);
            proposal._checkedOut = (item["CheckoutUser"] != null
                && item.File.CheckOutStatus != SPFile.SPCheckOutStatus.None);

            if (proposal._checkedOut) {
                string fieldValue = Convert.ToString(item["CheckoutUser"]);

                // NOTE: item.Fields requires the display name.
                SPFieldUserValue checkoutUser = (SPFieldUserValue)
                    item.Fields["Checked Out To"].GetFieldValue(fieldValue);

                proposal._checkedOutUserLoginName = checkoutUser.User.LoginName;
                proposal._checkedOutUserDisplayName = checkoutUser.User.Name;
            }

            // Set other fields here...

            proposal._managedItem = item;

            return proposal;
        }
    }
}

If nothing else, we now have a clean, domain specific way to access the SharePoint list item.  Team members and new developers don't have to guess which fields are valid for this items retrieved from this list; it would be hidden from them by the properties on the Proposal class instead.  If you look at the code to determine the check-out user, you can see that we now also have a single location to encapsulate this parsing logic and any associated error handling we may want to add; other developers don't need to duplicate this code when they use the list item.

It's debatable whether you should use a constructor to initialize the instance, some sort of implicit conversion operation, or a static method like I've used here.  I would rule out an implicit conversion operator since it may be hard for users to understand at first blush (i.e. XName and string).  As for a "natural" constructor that takes the SPListItem instance?  While it's slightly more discoverable (XDocument.Parse() is pretty hard to find for new users), I feel like that's a bit misleading and the intent isn't clear, however, YMMV.  Another possible improvement is to pull the abstraction up yet another level since many of the fields are common (like title and last modified by); this exercise is left to the reader ;-).

If we go back to our first example, we can now write this as:

public string GetLastModifiedBy(int id) {
    string lastModifiedBy;

    using(ProposalsLibrary library = new ProposalsLibrary()) {
        SPListItem item = ...; // Get the instance

        lastModifiedBy = Proposal.FromSPListItem(item).LastModifiedBy;
    }

    return lastModifiedBy;
}

That's pretty cool.  While this example is intentionally simple, you now have a nice object oriented way of working the the fields.  In addition, you have a very logical place to put all of your domain specific operations on specific list item types (or should that be content types?).  For example, it may make perfect sense to add a method here called CompressAndEmail(string recipientEmailAddress) which would compress the file contents of the proposal and email it to the specified recipient.

Admittedly, this still isn't ideal since 1) we still have to deal with SPListItem on some level ("leakage", if you will) and 2) what about the other side of this equation: updating the item?  With regards to (1), we can simply hide this by adding a method to the ProposalsLibrary class as discussed in the previous entry:

using Microsoft.SharePoint;

namespace SharePointExample
{
    /// <summary>
    /// Concrete implementation of <see cref="Library"/>
    /// </summary>
    public class ProposalsLibrary : Library
    {
        protected override string ListName
        {
            get { return "<listNameHere>"; }
        }

        protected override string WebName
        {
            get { return "<webNameHere>"; }
        }

        /// <summary>
        /// Finds and instance of <see cref="Proposal"/> the by ID.
        /// </summary>
        /// <param name="id">The ID.</param>
        /// <returns>An instance of <see cref="Proposal"/>.</returns>
        public Proposal FindById(int id)
        {
            SPListItem item = ...; // Find the list item

            return Proposal.FromSPListItem(item);
        }
    }
}

Now our method looks like this instead:

public string GetLastModifiedBy(int id) {
    string lastModifiedBy;

    using(ProposalsLibrary library = new ProposalsLibrary()) {
        Proposal proposal = library.FindById(id);

       lastModifiedBy = proposal.LastModifiedBy;
    }

    return lastModifiedBy;
}

That's pretty awesome since it means that we can now remove all references to and knowledge of SharePoint from a whole layer of our application code. 

Now onto point (2).  For a moment, let's say our proposals have properties called "Status" and "CustomerName".  From time to time, we may want to update these (from a source other than the web form, like through a custom web service call or if we have a batch process that runs on our SharePoint environment).  One way we can handle this is by implementing the set operation on the properties which can be updated and a Save() method:

using System;
using Microsoft.SharePoint;

namespace SharePointExample
{
    /// <summary>
    /// Models a sales proposal.
    /// </summary>
    public class Proposal
    {
        private SPListItem _managedItem;
        private string _title;
        private string _lastModifiedBy;
        private bool _checkedOut;
        private string _checkedOutUserLoginName;
        private string _checkedOutUserDisplayName;
        private string _status;
        private string _customerName;

        /// <summary>
        /// Gets or sets the status.
        /// </summary>
        /// <value>The status.</value>
        public string Status
        {
            get { return _status; }
            set { _status = value; }
        }

        /// <summary>
        /// Gets or sets the name of the customer.
        /// </summary>
        /// <value>The name of the customer.</value>
        public string CustomerName
        {
            get { return _customerName; }
            set { _customerName = value; }
        }

        // Other properties omitted...

        /// <summary>
        /// Private constructor; use the <see cref="FromSPListItem"/>
        /// call to create an instance.
        /// </summary>
        private Proposal() { }

        /// <summary>
        /// Saves this instance back to SharePoint.
        /// </summary>
        public void Save()
        {
            // Error checking first; i.e check if user has item checked out

            // Save
            _managedItem["Customer_x0020_Name"] = CustomerName;
            _managedItem["Status"] = Status;
            _managedItem.SystemUpdate(false);
        }

        /// <summary>
        /// Creates an instance from a SharePoint list item.
        /// </summary>
        /// <param name="item">The item.</param>
        /// <returns>An instance of <c>Proposal</c>.</returns>
        public static Proposal FromSPListItem(SPListItem item)
        {
            // Same as before; omitted
        }
    }
}

Nice.  In doing this, we've basically hidden most knowledge of the SharePoint list item from the users of our object and framework.  Downstream developers only need to know about the domain specific objects.  It's now far easier for a member of our team or a new developer to reuse this code and it cuts down on error prone duplication and leakage of field names across our codebase.  We also now have a nice place to put field validation logic (for example, in the Save() method).  If we wanted to, we can also add logic here (through some references to services or DAOs or something) to load additional metadata from external systems like databases.

If you want to get fancier, you could also add a dictionary keyed by string (internal field name) and with value type object and use that to hold your "set" properties until Save() is called.  In the save, you'd simply iterate the keys and set the values then update, only changing the values that were actually set by the caller.

Updating the status of a proposal would go from looking like this:

public void UpdateStatus(int id, string status) {
    using(SPSite site = new SPSite(_siteUrl)) {
        using(SPWeb web = site.OpenWeb(_web)) {
            SPList list = web.Lists[_list];

            SPListItem item = ...; // Get the list item
            
            // Perhaps add error checking.

            item["Status"] = status;

            item.SystemUpdate();
        }
    }
}

To this instead:

public void UpdateStatus(int id, string status) {
    using(ProposalsLibrary library = new ProposalsLibrary()) {
        Proposal proposal = library.FindById(id);

        proposal.Status = status;
        proposal.Save();
    }
}

It becomes even more compelling once we consider that in a real world implementation of an UpdateStatus() operation, it would probably involve checking to see if a user has the object checked out in the first place!  This would make our first example explode into a giant mess of code while in our second one, it would be one line (or integrated into the Save() method which could throw a custom NotCheckedOutException or framework InvalidOperationException or try to check it out automatically).

All this with relatively little work involved.  It's better in every way: less nesting of code, much more readable and natural, less error prone, better discoverability, and more domain specific (and less SharePoint-centric).  In the end, I think it dramatically improves usability and reuse of your SharePoint application code.  It also leads to a nice, logical place to encapsulate much of the common, repetitive, and error prone code that would otherwise be littered among your application (or worse, UI) code.

In future installments, we'll examine how flesh out the FindById() method on the ProposalsLibrary.  We'll also examine a GUI pattern to help promote reuse, improve tesatability, and cut down on duplication of business logic.

Design Patterns For SharePoint : Entry 1

Saturday, August 29, 2009 2:02:35 AM UTC

One thing that I've discovered is that it's easy to write sloppy, hard to read code when SharePoint is involved.  A lot of it may be due to the quick-and-dirty code samples out on the 'Net and a general lack of thought put into the structural details of the examples on MSDN (I can understand why as it can make example code much longer to write it well).  I have a whole series of blog post ideas on how to mitigate this (based on my own real world experiences) that I've been meaning to put together that kind of put together a big picture of how to make your team's SharePoint development experience less crappy.

While I have some general workflow related tips and tricks, I'll start off this series with a code based sample.  Something small that I think will make a big impact in cleaning up code.

One of the most common things that developers need to do when working with SharePoint is to create an instance of SPSite, open an instance of SPWeb and then get a list (or do something else against the web).

Typically, this takes on the following pattern:

private static readonly string  _siteUrl;
private static readonly string  _web;
private static readonly string  _library;
private static readonly string  _otherLibrary;
static MySharePointDAO() {
    _siteUrl = ...; // Hard coded, from a Constants class, or from config.
    _web = ...;
    _library = ...;
    _otherLibrary = ...;
}

public void DoSomething() {
    using(SPSite site = new SPSite(_siteUrl)) {
        using(SPWeb web = site.OpenWeb(_web)) {
            SPList list = web.Lists[_library];

            // Code goes here.            
        }
    }
}
public void DoSomethingElse() {
    using(SPSite site = new SPSite(_siteUrl)) {
        using(SPWeb web = site.OpenWeb(_web)) {
            SPList list = web.Lists[_otherLibrary];

            // Code goes here.            
        }
    }
}

There are many different variations of this basic pattern (in the worst case, this code is written right into layout pages or webpart code which I imagine to be quite a common practice...).  Sometimes, the strings are hard coded inline, sometimes the strings are from a Constants class, sometimes the strings are read from a configuration file.  No matter how it's done, it tends to muck up the code and it ends up all over the place.  One of my beefs with this is that it's not very aesthetically pleasing; deep nesting does that (plus, depending on how deep the rest of the code nests, it may start to scroll horizontally...yuck!).  You can also stack the using statements (instead of nesting), but that still leaves it a mess (my other beef) since you need to specify the site URL, the web, and potentially a list - it creates a lot of useless, duplicated code all over your codebase.

Yet there is a very simple solution: abstract this logic into a class that implements IDisposable:

using System;
using Microsoft.SharePoint;

namespace SharePointExample
{
    /// <summary>
    /// Abstraction for a SharePoint list.
    /// </summary>
    public abstract class Library : IDisposable
    {
        private static readonly string _siteUrl;

        private readonly SPSite _site;
        private readonly SPWeb _web;
        private SPList _list;

        /// <summary>
        /// Static initializer that sets the site URL.
        /// </summary>
        static Library()
        {
            _siteUrl = ...; // Hard coded, from config, or Constants
        }

        /// <summary>
        /// Default constructor that creates the site and web.
        /// </summary>
        protected Library()
        {
            _site = new SPSite(_siteUrl);
            _web = _site.OpenWeb(WebName);
            _list = _web.GetList(ListName);
        }

        /// <summary>
        /// Gets the name of the list.  Inheriting classes
        /// implement this property for a specific list.
        /// </summary>
        /// <value>The name of the list.</value>
        protected abstract string ListName { get; }

        /// <summary>
        /// Gets the name of the web.
        /// </summary>
        /// <value>The name of the web.</value>
        protected abstract string WebName { get; }

        #region IDisposable Members

        /// <summary>
        /// Performs application-defined tasks associated with freeing, 
        /// releasing, or resetting unmanaged resources.
        /// </summary>
        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        #endregion

        /// <summary>
        /// Releases unmanaged and - optionally - managed resources
        /// </summary>
        /// <remarks>
        /// See Cwalina, Abrams; Framework Design Guidelines, 1st Ed., p. 251
        /// </remarks>
        /// <param name="disposing"><c>true</c> to release both managed and 
        /// unmanaged resources; <c>false</c> to release only unmanaged resources.</param>
        protected virtual void Dispose(bool disposing)
        {
            if (disposing)
            {
                if (_web != null) _web.Dispose();
                if (_site != null) _site.Dispose();
            }
        }
    }
}

(It's generally not advised to call abstract/virtual members in a constructor, but in this case, they should simply return strings -- should be safe.)

And that's it; so simple.  What do our calls look like now?  From our presenters/controllers/views, we have something like this instead:

using(ConcreteLibrary library = new ConcreteLibrary()) {
	// Do stuff here with the library.
}

If we only consider this as a win, then it's a very small win.  But in the bigger picture, our downstream callers don't need to know about the SPSite, SPWeb, or the SPList if we take the abstraction far enough -- we'll examine this in future installments.  Furthermore, we now have a convenient place to put common list based operations and if our business requirements dictate that there are specific lists for specific types of documents, this abstraction becomes even more useful.

I've intentionally left the "Do stuff here..." empty.  In the simplest case, at this point, you can simply expose the list, web, and/or site as public properties to inheriting/calling classes and just access library.List or library.Web and use it as you normally would.  If nothing else, you've cleaned up the nesting and the leakage of the site URL, web names, and list names.  (To make this abstraction more useful, IMO, we would have you hide all of our operations in the ConcreteLibrary and expose the site, web, and list as protected properties to the inheriting classes only.)

This design assumes that you have specific, well known libraries in your SharePoint deployoment in which case it makes sense to create library specific methods and a concrete library sub-type for each library.  For example, if a library holds proposals that your sales department has written, you may want to have a class called ProposalsLibrary with a method called FindPendingProposals()which is specific to that library.  Of course, common functionality across all lists can be easily added to the base abstract Library class (and this usually involves a "find" operation of some sort).

In general, it's a very small, easy, and intuitive change that makes the code that implements your business logic (as it pertains to SharePoint) much easier to read and to maintain.  As we'll explore in future entries in this series, if we take the abstraction far enough, we can completely eliminate the Microsoft.SharePoint namespace from our business/presentation logic.

See also: SharePoint Design Patterns: Entry 2

# Tuesday, November 25, 2008

SharePoint And Kerberos

Tuesday, November 25, 2008 4:26:16 PM UTC

In short: don't do it to yourself!

Okay, so that's not a realistic solution.  But you know what?  It is a pain in the butt.  Terrible.  Teeeeeeerrible.  I've spent the last two days battling Kerberos, Active Directory, and NLB trying to figure out what the heck is going on with a test environment I'm building.

So this is what I came away with:

  • Setting up Network Load Balancing.  This is a great article on how to set up NLB before you set up your SharePoint environment.
  • Setting up SharePoint with Kerberos.  This is your starting point for configuring SharePoint for Kerberos.  It's confusing as heck and creates an inordinate number of domain accounts which I think are totally extraneous, but it's the most extensive document that I found on the subject.
  • Enabling Kerberos Logging.  You're going to need it.
  • Troubleshooting Kerberos/IIS Errors.  Good tips on working with Kerberos.
  • HTTP 401.1 with Kerberos.  This document probably had the most important tip:
    Important An SPN for a service can only be associated with one account. Therefore, if you use this suggested resolution, any other application pool that is running under a different domain user account cannot be used with Integrated Windows authentication only.
  • Ask The Directory Services Team.  A blog with many good posts on Kerberos issues.  The Kerberos introduction is useful to start with.
  • Wireshark.  This will let you watch the Kerberos and DNS traffic which can help surface errors and provide more diagnostic information than the Windows Kerberos event logging alone.  You can just trap all traffic on the physical interface and filter using "kerberos".

The tip in KB871179 was particularly useful since this, I think, was what was causing me all this trouble.  Be sure that you're not registering an SPN multiple times!

# Friday, October 10, 2008

Getting SharePoint Task Assignee

Friday, October 10, 2008 4:07:36 PM UTC

One of the more interesting problems I've been working with is trying to figure out how to get the user information for the assignee (whom the task is assigned to).  Getting the user ID and the the user display name is easy enough, but then that entails another lookup to find the the login name for the user.

For example, here is some code which retrieves the user information, but as a string:

using (SPSite site = new SPSite(siteUrl))
{
    using (SPWeb web = site.OpenWeb())
    {
        SPList tasks = web.Lists["Tasks"];

        foreach (SPListItem task in tasks.Items)
        {
            string assignedTo = Convert.ToString(task["Assigned To"]);

            Console.Out.WriteLine("> {0}", assignedTo);
        }
    }
}

This generates the following output:

> 37;#Brad Wright 
> 40;#Daniel OConnor 
> 46;#Charles 

As mentioned, this is only part of the information we're after.  To get the login name, we'd have to do some simple string parsing and then make a separate call to look up the user.  Not a terrible amount of work, but more work, nonetheless.

Stepping through the debugger, I could see that the actual type of the field was SPFieldUserValue.  My first attempt was to see if I could convert the value directly; no go.  It turns out that the value retrieved (before calling the ToString()) is already a string.  I ended up fumbling around with the very awkward GetFieldValue() method on the SPField class.  It's not at all intuitive on how this method is supposed to be used, so I'm hoping this is useful:

using (SPSite site = new SPSite(siteUrl))
{
    using (SPWeb web = site.OpenWeb())
    {
        SPList tasks = web.Lists["Tasks"];

        foreach (SPListItem task in tasks.Items)
        {
            string fieldValue = Convert.ToString(task["Assigned To"]);

            SPFieldUserValue assignedTo = (SPFieldUserValue)
                task.Fields["Assigned To"].GetFieldValue(fieldValue);

            Console.Out.WriteLine("> {0} ({1})", 
                assignedTo.User.Name, assignedTo.User.LoginName);
        }
    }
}

And here is the output

> Brad Wright (FP1\bwright)
> Daniel OConnor (FP1\doconnor)
> Charles (FP1\charles)

The call is all kind of awkward and is completely non-intuitive.  It's easier to think of it as item.Fields["Some Field"].ConvertToNativeOutputType(fieldValue).  So you can see, this is a much more convenient way of retrieving the user from a list item (or any typed return value) once you get the hang of the weird nomenclature and usability issues.

# Thursday, June 19, 2008

IE7, AJAX, And 400 "Bad Request"

Thursday, June 19, 2008 10:11:27 PM UTC

I spent bout two hours today tring to figure out why I kept getting a weird error while using prototype to call a WCF service.

There are various other resources on the web regarding how WCF has a few quirks with error handling using the default endpoints, however, none of these scenarios were applicable to me since the JavaScript call worked fine in FireFox.  To complicate things even further, it actually all worked fine in IE7 as well with one caveat: it only worked on the first load of the IE7 process.

After that, any refreshing of the page would return 400/"Bad Request" errors, mysteriously.  Just to make sure that it wasn't isolated to prototype, I also tried jQuery as well.  Still nothing.

Well, as it turns out, that setting the AJAX request content type to application/json wasn't enough; I had to set it to application/json; charset=utf-8.

So if you're encountering weird issues with any AJAX library, IE7, and WCF and/or ASP.NET, be sure to check your content type setting.

Update 6/20/2008: bah!  I woke up this morning and it's not working again!  Oddly, it works fine if I have Fiddler running and as soon as I shut off Fiddler, it stops working.  To complicate matters even more, it's only happening on methods where I have post content.

Update 6/22/2008: after further investigation, I'm concluding that it's related to SharePoint.  To isolate the different components, I created a custom WCF service and a simple runtime which models our current custom WCF runtime.  When I used a plain HTML page or even an ASPX page with a script manager on it, prototype AJAX requests worked fine.  Okay, so it wasn't how we were hosting the service and it wasn't an issue with a collision between ASP.NET AJAX and prototype.  I then hooked a SharePoint layout page up to the same service and boom, it all breaks :-S

I've broken it out into the following scenarios:

  1. Using FireFox, the requests always work.  No matter how many times I refresh the page, the AJAX request always works fine.
  2. Using IE7, it will always work on the first load of the IE process. 
    1. If I hit CTRL+F5, it will continue to work.
    2. If at any point, I hit F5, it will fail and even CTRL+F5 will not fix it.
    3. If I have Fiddler running, it will always work, even if it entered a failure state after hitting F5! (So Fiddler must be doing something to the HTTP message??). As far as I can tell, the only difference between the Fiddler request and the native request, from viewing the WireShark trace, is the "Authorization" header and the fact that the failed request has a "Content-length" header value of 0 (manually setting the header doesn't work either).  I verified that prototype was not somehow mangling the POST body content by writing a trace of the XHR request body right before and right after the request is sent (both came out okay).

One of the really weird things about this error is that it cannot be observed when I have Fiddler running; somehow, Fiddler "fixes" the issue.  I had to hook up WireShark on the server and watch the raw TCP messages to finally see that on the unsuccessful attempts, the POST content to my WCF service was empty.  The method required one parameter, so the WCF serializer threw a formatter excpetion when it received an empty message:

The server encountered an error processing the request. The exception message is 'Error in deserializing body of request message for operation 'GetRoutes'. The OperationFormatter could not deserialize any information from the Message because the Message is empty (IsEmpty = true).'. See server logs for more details. The exception stack trace is:at System.ServiceModel.Dispatcher.PrimitiveOperationFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.DemultiplexingDispatchMessageFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.UriTemplateDispatchFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.CompositeDispatchFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.DeserializeInputs(MessageRpc& rpc) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc) at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

I've tried playing around with various cache avoidance strategies to no avail (setting the "Cache-Control" header, setting the "If-Modified-Since" header, setting the "Pragma" header, addin randomness to the POST URL) so for now, I'm just going to have to give up on this.  I've confirme the error using three different OS environments (XP SP2 (desktop), XP64 SP2 (dev), and 2003 R2 SP2 (VM)), all running IE 7.0.5730.11.

I know there are various articles out there regarding IE issues with XHR and how the ordering of the open(), onreadystatechange, and send() operations must be called; however, I verified in the prototype source (lines 1213-1223) that the correct order of operations are being performed.

Thinking that perhaps some Microsoft mojo was behind all of this, I gave the Sys.Net.WebRequest a shot.  This also did not yield any postive results.  Oddly, ASP.NET AJAX to web service calls all work fine (we had some existing AJAX components which were calling to a web service, but the goal was to transition away from writing these "empty" proxy services and call directly to the WCF service).

If any SharePoint development team members or program managers are reading this, please look into it!  I'm not sure if it affects IIS hosted WCF services at this point, but a sample project can be downloaded from here: WcfAjaxWonkiness.zip.  Create a layout page which calls the service and add it to SharePoint and observe it blowing up.

Synchronizing SharePoint Files And Development Files

Thursday, June 19, 2008 1:37:27 PM UTC

I've previously posted on how to move compiled binaries into a remote GAC automatically from your development environment to your SharePoint environment using SysInternal's PsExec utility.

But what about moving content files from your SharePoint environment to your development environment?  When I'm developing content for layout pages (ASPX, js, css), for the most part, I don't want to develop in Visual Studio; I will usually use EditPlus (my weapon of choice) and edit the files directly on the remote server.

However, this poses a problem: synchronizing files between the SharePoint environment and the development environment.  It's quite time consuming and painful to do it manually since you may have to dig through various directories and keep tabs on which files you've edited on the server.

Enter 2BrightSparks' SyncBack utility.  Using this, I can create a synchronization automation between my SharePoint environment and my development environment.  And because I've set up my files to have the same structure on both sides, it's easy to set up the synchronization settings.

The image above shows the layout of my project. You can see in the image below at the directory layout mirrors the project layout, making automated synchronization a snap.

Working smarter > working harder.

# Tuesday, May 13, 2008

log4net With SharePoint Layout Page Applications

Tuesday, May 13, 2008 4:36:55 PM UTC

Using log4net with SharePoint layout page applications is really no different from using it with other types of web applications with the exception that there really isn't a convenient way to initialize the logging configuration from your custom binaries.

The answer lies in the oft overlooked AssemblyInfo.cs file.

Add the following line to the file:

[assembly: log4net.Config.XmlConfigurator(Watch = true)]

You may also want to add log4net binary to the /App_Bin directory of the WSS virtual directory (as well as the configuration into the web.config file of the application, of course).

# Tuesday, May 06, 2008

SharePoint "Unknown Error" Quirk

Tuesday, May 06, 2008 7:16:53 PM UTC

Anyone who's done any bit of SharePoint development is probably familiar with the completely useless "Unknown Error" view.  Well, in fact, SharePoint actually knows what the error is, it just doesn't want to tell you (okay, it's really just disabled for users).

In ASP.NET, you can usually get error messages to show up by setting <customErrors="Off"/>

However, this is not sufficient with SharePoint. As Stefan Keir Gordan points out, to get the nitty gritty details, you also need to set <SafeMode CallStack="True""/>

# Thursday, April 24, 2008

Nuking SharePoint Styles For Layout Page Applications

Thursday, April 24, 2008 12:59:46 PM UTC

SharePonit layout apps (as I call them) are the missing link in most developer's understanding of SharePoint and why it seems like it can be a pain in the butt as a development platform.  As I've discussed previously, SharePoint development only deviates from regular ASP.NET development ever so slightly.  A bit of creativity and ingenuity easily overcomes most hurdles.

As I'm doing development in the area, I'd like to help shed some light on the mystery of it all and show how developers and SharePoint can work together in harmony :-)

One of the first steps to making happy with SharePoint when working with layout pages is to nuke the SharePoint CSS which creates weird padding around the main work area.  I'm sure it may have some purpose, but it's irrelevant for us at the moment.

Let's take a look.

In the following screen, I've added a <div/> element with a one pixel, black border with width and height set to 100% (this is in IE7 and it looks about the same in FF2).

Notice the fat white padding on the top and the right.  This is useless and ugly.  Not only that, we didn't acheive the 100% height that we'd like to.  We can nuke it with some CSS:

/*--- override WSS styles ---*/
body { height: 100%; }
.ms-formareaframe { padding: 2px 4px 4px 4px; background: #fff; height:100%; }
.ms-propertysheet { height: 100%; }
#onetidMainBodyPadding { height: 0px; }
#onetidMainBodyPadding img { height: 0px; display: none; }
#onetidYPadding { width: 0px; }
#onetidYPadding img { width: 0px; display: none; }

And here's the result:

Bingo!  It looks kind of the same in FF2 (FF2 overdraws the 100% declaration and cuts off the bottom and right borders; this does require some CSS hackery to fix).

But the big picture is that now we have a blank slate with which to integrate with SharePoint.  You get all of the benefits of SharePoint authentication, document management, profiling, and so on just by putting your ASP.NET application into SharePoint as a layout page.  Development is easier than it would seem since the package, during development, only has to be deployed once.  Otherwise, all of the files can be deployed to SharePoint using an automated XCOPY or manually.  In some cases, for example working with ASPX page layout, the page can be accessed directly via file share (this is how I do all of my layout).

For more info on creating layout apps, check out the following articles on packaging SharePoint solutions:

# Wednesday, April 02, 2008

SharePoint As A Development Platform

Wednesday, April 02, 2008 1:19:53 PM UTC

I got an email today regarding a blog post by Jeffrey Palermo on the shortcomings of SharePoint as a development platform.

Now I have to say, SharePoint is not without fault (particularly in the area of feature packaging and deployment), but Jeffrey's perceived issues with SharePoint really show either the lack of personal development experience with SharePoint or a lack of creativity on the part of the team he's working with.

Let's get one thing clear first, okay?  SharePoint is meant as an enterprise collaboration and document storage platform, first and foremost.  One must always bear this in mind with regards to any discussion on SharePoint.  Why is it so big?  Why is it clunky in some places?  Why does it have to be installed on a server OS (okay, I admit, this one is probably more about licensing and $$ than anything technical)?

It seems that Jeffrey's major beef, that it must be installed on a server OS, kills any benefits of SharePoint as a development platform immediately.  So let's tackle this one first. 

First, a disclaimer: I'm not a SharePoint guru.  I don't have 5 years of experience with SharePoint.  I'm only going on what I've learned in working with SharePoint, day in and day out, for the last year.

In our development group of 5 or so people, we've managed to develop against SharePoint quite well, despite the fact that we all run XP as our primary development environment.  How?  Virtual machines.  Of course, some would view this as a hassle, as another stumbling block or quite clumsy.  I view it otherwise as there are numerous benefits in developing against a VM server environment:

  1. Every deployment is exactly the same.  This means that server names, file structures, databases, and so on are exactly the same for every developer.  This aids -- not hinders -- the development of automated scripts and deployment utilities.  On our team, we usually synchronize images once every month or so.  What this means is that we'll take turns updating a common image and then everyone will grab a copy of said image.
  2. Virtual Machines are easy to deploy.  It's called: copy-paste!  Nothing easier when adding a new developer to your team.  Instead of configuring a whole new SharePoint environment, just have the developer copy the latest development image.  So easy, a caveman could do it.
  3. Virtual Machines can be rolled back.  Try doing that with your development environment!  If every developer had SharePoint deployed on his or her machine, a mistake (deleting/altering some core database tables or records, for example) would be devastating...it would mean at least a day of lost work time trying to rebuild the development environment.  Using virtual machines to host the server environment protects it from inadvertant dismantling.  And even if it is somehow altered for the worse, no matter, get another copy from someone else!  And of course, being able to rollback the image (provided that you created a snapshot) makes updates (for example, adding a SQL Server service pack) painless.
  4. Virtual Machines are portable.  Why does this matter?  Demos!  There's nothing more convenient than having a sales guy pack up a copy of the VM with all of the software installed and tweaked for demos.  This is a huge bonus.

Look, the suggestion of developing with SharePoint in a "native" environment in a team is just plain stupid; it's a matter of working harder, not smarter and it shows a lack of creativity in terms of development management.  (One disclaimer: some Visual Studio tools from Microsoft for SharePoint cannot be installed on XP...this is a shortcoming, for sure, but it hasn't really affected our development.  If absolutely necessary, you can install a stripped down copy of Visual Studio on the VM).

Now let's address each of the other 7 points that Jeffrey brings up:

  1. SharePoint isn't easy to install.  I'm going to skip that one because clearly, this man has never installed SharePoint.  It's nothing more than point and click...my gosh, I don't understand how it could possibly be easier.  Aside from this, using a VM approach, it only has to be installed once.
  2. SharePoint isn't easy to configure.  See above.
  3. SharePoint does not integrate well with simple tools.  I'm not sure what he means here.  Many administrative tasks can be handled by stsadm.exe which in turn, means that many of the administrative tasks can be handled by batch scripts.  Aside from this, SharePoint is just an average ASP.NET application.  Stopping and starting IIS, xcopy, and so on...these are all still applicable to a SharePoint deployment.
  4. SharePoint isn't easily extended to make simple tools.  What?  Nothing could be further from the truth.  The only beef that I would hold is that Microsoft doesn't package the SharePoint DLLs separately so you have to extract the DLLs from the GAC of your server environment.  But once you do that, it's easy to reference and use the API by copying the DLLs to your development environment.
  5. SharePoint isn't easy to debug.  Again, I'm not sure what the issue here is.  The process of debugging SharePoint applications (web parts, layout pages, etc.) is no harder than debugging any standard, run of the mill ASP.NET web applications.  Okay, sure, you can't just hit F5 (oooh, the horror!), but seriously, I hate F5 developers - sometimes, F5 just isn't the way to go, dude.  I'm also pretty sure chimps (or macros) can be trained to hook up the debugger and hit F10 and F11.  As for debugging core SharePoint, well, I guess he has a point since the source and debugging files aren't available (not that I know of), but I don't see that as an issue.  I mean, how many platforms ship with debugging symbols included?
  6. It's not easy to create test automation for SharePoint.  There is some truth to this since it requires the developer to learn about the platform first (API, database tables, web services, and so on).  But I don't think it's any more difficult or challenging than setting up automated tests against any other third party platform.
  7. SharePoint configuration does not store easily in source control.  See above regarding VMs.  Aside from this, he's got it all wrong again.  We can look at this from a few different angles.  First, SharePoint is just an ASP.NET application.  Repeat that to yourself about 100 times.  With regards to application configuration (in terms of the web.config), it's as simple as copying the web.config to your source control system and using a script to deploy it on build.  Secondly, if we look at the configuration in terms of web parts and layout pages, it's possible to include these in source control as well using solution packages deployed either on build or manually as features.  Certainly requires a bit of research to get it working the first time, but it's not a task that takes more than 1-2 days of experimentation.

From personal experience, I've found SharePoint a very compelling application development platform (again, that's not to say that it doesn't have shortcomings) because it's nothing more than ASP.NET but with the added bonus of a document management/storage API, profiling, permissions, and it acts as an integration platform for a variety of applications. 

You can make it as hard or as easy as you want it to be with regards to developing applications for SharePoint.  It's only a matter of how much time you are willing to put into flipping through the API and understanding the fundamentals of working with SharePoint.  A big part of successfully and painlessly developing against SharePoint is creativity in terms of setting up your development environment and automation (batch scripts, pre/post-build scripts).

The points that Jeffrey, didn't bring up -- the true pain points -- are really "fringe" features so far as I'm concerned.  Namely, this centers around SharePoint hosted and integrated workflows and InfoPath (because no one likes and no one actually uses the otherwise useless and purposeless InfoPath).  You're not required to use InfoPath or SharePoint hosted workflow; in FirstPoint, absent the early documentation and tools required to be productive on this front, we made an early decision to host workflow in our own environment. Sure, we miss out on some of the native features of SharePoint like workflow state visibility and integrated forms via Forms Services, but I don't see it as something we can't overcome as documentation and tool support becomes better on this front.

# Friday, March 28, 2008

SharePoint: The Second Coming Of Lotus Notes?

Friday, March 28, 2008 12:51:33 PM UTC
As I was pondering the suckage of Lotus Notes, I came across an interesting little piece on a CMS Watch Report titled: "SharePoint Has Become the New Lotus Notes":

Microsoft Office SharePoint Server 2007 is repeating history as it mimics the allure and pitfalls of Lotus Notes, according to research released by CMS Watch, an independent analyst firm that evaluates content technologies.

SharePoint exploits traditionally underserved collaboration needs for information workers laboring within Office tools, and fulfills a common desire to easily create disposable workspaces, CMS Watch found.

Like Notes in a previous decade, IT often embraces SharePoint as a simple answer to myriad business information problems.  But the platform can morph into a technical and operational morass, as repositories proliferate, and IT comes to recognize that various custom applications require highly specialized expertise to keep running properly.

...

The SharePoint Report 2008 concludes by advising customers to establish clear boundaries on SharePoint services, to keep it from becoming their new Notes – the platform that everyone loved, but then loved to avoid.

While SharePoint does indeed have it's weaknesses (total lack of any integration with ASP.NET AJAX in the SharePoint implementation itself -- guess we'll just have to wait for 4.0, web services support is still kind of weak) and oddities (CAML?), it's nowhere near the steaming pile that is Lotus Notes.

On a serious note, I do kind of see the point in that last paragraph there.  SharePoint often gets evangelized as some silver bullet for collaboration ("Oh look, workspaces! Workflows! Tasks lists!") but I've never been in an organization that's used SharePoint in a way that was actually of any aid to productivity or collaboration; people just don't seem to want to log onto a corporate SharePoint portal unless they have to. 

That's not to say that the platform doesn't have its useful bits, but the real gem in SharePoint is its integration with Office applications as a platform for "seamless" sharing of documents and I think the idea of offering that to a much larger audience (via Office Live Workspaces) is long overdue from Microsoft.  Until recently, there were few integrated solutions for small businesses, students, and other non-business groups for the very simple act of sharing Office documents aside from using e-mail. 

Even when I joined Zorch Software, I would do a facepalm regularly when I got an email with a document attached with a "v15" suffix.  I'd save it in the same folder as the previous 14 "versions" that I received.  The irony.  The problem is that most of SharePoint just isn't that useful.  Even in a tech minded organization like Zorch Software, you just can't break some people out of old habits; to many, collaboration is synonymous with e-mail.  There is a whole generation that doesn't get wikis and doesn't want to learn wiki markup. 

Well, in any case, I'm still not over the fact that I'm being forced to use Lotus against my will and I'm still bitter over the fact that it's been so hard to get people to embrace our Trac wiki and embrace the ticket system for tracking issues.

# Friday, March 07, 2008

WSS And DateTime Error

Friday, March 07, 2008 5:16:40 PM UTC
In working with the SharePoint web services, I've noticed consistent errors with a few of the services which tend to return the following string:
String was not recognized as a valid DateTime
I hadn't been able to figure out what exactly the error was until today.

It's actually not related to a date/time string at all, but rather, the root of the error is in the service implementation itself.  The actual underlying error is an error indicating that a field value is no longer valid.  For example, if the item had a field referring to a lookup list, changes in the target list may invalidate a value on a given item.  Instead of returning a useful error, the web service returns the error above.

You may find this error when using the lists.asmx service or the dspsts.asmx service.

I "fixed" this by removing the offending field from the content type and also from the list and voila, no more errors.

It's hard to track down because the SharePoint itself will render the properties without any error indication in the view and edit properties view, but it will fail the entire request if one field is invalid and return a useless SOAP error.

# Tuesday, February 26, 2008

SharePoint Layout Pages With CodeBehind And Prototype

Tuesday, February 26, 2008 1:11:26 AM UTC

Let it be known that I hate out of the box ASP.NET.  Hate it, hate it, hate it, hate it.  I detest it.  The simplicity with which it allows the average developer to create applications leads to applications designed for RAD and not for scalability and it does not encourage good decoupling of business logic from UI logic.  Certainly, there are a number of frameworks which aim to alleviate this (the Web Client Software Factory, for example), but I like to take it to another level all together.

Some would argue that I take the separation of UI and application logic to the extreme: my preferred methodology relies almost purely on client side scripts to render UI and using only web services to supply data using ASP.NET AJAX.  Certainly, I lose design time support, but I gain in pure speed (all of the UI logic is in Javascript files which are cached by the client), data transfer sizes (since the only traffic is data, no presentation whatsoever), and the ultimate decoupling of UI development and server component development (the UI developer only needs to know the data model exposed by the web services).

The way I look at it, you'll only write the code a few times, but it could be in use for months (and if you're lucky in this Web 2.0 age, even a year or two).  Sure, you lose some productivity for a single developer with the loss of design time support, but you gain tremendously over time with each request serviced in terms of performance and bytes saved (a particularly important point for high traffic/high data volume applications).  As a bonus, I find it generally easier to think about application design in these terms.

Admittedly, this model seems to work better for "business applications" as opposed to "content applications".

In any case, I was interested to see if this methodology could be applied to SharePoint development as I've been working with SharePoint for quite a while now, but not at the UI level.  SharePoint allows you to deploy "application pages" which can be seemlessly integrated (kind of) into a SharePoint deployment.  This seemed like the perfect starting point to try to integrate ASP.NET AJAX and prototype, one of my favorite Javascript libraries.

The general steps are:

  1. Create an ASP.NET AJAX web application
    1. Add a reference to the Microsoft.SharePoint assembly
    2. Add the prototype.js script file to the project
    3. Add a strong name key file and sign the project
    4. Build a simple page
    5. Create a simple service
  2. Copy the content files (.js, .aspx, .asmx) over to the server to C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\ into a new directory.
  3. Test by visiting the URL: http://myserver/_layouts/mynewdirectory/default.aspx

The base page should be simple.  Use the default page created by the ASP.NET AJAX web application project template and change the base class for the _Default.aspx.cs file to LayoutsPageBase instead of of the default of Page.  If you're not using ReSharper ;-), you'll need to add a using statement to your file:

using System;
using Microsoft.SharePoint.WebControls;

namespace WssAjaxApplicationTest {
    public partial class _Default : LayoutsPageBase {
        protected void Page_Load(
            object sender, EventArgs e) {}
    }
}

Next, you will need to modify the Default.aspx file.  The gist of the modifications comes from an MSDN article by Ted Pattison:

<%@ Page Language="C#" 
    AutoEventWireup="true" 
    CodeBehind="Default.aspx.cs" 
    Inherits="WssAjaxApplicationTest._Default" 
    MasterPageFile="~/_layouts/application.master"%>

<asp:Content ID="Main" runat="server" ContentPlaceHolderID="PlaceHolderMain">
    <script type="text/javascript" src="_scr/prototype.js"></script>
    <script type="text/javascript" src="_scr/WssAjaxApplication.js"></script>
    <script type="text/javascript"> 
        var application;
           
        function Init() {
            application = new WssAjaxApplication();
        }
        
        Event.observe(window, "load", Init, false);
    </script>
    <asp:ScriptManager ID="ScriptManager1" runat="server" >
        <Services>
            <asp:ServiceReference Path="~/Services/EchoService.asmx" />
        </Services>
    </asp:ScriptManager>    
    <div>
        <input type="text" id="message-input" />
        <input type="button" id="action-button" value="Go!" />
        <br />
        <div id="message-output"></div>        
    </div>                
</asp:Content>

<asp:Content ID="PageTitle" 
    runat="server" 
    contentplaceholderid="PlaceHolderPageTitle" >
    Echo Page
</asp:Content>

<asp:Content ID="PageTitleInTitleArea" 
    runat="server" 
    contentplaceholderid="PlaceHolderPageTitleInTitleArea" >
    The Echo Page Test
</asp:Content>

I've bolded the key part above, which is linking to the master page for SharePoint layout application pages.  In addition, you can see that I've created three placeholder content sections with the key section being the PlaceHolderMain.  I've placed my Javascript references and my ScriptManager into this section, pointing to our simple service, EchoService.asmx.  Notice the use of the root squiggly "~" :-D and the lack of squiggly on the script references to prototype.js and WssAjaxApplication.js.

The service I'm going to be using for this demo is a simple "echo service" which just echoes the input string with the server timestamp attached.  The following is my simple implementation of this web service:

using System;
using System.ComponentModel;
using System.Web.Script.Services;
using System.Web.Services;

namespace WssAjaxApplicationTest.Services {
    /// <summary>
    /// Summary description for EchoService
    /// </summary>
    [WebService(Namespace = "http://tempuri.org/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [ToolboxItem(false)]
    [ScriptService]
    public class EchoService : WebService {
        [WebMethod]
        [ScriptMethod]
        public string Echo(string message) {
            message = string.Format("You said: \"{0}\" at {1}", 
                message, DateTime.Now);

            return message;
        }
    }
}

As you can see by the .aspx page above, I've organized my service in a sub-folder called "Services".  Now just make sure that you've added a copy of prototype.js under the _scr directory and my application script:

WssAjaxApplication = Class.create();

Object.extend(WssAjaxApplication.prototype, {
    initialize:function() {
        this.MessageInput = $("message-input");
        this.ActionButton = $("action-button");
        this.MessageOutput = $("message-output");
        
        Event.observe($('action-button'), "click", 
            this.OnClickActionButton.bindAsEventListener(this), false);
    },
    
    OnClickActionButton:function(e) {
        if(e) { Event.stop(e); } // Stop the event       
        
        // Perform the echo.
        WssAjaxApplicationTest.Services.EchoService.Echo(
            this.MessageInput.value,
            this.OnClickActionButtonSuccess.bindAsEventListener(this),
            this.OnClickActionButtonError.bindAsEventListener(this)
        );          
    },
    
    OnClickActionButtonSuccess:function(result) {
        this.MessageOutput.innerHTML = result;
    },
    
    OnClickActionButtonError:function(error, userContext, methodName) {
        window.alert(methodName + 
            " failed with the message: " + error.get_message());
    }
});

The script simply attachs an event listener to the "Go" button and handles the click event.  Notice how clean and simple the HTML portion of the page is and how clean the Javascript is as well (admittedly, this is a very simple example).  The client rendering is completely decoupled from the UI logic except for the data and operations contract. 

You should be good to go so far as code goes.  Now compile your project with a strong named key file. 

Hopefully, the project was compiled successfully.  The next step is to copy the output dll to the GAC of the SharePoint server.  Be sure to note the public key token value.

This is probably the trickiest part: now you need to carefully merge the configuration files (is there a better tool to do this with?) generated by the project template with the web.config file located at the virtual directory root of your SharePoint application.  For example, if you have an application deployed at port 8080, the web.config file should be located at C:\Inetpub\wwwroot\wss\VirtualDirectories\8080.  Be sure to save a backup copy of the configuration file first before you attempt to merge it!

Once merged, you will need to add one more element to the configuration file:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration> 
    <system.web>    
        <compilation batch="false" debug="false">
            <assemblies>
                <add assembly="Microsoft.SharePoint, 
                    Version=12.0.0.0, Culture=neutral, 
                    PublicKeyToken=71e9bce111e9429c" />
                <add assembly="System.Web.Extensions, 
                    Version=1.0.61025.0, Culture=neutral, 
                    PublicKeyToken=31bf3856ad364e35"/>
                <add assembly="WssAjaxApplicationTest,
                    Version=1.0.0.0, Culture=neutral, 
                    PublicKeyToken=97d3f1fd9f5212b9"/>            
            </assemblies>
        </compilation>
    </system.web>
</configuration>

I've highlighted the key line (the line above it should have been merged into the file previously).  The bolded entry above is for the web application binary.

To test whether you've succeeded, you can simply point your browser to the URL: http://myserver/_layouts/mywebapp/default.aspx and you will have a fully AJAX enabled application using ASP.NET AJAX to connect to a .NET web service with prototype as a general purpose Javascript utility library (and you can even add scriptaculous on top of that for more awesome).

I've included a self extracting 7z file of the solution (see link below) if you'd like a quick start.  Note that the Microsoft SharePoint binaries are not included and you will have to add them back manually before the project will build.

Happy coding!

WssAjaxApplicationTest.exe (162.02 KB)
# Tuesday, August 14, 2007

Normalizing And Denormalizing SharePoint Field Names

Tuesday, August 14, 2007 3:45:29 PM UTC

Frequently, when working with Office, SharePoint, and SharePoint web services, it is necessary to convert between the "normalized" (hex escaped string) version of a field name.

To that end, I found a useful JavaScript tool for normalizing strings into SharePoint's "static name" format.

In .Net, we can simplify this using Regex and Uri.

    private static readonly Regex specialCharactersPattern
        = new Regex("[\\[*($%&)<>!?\\/\"{}\\s+-='@~`#\\\\:;^\\]]", 
            RegexOptions.Compiled);

    private static readonly Regex encodedCharactersPattern
        = new Regex("_x00(\\d{2})_", RegexOptions.Compiled);

    /// <summary>
    /// Normalizes the name of the SharePoint property name.
    /// </summary>
    /// <param name="key">The "human readable" key/property name.</param>
    /// <returns>
    /// The string with hex representation in place of special ASCII
    /// characters.
    /// </returns>
    private static string NormalizeSharePointPropertyName(string key) {
        return specialCharactersPattern.Replace(key, ReplaceSpecialCharacter);
    }

    /// <summary>
    /// Custom match evaluator to replace special characters within the input
    /// string.
    /// </summary>
    /// <param name="match">The pattern match.</param>
    /// <returns>The formatted version of the string.</returns>
    private static string ReplaceSpecialCharacter(Match match) {
        string replacement = string.Format("_x00{0}_",
            Uri.HexEscape(match.Value[0]).TrimStart('%'));

        return replacement;
    }

    /// <summary>
    /// Denormalizes the name of the SharePoint property.
    /// </summary>
    /// <param name="key">The normalized key/property name (static name).</param>
    /// <returns>
    /// The denormalized form of the input string ("human readable"
    /// with hex patterns replaced).
    /// </returns>
    private static string DenormalizeSharePointPropertyName(string key) {
        return encodedCharactersPattern.Replace(key, DecodeSpecialCharacter);
    }

    /// <summary>
    /// Custom match evaluator to replace the hex characters with special characters.
    /// </summary>
    /// <param name="match">The pattern match.</param>
    /// <returns>The ASCII format of the special character encoded in hex.</returns>
    private static string DecodeSpecialCharacter(Match match) {
        int start = 0;

        string replacement =
            Convert.ToString(
                Uri.HexUnescape(Convert.ToString(match.Value[0]), ref start)
            );

        return replacement;
    }

Download the sample console project to test it out.

SharePointNormalizationConsole.zip (4.35 KB)

# Sunday, June 17, 2007

Working With SharePoint Web Services

Sunday, June 17, 2007 11:03:15 PM UTC

One of the most confounding things about working with the SharePoint web services is that the return values are all in XML strings (wrapped in an XmlNode).

To make working with the services even more puzzling, suppose you get a result like so:

<ContentType 
    ID="0x010100347433A509750F4D88880291599D314D04" 
    Name="My Content Type" 
    Group="My Custom Content Types" 
    Version="7" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
    <Folder TargetName="_cts/My Content Type" />
    <Fields>
        <Field ... />
        <Field ... />
        <Field ... />
        <Field ... />
    </Fields>
    <XmlDocuments>
        <XmlDocument 
         NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">
            <FormTemplates 
                xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">
                <Display>DocumentLibraryForm</Display>
                <Edit>DocumentLibraryForm</Edit>
                <New>DocumentLibraryForm</New>
            </FormTemplates>
        </XmlDocument>
    </XmlDocuments>
</ContentType>

You would expect to be able to retrieve all of the <Field /> elements by executing the following code:

XmlNodeList nodes = xmlResponse.SelectNodes("//Field");

But it's not quite so straight forward.  You actually have to instantiate an XmlNamespaceManager with a bogus prefix:

NameTable table = new NameTable();
XmlNamespaceManager manager = new XmlNamespaceManager(table);
manager.AddNamespace("sp", "http://schemas.microsoft.com/sharepoint/soap/");

XmlNodeList nodes = xmlResponse.SelectNodes("//sp:Field", manager);

In this case, I used "sp" as my prefix. Notice that it's utilized in the XPath query as well. 

# Thursday, May 31, 2007

SharePoint SoapServerException When Using Lists Service

Thursday, May 31, 2007 8:19:43 PM UTC

When using the lists service to query the "User Information List" (the SharePoint list where the users and groups is located), you may encounter the exception:

System.Web.Services.Protocols.SoapException:
    Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException'
    was thrown.
  at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(
      SoapClientMessage message,
      WebResponse response,
      Stream responseStream,
      Boolean asyncCall)
  at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(
      String methodName,
      Object[] parameters)
  at ListsDataServiceSample.SharePoint.Lists.Lists.GetList(String listName) in
      F:\Projects\Sandbox\SharePointWebServicesSample\ListsDataServiceSample\
          Web References\SharePoint.Lists\Reference.cs:line 213
  at ListsDataServiceSample.Program.Run() in
      F:\Projects\Sandbox\SharePointWebServicesSample\ListsDataServiceSample\
          Program.cs:line 28

This error will occur if you are not using the "Administrator" account, which of course, is not always ideal, especially if you are writing a client side app.

The solution to fix this is to grant the account permissions on the "User Information List":

And then give the user "Read" permissions on the list:

# Wednesday, March 21, 2007

Most Convoluted Licensing Model. Ever.

Wednesday, March 21, 2007 12:33:57 AM UTC

See if you can figure out how much your company's deployment of MOSS2007 will cost:

Mindblowingly convoluted.

# Saturday, January 13, 2007

Adding Users To A Document Workspace

Saturday, January 13, 2007 11:49:08 PM UTC

In WSS3, the process of adding users to a document workspace (or any sub-web) has changed from WSS2.  The following snippet will allow you to add a user (I've only tested with users mapped to domain accounts) to the workspace:

using(SPSite site = new SPSite("http://ashelia:2345")) {
    using(SPWeb workspace = site.OpenWeb()) {
        string resourceLogin = "ASHELIA\\cchen";

        // Ensure that the user exists and conveniently, get
        // an SPUser reference.
        SPUser user = workspace.EnsureUser(resourceLogin);

        // Create a new SPRoleAssignment for the user.
        SPRoleAssignment assignment =
            new SPRoleAssignment(
                user.LoginName, user.LoginName,
                user.Name, user.Notes
            );

        // Add the "Contribute" role definition to the role
        // assignment.
        assignment.RoleDefinitionBindings.Add(
            workspace.RoleDefinitions["Contribute"]
        );

        // Add the assignment to the web.
        workspace.RoleAssignments.Add(assignment);

        // Update the web.
        workspace.Update();
    }
}

Note that when you create a new sub-web, by default, there are 5 role definitions defined for you already.  These are:

  1. Full Control
  2. Design
  3. Contribute
  4. Read
  5. Limited Access
# Monday, January 08, 2007

ContentTypeIds In WSS3

Monday, January 08, 2007 9:50:43 PM UTC

In WSS3, if you execute the following SQL:

    SELECT
        ContentTypeId
    FROM 
        ContentTypes

You will notice that the content types are represented in the output as hex.  If you take a look at the table definition, you'll see that the actual data type of the column is VARBINARY(512).

Doing a lookup like so:

    SELECT
        ContentTypeId
    FROM 
        ContentTypes
    WHERE 
        ContentTypeId = '0x101'

Will not work since you cannot perform a comparison between VARBINARY and a character data type directly.

Doing the following will also not work:

    SELECT
        ContentTypeId
    FROM 
        ContentTypes
    WHERE 
        CAST(ContentTypeId AS VARCHAR(512)) = '0x101'

This doesn't work because the underlying type of the binary data isn't character data.  It's integer data.  You can confirm this by running the following query:

    SELECT
        ContentTypeId,
        CAST(ContentTypeId AS VARCHAR(512))
    FROM 
        ContentTypes

You'll see that it's just a bunch of gibberish.  Try the same query with INT and you'll see that the data makes much more sense.  What you'll notice is that content types that inherit from a base content type will have numerical values that increment by 1.

This information is useful, but not nearly as useful as the data that you can glean from the hex string representation of the ContentTypeId.  You see, in the hex string representation, the base ID is a substring of the ID of any inheriting content type.  For example, if I have a content type which has a ID (as a hex string) of 0x0101345346345312234346, then any child content types will have 0x0101345346345312234346 as a substring (e.g. 0x010134534634531223434601, 0x010134534634531223434602).

So how do we get this data in SQL Server for comparison purposes?  We need to use an "undocumented" SQL function: fn_varbintohexstr().

This allows you to do nifty queries to find a given content type and all child content types (or any query where you have to retrieve information about a hierarchy of content types) like so:

    SELECT
        *
    FROM 
        ContentTypes
    WHERE 
        master.dbo.fn_varbintohexstr(ContentTypeId) LIKE 
            '0x0101345346345312234346%'

You can find out more information on this function here.

# Sunday, January 07, 2007

A Note On Copying Files In WSS3

Sunday, January 07, 2007 7:52:33 PM UTC

I dunno if this was supported in WSS2 or not, but in WSS3, when a file is copied to a new destination, a link is stored which indicates where the new document is copied from.

In the database, you can see this by running the query:

SELECT
    tp_dirname,
    tp_leafname,
    tp_copysource,
    tp_hascopydestinations,
    tp_guid,
    *
FROM ALLUSERDATA

The tp_CopySource column holds the URL of the source document from which the given file is copied from.  If you simply change the URL here, you can point it to any file you want.  If the file is a copy, then the following information bar will be displayed on top of the file properties in the properties page:

copy-indicator.jpg

However, it's not so obvious how to do this programmtically.  First, I tried using the CopyTo() method of SPFile.  Aside from not having the desired effect, this method does not seem to allow copying files across site boundaries (for example, from a root site to a document workspace -- for that, you have to use the Add() method on the Files property of the target SPWeb).

On my second attempt, I tried to set the "Copy Source" property of the file.

foreach (object key in file.Properties.Keys) {
    Console.Out.WriteLine("{0} : {1}", key, file.Properties[key]);
}

Iterating through the properties of an SPFile instance, I found that one of the properties, was "Copy Source" (internal name of "_CopySource") and in fact, held the URL of the source document.  I tried to set this value and update the file, but this was unsuccessful yet again.

On my third attempt, I came across the CopyTo() method on the SPListItem class and this did it for me :-) Conveniently, it also allows you to copy an item across site boundaries.

I felt so stupid afterwards because it should have been obvious that I had to use the CopyTo() method on the SPItem because the SPItem is also where the UnlinkFromCopySource() method is located.

RSS 2.0 Atom 1.0 CDF