Random Thoughts of a Scatterbrain.
 Friday, August 08, 2008

Fatal Execution Engine Failure(0x7927e03e) and WCF

8/8/2008 7:48:13 PM (Eastern Daylight Time, UTC-04:00)

An interesting issue cropped up during development this week.  It centered around a mysterious error that was happening in our web applications and causing IIS to crash...hard. 

Peeking in the event log, I found the following error:

Event Type:	Error
Event Source:	.NET Runtime
Event Category:	None
Event ID:	1023
Date:		8/8/2008
Time:		7:07:28 PM
User:		N/A
Computer:	ZORCH6
Description:
.NET Runtime version 2.0.50727.1433 - Fatal Execution Engine Error (79FFEE24) (80131506)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

It was mind boggling even more so because the app worked on some environments and not others.  We chased various options around and none of them seemed to solve the issue.  Further confounding the issue was that IE would throw up an authentication dialog instead of outright generating an error on the AJAX call where the error was originating from.  This lead us in circles fiddling with the domain controller and permissions for a while, too.

There was no concrete info anywhere in the system which explained why we were getting this error.  Then I came across a post in the Microsoft Connect site which mentioned this issue.  I wasn't completely sure that this was the same as our issue, since we weren't using IEnumerable<T> in our contracts, but it was a lead.  So I built a small sample project to model our service calls.

It turns out that there's something borked with WCF data contract serialization in IIS when running in Windows 2003 environments.

In our case, we did not have IEnumerable<T> in our contracts (well, not explicitly anyways).  However, we had a few methods which took Collection<T> as an argument.  This worked all fine and dandy when sending a normal collection like so:

Collection<Person> c1 = new Collection<Person>();

Person thuy = new Person("Thuy", "Ha", "UT");

c1.Add(thuy);
client.ProcessPeople(c1);

Collection<Person> c2 = new Collection<Person>();

client.ProcessPeople(c2);

However, using this approach breaks it (I repeat, this does not work):

Person chuck = new Person("Charles", "Chen", "NJ");
Person[] a3 = new Person[1];
a3[0] = chuck;

client.ProcessPeople(new Collection<Person>(a3));

To work around this, you can use an intermediate step like so (of course, this is only necessary if you have data coming from an array to begin with):

Person chuck = new Person("Charles", "Chen", "NJ");
Person[] a3 = new Person[1];
a3[0] = chuck;

List<Person> l3 = new List<Person>();
l3.AddRange(a3);

client.ProcessPeople(new Collection<Person>(l3));

In any case, what further confounded the issue is that it worked in an XP environment but did not work in a server 2003 environment.  Glad to have this issue behind me :-D; hopefully, you won't waste as much time fiddling with this as I did.

 Tuesday, July 15, 2008

Deleting ExtensionData From JavaScript

7/15/2008 12:06:07 PM (Eastern Daylight Time, UTC-04:00)

WCF DataContracts include an ExtensionData property which is a bit troublesome if you are trying to send a modified object back up to the service if it has no properties in Javascript.

So I wrote a little piece of Javascript to clean it up:

function DeleteExtensionData(obj) {
	var keys = Object.keys(obj);

	keys.each(function(key) {
		if(!Object.isFunction(obj[key])) {
			if(obj[key] instanceof Object) {
				DeleteExtensionData(obj[key]);
			}

			if(key == "ExtensionData") {
				delete obj[key];
			}
		}
	});
}

It will recusively delete all ExtensionData properties from the object.  You can call as soon as you get the result from a completed AJAX request or you can call before sending an object parameter to a service.

Note that it uses constructs from prototype.

If you want to get fancy, you can also write a custom serializer.

 Thursday, June 19, 2008

IE7, AJAX, And 400 "Bad Request"

6/19/2008 6:11:27 PM (Eastern Daylight Time, UTC-04:00)

I spent bout two hours today tring to figure out why I kept getting a weird error while using prototype to call a WCF service.

There are various other resources on the web regarding how WCF has a few quirks with error handling using the default endpoints, however, none of these scenarios were applicable to me since the JavaScript call worked fine in FireFox.  To complicate things even further, it actually all worked fine in IE7 as well with one caveat: it only worked on the first load of the IE7 process.

After that, any refreshing of the page would return 400/"Bad Request" errors, mysteriously.  Just to make sure that it wasn't isolated to prototype, I also tried jQuery as well.  Still nothing.

Well, as it turns out, that setting the AJAX request content type to application/json wasn't enough; I had to set it to application/json; charset=utf-8.

So if you're encountering weird issues with any AJAX library, IE7, and WCF and/or ASP.NET, be sure to check your content type setting.

Update 6/20/2008: bah!  I woke up this morning and it's not working again!  Oddly, it works fine if I have Fiddler running and as soon as I shut off Fiddler, it stops working.  To complicate matters even more, it's only happening on methods where I have post content.

Update 6/22/2008: after further investigation, I'm concluding that it's related to SharePoint.  To isolate the different components, I created a custom WCF service and a simple runtime which models our current custom WCF runtime.  When I used a plain HTML page or even an ASPX page with a script manager on it, prototype AJAX requests worked fine.  Okay, so it wasn't how we were hosting the service and it wasn't an issue with a collision between ASP.NET AJAX and prototype.  I then hooked a SharePoint layout page up to the same service and boom, it all breaks :-S

I've broken it out into the following scenarios:

  1. Using FireFox, the requests always work.  No matter how many times I refresh the page, the AJAX request always works fine.
  2. Using IE7, it will always work on the first load of the IE process. 
    1. If I hit CTRL+F5, it will continue to work.
    2. If at any point, I hit F5, it will fail and even CTRL+F5 will not fix it.
    3. If I have Fiddler running, it will always work, even if it entered a failure state after hitting F5! (So Fiddler must be doing something to the HTTP message??). As far as I can tell, the only difference between the Fiddler request and the native request, from viewing the WireShark trace, is the "Authorization" header and the fact that the failed request has a "Content-length" header value of 0 (manually setting the header doesn't work either).  I verified that prototype was not somehow mangling the POST body content by writing a trace of the XHR request body right before and right after the request is sent (both came out okay).

One of the really weird things about this error is that it cannot be observed when I have Fiddler running; somehow, Fiddler "fixes" the issue.  I had to hook up WireShark on the server and watch the raw TCP messages to finally see that on the unsuccessful attempts, the POST content to my WCF service was empty.  The method required one parameter, so the WCF serializer threw a formatter excpetion when it received an empty message:

The server encountered an error processing the request. The exception message is 'Error in deserializing body of request message for operation 'GetRoutes'. The OperationFormatter could not deserialize any information from the Message because the Message is empty (IsEmpty = true).'. See server logs for more details. The exception stack trace is:at System.ServiceModel.Dispatcher.PrimitiveOperationFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.DemultiplexingDispatchMessageFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.UriTemplateDispatchFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.CompositeDispatchFormatter.DeserializeRequest(Message message, Object[] parameters) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.DeserializeInputs(MessageRpc& rpc) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc) at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

I've tried playing around with various cache avoidance strategies to no avail (setting the "Cache-Control" header, setting the "If-Modified-Since" header, setting the "Pragma" header, addin randomness to the POST URL) so for now, I'm just going to have to give up on this.  I've confirme the error using three different OS environments (XP SP2 (desktop), XP64 SP2 (dev), and 2003 R2 SP2 (VM)), all running IE 7.0.5730.11.

I know there are various articles out there regarding IE issues with XHR and how the ordering of the open(), onreadystatechange, and send() operations must be called; however, I verified in the prototype source (lines 1213-1223) that the correct order of operations are being performed.

Thinking that perhaps some Microsoft mojo was behind all of this, I gave the Sys.Net.WebRequest a shot.  This also did not yield any postive results.  Oddly, ASP.NET AJAX to web service calls all work fine (we had some existing AJAX components which were calling to a web service, but the goal was to transition away from writing these "empty" proxy services and call directly to the WCF service).

If any SharePoint development team members or program managers are reading this, please look into it!  I'm not sure if it affects IIS hosted WCF services at this point, but a sample project can be downloaded from here: WcfAjaxWonkiness.zip.  Create a layout page which calls the service and add it to SharePoint and observe it blowing up.

 Monday, June 09, 2008

WCF, Prototype, And Awesome-sauce

6/9/2008 4:00:43 PM (Eastern Daylight Time, UTC-04:00)

One of the most awesome features of WCF in .NET 3.5 is support for REST-style AJAX requests to the services.

Rick Strahl has an excellent post on how to get it up and running with jQuery.

Just some notes on working with it myself which I hope can help clarify some some issues:

  • When using POST and the postBody option with Ajax.Request, the format must be {"argument-name":"value"}.  I struggled with this for a while using single quotes and no quotes.  For example: {arg:'hello'}, {'arg':'hello'}, {arg:"hello"} will all result in 400 "Bad Request" errors.
  • The contentType parameter must be set to "application/json".
  • Be sure to decorate the service contract using the WebInvokeAttribute with ResponseFormat and RequestFormat set to WebMessageFormat.Json.

You can download a sample project from here: WcfAjaxSample.zip (3.83 KB)

RSS 2.0 Atom 1.0 CDF