Four More Months!

It's mah birfday!

From this morning:

Brad says:
FYI - Lotus can only search people by their last name.  Lame.
Keyboard Jockey © says:
If you looked up lame in the dictionary, there'd be a screenshot of Lotus Notes
Brad says:
And that would be a compliment

Now the issue isn't whether it's possible to do it or not because the fact of the matter is that even if it is possible, it's so obtuse that it's difficult to figure it out without consulting a manual. 

An interesting issue cropped up during development this week.  It centered around a mysterious error that was happening in our web applications and causing IIS to crash...hard. 

Peeking in the event log, I found the following error:

Event Type:	Error
Event Source:	.NET Runtime
Event Category:	None
Event ID:	1023
Date:		8/8/2008
Time:		7:07:28 PM
User:		N/A
Computer:	ZORCH6
Description:
.NET Runtime version 2.0.50727.1433 - Fatal Execution Engine Error (79FFEE24) (80131506)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

It was mind boggling even more so because the app worked on some environments and not others.  We chased various options around and none of them seemed to solve the issue.  Further confounding the issue was that IE would throw up an authentication dialog instead of outright generating an error on the AJAX call where the error was originating from.  This lead us in circles fiddling with the domain controller and permissions for a while, too.

There was no concrete info anywhere in the system which explained why we were getting this error.  Then I came across a post in the Microsoft Connect site which mentioned this issue.  I wasn't completely sure that this was the same as our issue, since we weren't using IEnumerable<T> in our contracts, but it was a lead.  So I built a small sample project to model our service calls.

It turns out that there's something borked with WCF data contract serialization in IIS when running in Windows 2003 environments.

In our case, we did not have IEnumerable<T> in our contracts (well, not explicitly anyways).  However, we had a few methods which took Collection<T> as an argument.  This worked all fine and dandy when sending a normal collection like so:

Collection<Person> c1 = new Collection<Person>();

Person thuy = new Person("Thuy", "Ha", "UT");

c1.Add(thuy);
client.ProcessPeople(c1);

Collection<Person> c2 = new Collection<Person>();

client.ProcessPeople(c2);

However, using this approach breaks it (I repeat, this does not work):

Person chuck = new Person("Charles", "Chen", "NJ");
Person[] a3 = new Person[1];
a3[0] = chuck;

client.ProcessPeople(new Collection<Person>(a3));

To work around this, you can use an intermediate step like so (of course, this is only necessary if you have data coming from an array to begin with):

Person chuck = new Person("Charles", "Chen", "NJ");
Person[] a3 = new Person[1];
a3[0] = chuck;

List<Person> l3 = new List<Person>();
l3.AddRange(a3);

client.ProcessPeople(new Collection<Person>(l3));

In any case, what further confounded the issue is that it worked in an XP environment but did not work in a server 2003 environment.  Glad to have this issue behind me ; hopefully, you won't waste as much time fiddling with this as I did.

One mistake that I see time and time again, even from experienced developers, is trying to model new ideas, new techniques, and/or new designs directly into a working codebase. 

In my own experience, I've come to find this to be a bad approach.  In particular:

1. I have to work around existing errors, bugs, and other conditions in the code.  When this happens, it becomes difficult to figure out what I'm doing wrong.  Is it the new code?  Did I introduce a new bug? 
2. I have to worry about breaking existing code.  Sure, I can make a branch to work in, but sometimes it's not worth the effort to test some simple design ideas.  You never want to test out new concepts in working code without trying it in another environment first.
3. It makes it hard for others to review the code.  Without checking it in and without creating a branch, it's hard to share the code with your coworkers for review or for testing.  It's too much to ask a coworker to download a branch to review a bit of code.  Also, if you're working in the branch, you may end up checking in broken code if you need someone to help you fix a bug.

In our environment, we have a repository specifically for what I call "sandbox" code.  Basically, it consists of small standalone models of the more complex designs in the main code base.  I use the sandbox to test design ideas in a simplified environment with less noise and interference.  In addition, if I need to have a coworker review the design, I can check it into the sandbox and not worry about breaking the main trunk or integration branch.  In general, it's just easier to work with; you're not sifting through a huge project to find code, the projects load fast and compile fast, and it's easy to test and run multiple dev-test cycles.

It's true, sometimes, to properly model a scenario, I may have to build out a lot of the code, but I think it's ultimately worth it to have a simplified environment to test and build in over time.  It makes it easier to think about the concept and design assumptions as well as to think about architecture...there's just less moving pieces to think about in such a scenario.

Next time you get stuck looking at a complex piece of code or refactoring or designing around a tricky scenario, try simplifying pulling out the code into a sandbox project and model.  It'll help give a new perspective on the problem and let you think about it in much simpler terms.

I've been working with Python for the last few weeks on a pet project that I have going on.  I've been using PyDev with Eclipse and using Django as the application framework for this web app.  Part of it is because I'm kind of disillusioned with ASP.NET (Django is very clean, out of the box) and part of it is because it's a challenge and it's been fun to switch gears away from the daily grind.

If there is one lesson I've learned, it's that Microsoft developers are spoiled rotten.  Visual Studio is such an excellent development environment with so much flexibility that it makes it so easy, that perhaps there are more .NET developers than should have a right to call themselves professionals.  Switching to Eclipse has been a revalation in what I've been missing since I've started using Visual Studio and ReSharper.

I know this feeling; it's the same feeling I had when I was in college, writing Java in vi and emacs.  No drag and drop (not that I use this much in VS anyways as I've almost completely sworn off control based ASP.NET development).  No hand holding.  It's a raw and visceral programming experience.  It's hard to explain; it comes out kind of like a knock against Eclipse and PyDev, but a more fitting comparison is like the difference between shooting a semi-automatic rifle and a bolt action rifle; there's just something about the bolt action that makes the experience much more enjoyable.

In truth, it's been very challenging and not at all easy to transition.  At the same time, it's been fun and rewarding.  While I feel less productive with Eclipse at the moment due to the newness of it all, I do feel more productive with Python and the runtime interpreter than C#.

I've come to find that Django, while much, much better than ASP.NET in some respects, lacks a lot of the conveniences and facets that I've come to expect from a modern web framework like a rich and easy to use, out of the box JSON serialization library or pattern (it'll get there some day).  ASP.NET makes this nearly painless while the JSON serializer which ships with Django isn't really suited for working with arbitrary objects (I've been using jsonpickle instead).  PyDev and PyDev Extensions are also no match for ReSharper (I dunno, as a random guess, I would say that R# affords a 30-40% improvement in productivity).

So, that's the story so far.  Perhaps when I have more time, I'll round up some of the resources I've used to ramp up on Python development.

Once in a while, I toss my resume out there on Dice to see what the market's like and what opportunities are out there; you never know, right?

It always amazes me how terrible the process is and how it's so hard to find a job that's just right. Among the top of my lists for pet peeves -- in so far as head hunters are concerned -- are:

• A total lack of reading skills. I put the resume and profile together to help save time not just for me, but also for the head hunters. No need to send me a requirement for a position in California, you know, espeically when I've listed NYC and Princeton as my preferred locations.
• A total lack of courtesy. While my profile asks nicely to use email as the preferred contact method, that doesn't stop head hunters from calling. More annoying is that some just keep on talking, and talking, and talking...for me, it's a waste of time to listen to them read requiremets to me; I can read just fine, thank you. 
• Asking invasive questions. I always get the "So how much are you making now?" question and I hate it. None of your damn business buddy! Your client can either meet the salary requirements or not...why in the world should I tell some total stranger how much I make? The bottom line is that whatever my current salary is has no bearing in the conversation. If a client can't meet my salary requiremets and the job just isn't that interesting, I'm not interested.  Not only that, it's not like whatever number I give can be verified easily; asking this question is pointless because you'll get a batch of people who'll just make up numbers anyways.  What if I said I made a million dollars last year?  Would anyone believe me?  What if I said I made $500,000 last year, would anyone believe me?  What if I said I made $250,000 last year, would anyone believe me?  Then why should anyone believe any unverified income number over the phone (isn't this how we got into this sub-prime mess)?

Well, anyways, once you make it past that morass, then you have to deal with the actual companies and phone interviews. This is where the fun begins (no, really)! I, for one, love being teched out. There is nothing more enjoyable than a match of wits to see if the person on the other end of the line can actually out tech me (and believe me, it would make me incredibly happy and excited if that were the case).

My new favorite part of the process is when the interviewer gets to the "So, do you have any questions for me?" part. Instead of asking boring, standard fare type questions, I've decided that this is my opening to gauge the technical skill of the developers in the organization.  There is positively nothing more satisfying than doing some in depth tech grilling to kind of figure out whether an environment is right for you. I've kind of come to the conclusion that I can really only be happy where I can be out-teched; you know, an environment where I can learn from those around me and drive me to continue to dive into the technologies.  It's a way to make sure that you're going to end up in a position where you'll feel challenged and look forward to learning and solving new problems.

In any case, here are some grilling points which I've come up with:

What's your approach to designing data access?

I like this question because it tells me a lot about the interviewer. In my opinion, data access is essentially the core of any application; it must be simple in design and easy to extend. It must be easy to understand and easy to use by the application layers above. Yet it must not be so basic that it's raw and verbose.

The worst possible answer is a response with any mentioning of datasets (even worse - and an absolute deal breaker - if that's not prefixed with "strongly typed").

Ideally, I'd like to hear something like:

• "We usually build a domain model built on top of Microsoft Enterprise Library." Most companies don't have the liberty of working with open source libraries. Enterprise Library is at least a baseline. The great thing about it is that it's very well documented and a well understood quantity.  Pluse, having EL as a base encourages or enables at least a baseline level of uniformity in the code.
• "We use NHibernate (or substitute another ORM/persistence library)." I realize that not all companies and all projects have the freedom to use open source 3rd party libraries, but it's nice to see if they do or have used them in the past.
• "We're using LINQ." If you get a response like this, you know you're dealing with a group on the cutting edge of technologies and you're dealing with a group that doesn't mind the challenge of designing around new technologies; the developers probably read up on this stuff and work on it in their spare time. This is a group that you want to work with.

The answers to this question offer a rich view into the development resources that a company has and whether they have a strong indication of the Not Invented Here syndrome.

I tend to think that it takes a more advanced developer and development team to understand the landscape of libraries out there and how to utilize them since learning a new library is usually far more challenging, not to mention productive, than hacking together an inferior custom solution. As such, I also like to ask the following:

What's your approach to runtime logging?

It's a shock to me that many consultants I've worked with in the past either:

• Incorporate no runtime logging or tracing capabilities into their code or...
• Roll their own logging library.

If you've been writing applications without logging, then you haven't written any applications of any worth. If you're rolling your own, it means you're not interested in delivering value to your clients by wasting their time and money or, even worse, you don't know any better; you've never spent the time to look into the various off the shelf logging options. log4net would be a great answer, but Microsoft Enterprise Library Logging Application Block would be awesome as well.

I've never gotten to a point yet where I've been able to pop this next question, but I think that nothing would be a better indication of "this is a place where I want to work and these are people that I want to work with" than if I got a satisfying answer to this:

Are you familiar with Inversion of Control/Dependency Injection? Do you use any libraries to implement it?

This is a great quesiton because there are some design challenges in software, particularly around extensibility and orthogonality, that can really only be cleanly addressed by using the Inversion of Control (IoC) pattern. It enables the creation of far more extensible frameworks, libraries, and applications.

Low level developers on simple projects have no need for IoC or they end up writing a lot of code that's not extensible or modular. The more complex the application and the greater the need for extensibility, the more important IoC becomes as part of the glue that makes it all work together without a huge mess of dependencies.

If you get a response of "We use Spring.NET/Castle Project/Unity/CAB", then you know you're not dealing with some junior programmers. (CAB isn't really an IoC/DI container, but it utilizes some of the concepts of IoC).

Speaking of late binding, I'm also thinking this would be a good question as well:

Are you familiar with the Fusion Log and why you would need to use it?

Only developers who've worked extensively with late binding would ever have a need to enable this and you can tell the experience level of a developer if he/she can even give you a straight answer on what late binding is.

If I could make it past that point, I'd pretty much surely ask:

What's the difference between an interface and an abstract class? Which do you prefer when you design a framework or application? Why would you choose an interface over an abstract class? Why would you choose an abstract class over an interface?

In general, I don't like being asked or asking low level nitty-gritty questions like "Can you explain how CLR garbage collection works?" or "How many generations does the CLR GC have?" or questions like "Explain the ASP.NET page lifecycle?"; these questions aren't useful in the big picture and most of it can be looked up.  These are just mere facts, the knowledge of which, doesn't indicate much.

On the otherhand, knowing the similarities and differences in iterfaces and abstract classes and how to use them properly gives insight into a developers approach to object oriented programming.  It's a great question and a tough one as well. Nothing would be more awesome than a reply of "Well, according to Cwalina and Abrams in 'Framework Design Guidelines'...".  Knowledge of interfaces and abstract classes is foundational to an understanding of good object oriented design and programming. You cannot write a well designed object oriented system without the judicious use of abstractions.

Another great question along these lines is:

Do you or does your team use any code generation tool?

I think that in general, it takes a great deal of thought to utilize code generation.  It means that the developer or team in question understands the value proposition that it brings to development.  Of note:

• It leads to more consistent code which means that in the long run, it's more maintainable and easier to document; it leads to repeatable and predictable results from every developer on the team, regardless of whether they've been writing code professionally for 10 years or 10 months.
• It leads to less error prone code since it's easy to fix small errors across the board by fixing the templates or the driver.  For example, writing data contracts by hand is extremely error prone since it's easy to forget to put a [DataMember] attribute on property which needs to be serialized.  Generating it from a template mitigates these types of simple mistakes.
• It increases productivity by allowing developers to get away from writing the plumbing and focusing on the business logic and UI, places where the ROI on code generation is lower.

It indicates to me that the developer or team is able to bring abstractions to the next level; not only are they abstracting in their object models, they are also abstracting the very act of coding.  To successfully utilize code generation means that the developer or team is able to see the big picture and not just a class here and a class there.  The develper or team has analyzed the code, identified the patterns, and encapsulated those patterns into templates and drivers.  You know you're dealing with a high level team if they properly utilize code generation tools to reduce the amount of time spent doing grunt work.

Well, I'm sure I'll think of more if I can ever get past these questions. But overall, this outing has been disappointing.  In general, I don't think interviewers take well to a technical grilling; whether they are unprepared to answer these types of questions or they simply don't know the answers, it hasn't been too promising.  For me, it's important in gauging the technical competance of my potential colleagues and the type of technical training/staff development that the company provides and/or encourages.  It's one way to avoid ending up in a company staffed by 5:01 developers.

I still haven't figured out how to respond to the "So what year did you graduate" question, as it's clearly a form of age discrimination but I'm not sure how to call someone out on that yet.  More importantly, it implies that the groups and personnel are not necessarily organized by merit, but by seniority or, even worse, cronyism.  I think next time, I'll just be blunt about it and ask if the interviewer realizes that it can be construed as age discrimination.

I caught this phrase scrawled on Woody Paige's blackboard last night on Around the Horn (one of the few television shows I watch somewhat regularly).

I love it.

Glenn Campbell on "stupidity":

During our childhood, we are given a certain amount of protection from reality. Our parents dole out rewards and punishments that are often detached from the conditions we must eventually face. Some parents, for example, may reward their children no matter what they do. This sets the stage for stupidity in adulthood, as the subject expects the outside world to hand him the same unconditional reward.

The habits of stupidity can be terribly difficult to change, especially in others. This is why we label some people "stupid" as an overall systemic condition. They are never going to "get it" because they have made a fundamental philosophical decision not to. Their emotional needs are so great and cause them so much internal panic, that they can never accept reality the way it is.

The worst thing you can do for a stupid person is protect them from their mistakes. Maybe stupid should hurt. If it doesn't, then they're going to get even more stupid, and they will be totally unable to deal with life when the protection finally collapses.

When you have a boss, client, parent, spouse or adolescent child like this, that's when you find out what a tragic and terrible disease stupidity is. You clean up one stupid mess, but then there's another and another. There's never going to be an end to it until the stupid person touches reality himself is able to directly experience the results of his actions as they occur.

Smart people, by definition, learn quickly from their mistakes, but stupid people don't. They may have to hit their head against a wall many times before they realize, "Hey, this isn't a good idea." Even then, it's only that particular wall they've learned about. If you put up another wall, they'll insist that it shouldn't be there and repeat their mistakes all over again.

This last paragraph is particularly relevant in software development.  I wouldn't call anyone stupid (you have to have above average intelligence to make it anywhere in software development; more like misguided or unmotivated), but many times, people just don't learn from pain and mistakes.  They settle into a methodology or a style and build up mental inertia.  No matter how many times you try to tell them that there is a better way, a more efficient way, a better tool, they refuse to adapt and expand their boundaries.

It's not that "smart developers" are infallible -- everyone makes goofy design decisions from time to time (especially when "The Big Picture" is not a known quantity), but that they can adapt quickly and learn from their mistakes.  They look for ways to ease the pains of the development process.  They are curious about how to make processes more efficient and less error prone.

Just a random Thursday lunchtime rant

Fred Brooks' The Mythical Man Month is one of my favorite software engineering books.  I first read it in my senior software engineering class at Rutgers.  From time to time in my daily routine of software implementaion, I flash back to bits and pieces from the book and from the class.

One of the most relevant issues that Brooks' touches upon is the issue of communication and how it impacts productivity.  Of this, Brooks writes:

The added burden of communication is made up of two parts, training and intercommunication.  Each worker must be trained in the technology, the goals of the effort, the overall strategy, and the plan of work.  This training cannot be partitioned, so this part of the added effort varies linearly with the number of workers.

Intercommunicaion is worse.  If each part of the task must be separately coordinated with each other part, the effort increases as n(n-1)/2.  Three workers require three times as much pairwise intercommunication as two; four require six times as much as two.  If, moreoever, there need to be conferences among three, four, etc., workers to resolve things jointly, matters get worse yet.  The added effort of communicating may fully counteract the division of the original task...

This issue seems particularly relevant today as teams are more geographically dispersed than ever (whether due to working with offshore teams or with developers working across the country). As you increase the number of developers working on dependent and communicating components, you dramatically increase the development time.

One of the most common approaches to working around this is to partition the work such that each developer is responsible for minimal cross component communications (each developer implements a full stack). 

The approach works because there is less turnaround and less miscommunication between developers.  It reduces dependencies between one component and another, allowing a develper to progress as fast as she can implement the requisite bits.  Need to store an additional field?  No problem.  Add a column, add it to your data model, and then render it in the UI.  No need to ping someone else or write a change request; just make it happen.

The downfall of this approach is that it leads to a lot of duplicate code for common logic; it tends to work well for a very small team with geographic proximity, but breaks down very quickly as the number of developers or geographic proximity increases.  A developer writing data access for one component may have her own practices while another will abide by a completely different set of practices.  In the long run, this leads to code that's hard to maintain as well as adding much more complexity to the product itself. 

Not only that, any given developer may not be suited for developing a given part of an application stack.  A UI developer will be much more proficient at and capable of designing an attractive, easy to use UI than a backend service developer.  An application developer may be capable of writing a much more cohesive domain layer and business layer than a database developer.

Service contracts are an oft touted solution.  The promise is that by agreeing on a service contract, each side can develop independently of a concrete implementation of the other, so long as the contract is followed to a T.  However, at least in my experience, this is usually far from the truth, especially if you work in a team where the contract itself is rarely stable for more than a day or two.  UI tweaks or new features can cause dramatic changes in the service (for example, requiring more data or a change to the model) which comes at a cost of time spent on change requests and communicating those changes to the other side.

For the time being, I've come to conclude that full stack development (hmm...maybe with the exception of the UI layer) is the best approach, so long as it's supported by the necessary tools, templates, and frameworks; it's all about making it easy to make the guts work (and to do so in a consistent manner) so that the implementation differences where it matters is minimized (for example, data access). Enter code generation,  automation, and application frameworks to the rescue.  Tools like MyGeneration, Spring, Smart Client Software Factory, and Enterprise Library are really the answer but I've found it quite difficult to get team members to buy into embracing these tools (it's one thing to use a framework; it's another, entirely, to embrace a framework or tool).  When properly wielded with a cohesive software architecture, you can get the best of both worlds: a coherent codebase with common patterns across component stacks while allowing developers to work with less dependencies.

What do you think?  How do your teams handle working on multiple dependent components?

WCF DataContracts include an ExtensionData property which is a bit troublesome if you are trying to send a modified object back up to the service if it has no properties in Javascript.

So I wrote a little piece of Javascript to clean it up:

function DeleteExtensionData(obj) {
	var keys = Object.keys(obj);

	keys.each(function(key) {
		if(!Object.isFunction(obj[key])) {
			if(obj[key] instanceof Object) {
				DeleteExtensionData(obj[key]);
			}

			if(key == "ExtensionData") {
				delete obj[key];
			}
		}
	});
}

It will recusively delete all ExtensionData properties from the object.  You can call as soon as you get the result from a completed AJAX request or you can call before sending an object parameter to a service.

Note that it uses constructs from prototype.

If you want to get fancy, you can also write a custom serializer.