Random Thoughts of a Scatterbrain.
 Saturday, April 26, 2008

The Sad State of Secularism

4/26/2008 4:26:01 PM (Eastern Daylight Time, UTC-04:00)

This story is kind of upsetting.

Like hundreds of young men joining the Army in recent years, Jeremy Hall professes a desire to serve his country while it fights terrorism.

Known as "the atheist guy," Hall has been called immoral, a devil worshipper and -- just as severe to some soldiers -- gay, none of which, he says, is true. Hall even drove fellow soldiers to church in Iraq and paused while they prayed before meals.

 "I was ashamed to say that I was an atheist," Hall said.

"Religion brings comfort to a lot of people," he said. "Personally, I don't want it or need it. But I'm not going to get down on anybody else for it."

I dunno...it's just kind of unnerving that in this day and age, a person should feel ashamed to be known as an atheist (especially one that is respectful of others' right to believe).  In fact, recent polls have shown that atheists are the most distrusted group in America, even below Muslims:

From a telephone sampling of more than 2,000 households, university researchers found that Americans rate atheists below Muslims, recent immigrants, gays and lesbians and other minority groups in “sharing their vision of American society.” Atheists are also the minority group most Americans are least willing to allow their children to marry.

It just doesn't make sense.

 Tuesday, April 15, 2008

On The Suckage Of Lotus Notes (6.5)

4/15/2008 12:49:50 PM (Eastern Daylight Time, UTC-04:00)

Lotus Notes continues to suck as much as ever.  Every day, I wake to find more suckage.

For example, today, when a coworker forwarded a meeting invitation to me, instead of allowing me to accept it in my calendar and schedule the event, it came across as a regular e-mail with no option to accept the meeting invitation which was attached.

Furthermore, there's this little gem:

Uhhh...thank you Lotus Notes for informing me that the meeting occurs in a different time zone.  It would be nice if you could tell me which time zone.  You know, since I can't automatically schedule the event, it would be nice if you could just tell me what time zone it's in so I can manually schedule it.  Furthermore, see that little outline around the globe and arrow?  This would lead one to believe that clicking on it does something like maybe map the actual time...but nope, nothing happens (the same with the text).  LotusNotes = Productivity--

Meanwhile, Amazon Prime is the polar opposite of suckage.  I ordered two books yesterday and used the default Prime "free" 2nd day shipping option.  To my surprise, they've arrived on my doorstep less than 24 hours later.  If that isn't awesome, then I don't know what is.

 Wednesday, April 02, 2008

SharePoint As A Development Platform

4/2/2008 9:19:53 AM (Eastern Daylight Time, UTC-04:00)

I got an email today regarding a blog post by Jeffrey Palermo on the shortcomings of SharePoint as a development platform.

Now I have to say, SharePoint is not without fault (particularly in the area of feature packaging and deployment), but Jeffrey's perceived issues with SharePoint really show either the lack of personal development experience with SharePoint or a lack of creativity on the part of the team he's working with.

Let's get one thing clear first, okay?  SharePoint is meant as an enterprise collaboration and document storage platform, first and foremost.  One must always bear this in mind with regards to any discussion on SharePoint.  Why is it so big?  Why is it clunky in some places?  Why does it have to be installed on a server OS (okay, I admit, this one is probably more about licensing and $$ than anything technical)?

It seems that Jeffrey's major beef, that it must be installed on a server OS, kills any benefits of SharePoint as a development platform immediately.  So let's tackle this one first. 

First, a disclaimer: I'm not a SharePoint guru.  I don't have 5 years of experience with SharePoint.  I'm only going on what I've learned in working with SharePoint, day in and day out, for the last year.

In our development group of 5 or so people, we've managed to develop against SharePoint quite well, despite the fact that we all run XP as our primary development environment.  How?  Virtual machines.  Of course, some would view this as a hassle, as another stumbling block or quite clumsy.  I view it otherwise as there are numerous benefits in developing against a VM server environment:

  1. Every deployment is exactly the same.  This means that server names, file structures, databases, and so on are exactly the same for every developer.  This aids -- not hinders -- the development of automated scripts and deployment utilities.  On our team, we usually synchronize images once every month or so.  What this means is that we'll take turns updating a common image and then everyone will grab a copy of said image.
  2. Virtual Machines are easy to deploy.  It's called: copy-paste!  Nothing easier when adding a new developer to your team.  Instead of configuring a whole new SharePoint environment, just have the developer copy the latest development image.  So easy, a caveman could do it.
  3. Virtual Machines can be rolled back.  Try doing that with your development environment!  If every developer had SharePoint deployed on his or her machine, a mistake (deleting/altering some core database tables or records, for example) would be devastating...it would mean at least a day of lost work time trying to rebuild the development environment.  Using virtual machines to host the server environment protects it from inadvertant dismantling.  And even if it is somehow altered for the worse, no matter, get another copy from someone else!  And of course, being able to rollback the image (provided that you created a snapshot) makes updates (for example, adding a SQL Server service pack) painless.
  4. Virtual Machines are portable.  Why does this matter?  Demos!  There's nothing more convenient than having a sales guy pack up a copy of the VM with all of the software installed and tweaked for demos.  This is a huge bonus.

Look, the suggestion of developing with SharePoint in a "native" environment in a team is just plain stupid; it's a matter of working harder, not smarter and it shows a lack of creativity in terms of development management.  (One disclaimer: some Visual Studio tools from Microsoft for SharePoint cannot be installed on XP...this is a shortcoming, for sure, but it hasn't really affected our development.  If absolutely necessary, you can install a stripped down copy of Visual Studio on the VM).

Now let's address each of the other 7 points that Jeffrey brings up:

  1. SharePoint isn't easy to install.  I'm going to skip that one because clearly, this man has never installed SharePoint.  It's nothing more than point and click...my gosh, I don't understand how it could possibly be easier.  Aside from this, using a VM approach, it only has to be installed once.
  2. SharePoint isn't easy to configure.  See above.
  3. SharePoint does not integrate well with simple tools.  I'm not sure what he means here.  Many administrative tasks can be handled by stsadm.exe which in turn, means that many of the administrative tasks can be handled by batch scripts.  Aside from this, SharePoint is just an average ASP.NET application.  Stopping and starting IIS, xcopy, and so on...these are all still applicable to a SharePoint deployment.
  4. SharePoint isn't easily extended to make simple tools.  What?  Nothing could be further from the truth.  The only beef that I would hold is that Microsoft doesn't package the SharePoint DLLs separately so you have to extract the DLLs from the GAC of your server environment.  But once you do that, it's easy to reference and use the API by copying the DLLs to your development environment.
  5. SharePoint isn't easy to debug.  Again, I'm not sure what the issue here is.  The process of debugging SharePoint applications (web parts, layout pages, etc.) is no harder than debugging any standard, run of the mill ASP.NET web applications.  Okay, sure, you can't just hit F5 (oooh, the horror!), but seriously, I hate F5 developers - sometimes, F5 just isn't the way to go, dude.  I'm also pretty sure chimps (or macros) can be trained to hook up the debugger and hit F10 and F11.  As for debugging core SharePoint, well, I guess he has a point since the source and debugging files aren't available (not that I know of), but I don't see that as an issue.  I mean, how many platforms ship with debugging symbols included?
  6. It's not easy to create test automation for SharePoint.  There is some truth to this since it requires the developer to learn about the platform first (API, database tables, web services, and so on).  But I don't think it's any more difficult or challenging than setting up automated tests against any other third party platform.
  7. SharePoint configuration does not store easily in source control.  See above regarding VMs.  Aside from this, he's got it all wrong again.  We can look at this from a few different angles.  First, SharePoint is just an ASP.NET application.  Repeat that to yourself about 100 times.  With regards to application configuration (in terms of the web.config), it's as simple as copying the web.config to your source control system and using a script to deploy it on build.  Secondly, if we look at the configuration in terms of web parts and layout pages, it's possible to include these in source control as well using solution packages deployed either on build or manually as features.  Certainly requires a bit of research to get it working the first time, but it's not a task that takes more than 1-2 days of experimentation.

From personal experience, I've found SharePoint a very compelling application development platform (again, that's not to say that it doesn't have shortcomings) because it's nothing more than ASP.NET but with the added bonus of a document management/storage API, profiling, permissions, and it acts as an integration platform for a variety of applications. 

You can make it as hard or as easy as you want it to be with regards to developing applications for SharePoint.  It's only a matter of how much time you are willing to put into flipping through the API and understanding the fundamentals of working with SharePoint.  A big part of successfully and painlessly developing against SharePoint is creativity in terms of setting up your development environment and automation (batch scripts, pre/post-build scripts).

The points that Jeffrey, didn't bring up -- the true pain points -- are really "fringe" features so far as I'm concerned.  Namely, this centers around SharePoint hosted and integrated workflows and InfoPath (because no one likes and no one actually uses the otherwise useless and purposeless InfoPath).  You're not required to use InfoPath or SharePoint hosted workflow; in FirstPoint, absent the early documentation and tools required to be productive on this front, we made an early decision to host workflow in our own environment. Sure, we miss out on some of the native features of SharePoint like workflow state visibility and integrated forms via Forms Services, but I don't see it as something we can't overcome as documentation and tool support becomes better on this front.

 Friday, March 28, 2008

SharePoint: The Second Coming Of Lotus Notes?

3/28/2008 8:51:33 AM (Eastern Daylight Time, UTC-04:00)
As I was pondering the suckage of Lotus Notes, I came across an interesting little piece on a CMS Watch Report titled: "SharePoint Has Become the New Lotus Notes":

Microsoft Office SharePoint Server 2007 is repeating history as it mimics the allure and pitfalls of Lotus Notes, according to research released by CMS Watch, an independent analyst firm that evaluates content technologies.

SharePoint exploits traditionally underserved collaboration needs for information workers laboring within Office tools, and fulfills a common desire to easily create disposable workspaces, CMS Watch found.

Like Notes in a previous decade, IT often embraces SharePoint as a simple answer to myriad business information problems.  But the platform can morph into a technical and operational morass, as repositories proliferate, and IT comes to recognize that various custom applications require highly specialized expertise to keep running properly.

...

The SharePoint Report 2008 concludes by advising customers to establish clear boundaries on SharePoint services, to keep it from becoming their new Notes – the platform that everyone loved, but then loved to avoid.

While SharePoint does indeed have it's weaknesses (total lack of any integration with ASP.NET AJAX in the SharePoint implementation itself -- guess we'll just have to wait for 4.0, web services support is still kind of weak) and oddities (CAML?), it's nowhere near the steaming pile that is Lotus Notes.

On a serious note, I do kind of see the point in that last paragraph there.  SharePoint often gets evangelized as some silver bullet for collaboration ("Oh look, workspaces! Workflows! Tasks lists!") but I've never been in an organization that's used SharePoint in a way that was actually of any aid to productivity or collaboration; people just don't seem to want to log onto a corporate SharePoint portal unless they have to. 

That's not to say that the platform doesn't have its useful bits, but the real gem in SharePoint is its integration with Office applications as a platform for "seamless" sharing of documents and I think the idea of offering that to a much larger audience (via Office Live Workspaces) is long overdue from Microsoft.  Until recently, there were few integrated solutions for small businesses, students, and other non-business groups for the very simple act of sharing Office documents aside from using e-mail. 

Even when I joined Zorch Software, I would do a facepalm regularly when I got an email with a document attached with a "v15" suffix.  I'd save it in the same folder as the previous 14 "versions" that I received.  The irony.  The problem is that most of SharePoint just isn't that useful.  Even in a tech minded organization like Zorch Software, you just can't break some people out of old habits; to many, collaboration is synonymous with e-mail.  There is a whole generation that doesn't get wikis and doesn't want to learn wiki markup. 

Well, in any case, I'm still not over the fact that I'm being forced to use Lotus against my will and I'm still bitter over the fact that it's been so hard to get people to embrace our Trac wiki and embrace the ticket system for tracking issues.

 Thursday, March 27, 2008

Lotus Notes: It Sucks...Hard

3/27/2008 3:30:39 PM (Eastern Daylight Time, UTC-04:00)

As part of the transition from FCG to CSC (Computer Sciences Corporation), we are also switching our mail system from Outlook/Exchange to Lotus Notes/Domino/Whatever.

There can only be one immediate reaction up on switching from Outlook 2007 to Notes 6.5:

WTF?? AHHHHH!!11 MY EYES!! GIVE ME BACK MY OUTLOOK!

Surely, this application which looks like it crawled out of the late 90's cannot be the primary enterprise messaging system for a large, publicly traded company, can it?  Maybe it's a clever ruse to get people to stop using email and actually interact with one another by ecouraging calls instead of email...maybe.  I can't believe IBM actually sells this product and I can only imagine the painful life endured by the Notes sales team and all of the mockery they must live with.  If my parents worked on the Notes team in development or sales, I'd ask to be disavowed for their crime of bringing such a shitty piece of software into existence.

I think I'm going to start my own trail of why Lotus sucks posts, but Lotus Notes Hater already has a headstart on me.

The truly absurd thing that I've learned is that there are actually standalone programs that have been written to clean up crashed instances of Notes!  No kidding, I got the following email from a company wide distribution list:

If Lotus Notes crashes, you don't have to restart your PC anymore! Just run ZapNotes and allow it to clean up what Notes left behind so you can restart Lotus Notes. ZapNotes works on Windows 9x/NT/2000/XP

Lotus Notes Hater comments:

Cassetica charges US$1,500 for NotesMedic Pro and US$2,500 for the enterprise level of NotesMedic. The fact that a company makes money from a product that shouldn't even exist makes us ask, "Why haven't the owners of Lotus Notes put Cassetica out of business by incorporating the feature of NotesMedic directly into Lotus Notes?"

NotesMedic is not the only product out there. There is also ZapNotes. And KillNotes. There are at least three products that have no reason to exist.

Awesome!  Now I'm really gonna look forward to using Notes!

I also got to chat with some of my teammates in Vietnam just to see how they felt about it.  Who knows, maybe their Asian sensibilities would lead them to be more polite and less judgemental.  Maybe their allegiance to their employer would cause them to accept Lotus as their fate and use it dutifully.

Mumble mumble says:
  Did you guys install Lotus yet?
Vietnamese Guy says:
  Yes
Mumble mumble says:
  What do you think?
Vietnamese Guy says:
  Slow
Vietnamese Guy says:
  very slow
Vietnamese Guy says:
  urgly

So there you have it, even the Vietnamese hate it.

 Tuesday, March 25, 2008

DRM = Doesn't Really Matter

3/25/2008 10:52:05 AM (Eastern Daylight Time, UTC-04:00)
It's true, DRM just doesn't really matter.  The music industry has pretty much seen the light at this point with Amazon now offering tracks from all four major labels DRM free.  How much longer will it take for the movie industry to realize the same truth?

Blu-Ray's vaunted BD+ encryption scheme was supposed to give it the upper edge over HD-DVD's ill-fated AACS encryption scheme.  The news on this front the last couple of days has been software maker SlySoft's crack of BD+.

My favorite little tidbit is this statement by the Blu-Ray camp:

Richard Doherty of the Envisioneering Group will have to revise his statement from July, 2007 regarding BD+: "BD+, unlike AACS which suffered a partial hack last year, won't likely be breached for 10 years". It is worth mentioning that since he made that statement only eight months have gone by.
We'll see how this shakes up.  DRM, once again, proves that it is barely a deterrent to pirates while a genuine hassle for legitimate consumers who wish to back up their physical media.

 Saturday, March 08, 2008

Random Rant

3/8/2008 12:32:42 AM (Eastern Standard Time, UTC-05:00)

One of the most awesome pieces of software that I've come across in my career as a consultant and software engineer is Trac.  Among the various awesome features of Trac include:

  • An awesome plug-in system and library for all sorts of add-ons to the core system
  • RSS feeds on available on the timeline which acts as a customizable, almost realtime data feed
  • Integrated source browser...it makes pointing out code isssues so much easier when you can link it to source
  • An awesome wiki system that integrates with the ticket and changeset systems

Did I mention the ticket system?  Sure it lacks a bit in terms of workflow and it is rather simplistic -- being geared almost exclusively towards bugs, but I think the most important aspect that it brings is, well, tracking.

Having worked with it for almost a year, I find it really hard to imagine working on any sizable software project without it (or some other piece of comparable software project management tool).

Perhaps the only thing that I've found frustrating -- surprisingly, it is not with Trac itself -- is the inability to really win anyone else over in my group; you'd think its like pulling teeth to get a ticket created for tracking purposes instead of using email. Let's not even get into wiki editing or proper usage of wiki markup or taking advantage of the integrated wiki syntax for tickets and what not.  It just seems like some old habits are hard to break (much to my dismay).

But don't let this dissuade you! If you haven't given it a look yet, there's no time like the present to download it and give it a go. Oddly enough, if you're looking to get started, VisualSVN Server may be the way to go since it includes an integrated installation of Apache, Subversion, and a mangement console.  Also check out the awesome documentation on installing Trac on a variety of systems, including Windows.

 Monday, February 18, 2008

Education And What We Can Learn From Other Countries

2/18/2008 9:54:04 PM (Eastern Standard Time, UTC-05:00)

As the spouse of a teacher, I know first hand the challenges that many young teachers face in the US today.  From waning parent interest and participation to lack of administrative support to poor professional development plans.  Of course, there is also the issue of compensation; let's face it: $40,000 just doesn't go very far.

I'd like to think that my wife is one of those teachers that takes her work very personally and always takes the initiative to help her students.  As Claudia Wallis writes in "How to Make Great Teachers", she writes that one of the key characteristics of successful teachers is "an unshakable belief in children's capacity to learn."  Indeed, I think my wife has a certain stubborness when it comes to her students in that she doesn't accept excuses for failure and always pushes her kids.  I'm constantly surprised by the number of parents of former students who come up to her and tell her how she's changed their children's lives, especially when it comes to the topic of mathematics.

One thing that amazes me is the incredible cost of education for teachers in the US and how much of that burden falls on the shoulders of America's teachers and their families, especially for post-graduate education.  For example, my wife's graduate school loans total over $25,000!  Her commitment from her school district? $750 per year.

Linda Darling-Hammond comments:

All teacher candidates in Finland, Sweden, Norway and the Netherlands, for example, receive two to three years of graduate-level preparation for teaching, at government expense, plus a living stipend.  Unlike the U.S., where teachers either go into debt to prepare for a profession that will pay them poorly or enter with little or no training, these countries made the decision to invest in a uniformly well-prepared teaching force by recruiting top candidates and paying them while they receive extensive training.

For certain, it seems like the US doesn't treat teaching like a first class profession.  Contrast this with Singapore's approach:

To get the best teachers, the [National Institute of Education] recruits students from the top third of each graduating high school class into a fully paid four-year teacher-education program and puts them on the government's payroll.  When they enter the profession, teachers' salaries are higher than those of beginning doctors.

Aside from the greather respect for the teaching profession, it seems like many countries also take a different approach to continued on the job professional development.

[In Singapore,] the government pays for 100 hours of professional development each year for all teachers.  In addition, they have 20 hours a week to work with other teachers and visit on another's classrooms.

Most US teachers, on the other hand, have no time to work with colleagues during the school day.  They plan by themselves and get a few hit-and-run workshops after school, with little opportunity to share knowledge or improve their practice.

Harold Stevenson noted that "Asian class lessons ae so well crafted [because] there is a very systematic effort to pass on the accumulated wisdom of teaching practice to each new generation of teachers and to keep providing teachers the opportunities to continually learn from each other."

In a paper titled Speeding Up Team Learning, Edmondson, Bohmer, and Pisano write, with regards to their study on surgical teams learning new procedures and practices in the area of cardiac surgery:

Teams that learned the new procedure most quickly shared three essential characteristics.  They were designed for learning; their leaders framed the challenge in such a way that team members were highly motivated to learn and the leaders' behavior created an environment of psychological safety that fostered communication and innovation.

I think we can take some of these ideas and merge apply them to the teaching profession as well by identifying what Darling-Hammond terms "expert teachers" and emphasizing the fostering of these individuals into drivers for team and mentor based professional development.  Furthermore, I think greater structured team based learning and communications would greatly enhance the experience of young teachers learning the craft.

 Tuesday, January 29, 2008

"Free" Money? Think Again...

1/29/2008 9:20:01 AM (Eastern Standard Time, UTC-05:00)

Michael Kinsley has some sobering commentary on the economic stimulus package idea that's floating around in Washington right now, perhaps one of the most hairbrained ideas to come out of DC in recent years.  Kinsley's subtitle says it all: "We need a 'fiscal stimulus' the way a drunk needs another drink.  Let's sober up first."

Kinsley brings up the core reason why this idea is doomed to failure: in a time of impending recession, it is only natural that a good portion of the beneficiaries of the rebate will act responsibly and save it or use it to pay down debt instead of instantly pumping it back into the economy.

"Direct government spending is a more efficient stimulus than an equivalent tax cut because all of it gets spent.  When actual people get hold of the money, a few might have an unpatriotic tendency to save some of it."

Kinsley further chastises this proposal by pointing out the obvious:

"My gripe is that telling Americans that they need to borrow and spend just a little bit more to get us past this recession -- and then reform their ways -- is like telling an alcoholic he needs one more drink before sobering up."

It is amazing that, quite possibly, a large percentage of Americans must think that money comes out of thin air and the gubmint can just print more of it; personal and government fiscal responsiblity is a distant reality, someone else's problem.

Like adhering to that New Year's resolution to lose weight or start saving more money, if the promise of a magic pill or surefire investment seems too painless to be true, it probably is.  There is still no substitution for daily exercise and, likewise, financial responsibility.  Kinsley points out:

"If we must have a fiscal stimulus, let's make sure it's not too enjoyable."

Spending our way out of a recession driven by a debt crisis just seems too enjoyable to work.  It is the proverbial magic diet pill or instant get rich scheme...doomed to fail.

 Sunday, January 13, 2008

Confucius On Programming

1/13/2008 4:18:02 PM (Eastern Standard Time, UTC-05:00)

One of the favorite classes that I took in college was an introductory course to Chinese philosophy. 

From time to time, I found one or two passages from The Analects that I could apply to my computer science major.

One of my favorites is passage 13:3:

Tzu-lu said, "The ruler of Wei is waiting for you to serve in his administration.  What will be your first measure?"  Confucius said, "It will certainly be the rectification of names."  Tzu-lu said, "Is that so?  You are wide of the mark.  Why should there be such a rectification?"  Confucius said, "Yu! How uncultivated you are!  With regard to what he does not know, the superior man should maintain an attitude of reserve.  If names are not rectified, then language will not be in accord with truth, then things cannot be accomplished.  If things cannot be accomplished, then ceremonies and music will not flourish...."

"Therefore, the superior man will give only names that can be described in speech and say only what can be carried out in practice."

To me, this applies to software engineering in the naming of classes, methods, properties, variables, and so on.  As McConnell writes in Code Complete,

The smaller part of the job of programming is writing a program so that the computer can read it; the larger part is writing it so that other humans can read it.  let [the reader] use their brain cells to understand the larger question of how your code works rather than the syntactic details of a specific expression.  You write readable code because it helps other people to read your code.

Indeed, sensible naming is a big part of this idea of legibility.  What I dig about Visual Studio 2008 is that the installed documentation includes excerpts from Framework Design Guidelines by Cwalina and Abrams which aims to include some of the key naming and design best practices into the core literature on the .NET Framework.  It's a necessary and needed move to help forward the education of the legion of .NET developers who continue to perpetrate terrible naming practices carried over from VB.

I mean seriously: if I have to read one more method name as a noun (or a property as a verb or a class as a verb), I'll just have to start researching a way to let me reach across the Internet and punch someone in the face -- nothing ruins my morning quite like finding a new class in source control named as a verb.  I'm also midly irritated when developers choose unconventional names for object types.  For example, the use of "Checker" as opposed to "Validator" -- ugh!

Admittedly, some artifacts are just really hard to name.  But in such cases, proper naming is probably even more important since if it's hard for the writer of the code to come up with a proper name, the wrong name will make the artifact even harder to understand for secondary readers.  There have been times where I've spent days trying to come up with a suitable name for a class (I'll name it, but I'll continue to mull it over through restless nights (true story) and refactor it once I do come up with a suitable name) since the last thing I want to do is to give something a cryptic, hard to understand name.

One of the easiest ways around this is to spend some time digging around the SDK and look for similar patterns in naming and see how the .NET framework designers structured their class and member names.  I also think that Cwalina and Abrams' Framework Design Guideline is a stalwart component of any .NET developer's library.

So the next time you're about to write a class name or create a new member, just remember,  "If names are not rectified, then language will not be in accord with truth, then things cannot be accomplished.  If things cannot be accomplished, then ceremonies and music will not flourish". I mean, who doesn't want their project to end with ceremonies and music :-D

 Monday, October 01, 2007

The Slow Death of DRM

10/1/2007 12:27:50 PM (Eastern Daylight Time, UTC-04:00)

I've been in a somewhat heated debate with my once CEO and now VP regarding the effectiveness (or rather, the ineffectiveness) of DRM and how the media companies are really just screwing themselves (whilst also screwing customers) by not adapting and accepting digital as this generation's radio.

I've always held the stance that DRM is a useless encumbrance to legitimate users of the content while providing merely a false sense of security to the copyright holders; those who want the content bad enough will circumvent the DRM somehow.  In the end, regardless of how good the DRM is, the simple fact is that the end product must be output at some point in time.  The content can always be captured as output from some trusted system (though some quality may be sacrificed).

Time and again, we've seen that the application of DRM is a fruitless effort in the cat and mouse game with hackers that the hackers have won every time.  Witness:

In his open letter, Steve Jobs comments on DRM and states:

Imagine a world where every online store sells DRM-free music encoded in open licensable formats. In such a world, any player can play music purchased from any store, and any store can sell music which is playable on all players. This is clearly the best alternative for consumers, and Apple would embrace it in a heartbeat. If the big four music companies would license Apple their music without the requirement that it be protected with a DRM, we would switch to selling only DRM-free music on our iTunes store. Every iPod ever made will play this DRM-free music.

Why would the big four music companies agree to let Apple and others distribute their music without using DRM systems to protect it? The simplest answer is because DRMs haven’t worked, and may never work, to halt music piracy. Though the big four music companies require that all their music sold online be protected with DRMs, these same music companies continue to sell billions of CDs a year which contain completely unprotected music. That’s right! No DRM system was ever developed for the CD, so all the music distributed on CDs can be easily uploaded to the Internet, then (illegally) downloaded and played on any computer or player.

I tend to think that a technology visionary like Jobs "gets it".  He understands that it is quite likely that no perfect DRM system can ever be created but in an effort to use these imperfect DRM systems, the only people that are being punished are legitimate consumers of the media by being locked into proprietary stacks of players, online stores, and digital media.

He also touches upon an oft ignored point: the CD, a digital source, itself does not contain any form of DRM.  It's true that the designers of the format perhaps did not foresee a world digitally connected and able to distribute 650MB worth of data in mere seconds (BitTorrent),  but that does not absolve the fact that they're plugging the crack in the dam while ignoring the gaping hole.

It is my view that the actual number of people who actually rip and distribute music from CDs and DVDs are a very small percentage of all consumers.  Meanwhile, there is a much larger percentage of consumers who get their copies illegally from these sources via peer to peer and file sharing networks.  And yet a larger percentage of people are actual legitimate consumers who plunk down the full price of the CD or DVD in stores and take it home with them.

What the music industry should be concerned about is not that marginal percentage of sources (those who hack the DRM systems or use the resultant software to rip and distribute the content - this group will continue to do so, indefinitely), but the much larger portion of the consumer population that illegally downloads the output of these providers even in the face of the minor threat of legal action.  The question of course is how they can reach this consumer (or at least a large proportion of this consumer).

The secret seems to be offering a "fair deal" to the consumers.  I can still remember the days when CD singles cost $5, 6, 7, even 8 dollars!  Of course, what is "fair" is arbitrary and, as my VP would say, "determined by the market" (what he seems to disregard is that the music industry was guilty of price fixing to artificially inflate the cost of CDs instead of allowing for the market to decide the fair price), but clearly, this price seems absurd!  Of course, then the question is, what is fair?  Is iTunes' $0.99 model, "fair"?  It's difficult to say since "fair" is relative to the consumer.  To some, $25,000 is "fair" for a cell phone while to others, $250 seems absurd for a cell phone (obviously different products, paying for brand, etc.; but the essence is that they are functionally equivalent in damn near every way).  Price is not the only factor: consumers, as Jobs noted, expect that once they've paid for the content, they can reuse it (not redistribute it) in their cars, on their cell phones, on their portable music players, and so on.  In the consumer's eyes, DRM is but a nuissance driving them to find DRM free, illegally distributed versions of the content.

Today's news that's buzzing around the Internet community is the upcoming release of Radiohead's next album.  Most of the buzz centers around the fact that this is the first major artist/group to release their music completely independently...no music labels involved.  Not only that, this is the first mass live experiment in determining "fair" pricing in terms of music and media:

From Time:

There's no label or distribution partner to cut into the band's profits — but then there may not be any profits. Drop In Rainbows' 15 songs into the on-line checkout basket and a question mark pops up where the price would normally be. Click it, and the prompt "It's Up To You" appears. Click again and it refreshes with the words "It's Really Up To You" — and really, it is. It's the first major album whose price is determined by what individual consumers want to pay for it. And it's perfectly acceptable to pay nothing at all.

It will be an interesting experiment indeed; the results of which, if shown to be successful, will shake the music industry to the core as other artists start to adopt the model.  The music industry has been put on notice: adapt or die.

Will Radiohead be successful?  Will they earn a dime?  One thing is for sure, they will gain a new audience of listeners who would otherwise not have been willing to purchase a CD for $16.00, but will surely download and sample the new tracks for free or for a nominal price.  But from this, it's easy to predict that Radiohead will surely increase sales of their previous albums as a new set of listeners discover the group because they've opened their content to the consumer.

It is the same with Internet radio stations, where the absurdity over the proposed rates to be paid by Internet radio stations was just recently put on hold.  In the age of HD radio broadcasts and radio-to-computer devices, what sense did it make to treat Internet radio any differently from traditional FM radio and even satellite radio?  Like traditional broadcast radio, Internet radio serves the same purpose in that it allows consumers to discover artists that would otherwise not have been given a glance (every CD I've purchased in the last 5 years has been a result of hearing the artist or group on an Internet radio station first).  It's simply that I'd rather listen to music from my computer than from my stereo.  To the consumer, the nuances of distribution and control of the media are irrelevant: the consumer just wants to listen to the music and it's really no different than an HD radio broadcast.

The media companies need to adapt and embrace technology.  They need to study how consumers want to use the content.  They need to understand that the old models won't work anymore in a connected world where content is expected to be transferrable with little hassle and reusable by the consumer (just as a CD should play in your car, in your desktop stereo, on your computer, or from a portable CD player (do people still use those?)).

 Thursday, September 27, 2007

Dynamic SQL: Yea or Nay?

9/27/2007 2:42:01 PM (Eastern Daylight Time, UTC-04:00)

I've always been on the side of stored procedures in the classic debate over the merits of dynamic SQL.  In reality, I can only think of one good scenario where dynamic SQL at the application layer should be used: programmatic batch inserts.

I won't go into the performance debate, since there are tons of articles that already cover this area, but rather, I'd like to discuss the usability and development and architectural aspect of it.

In almost all other cases, it seems like the best choice is to have the application not generate dynamic SQL and use a stored procedure...always.  There are certainly times when dynamic SQL is necessary, for example, when generating selects against a dynamic table structure, but in those cases, the variable portions of the query can be parameterized into the stored procedure and the procedure should generate the dynamic SQL.

Some would argue that if the underlying data models change, the application layer will usually be forced to change are ignoring other aspects of model changes that don't necessitate application model changes.  These include performance tuning, filtering by table JOINs and reuse of the data logic in nested stored procedures or functions.

When working with compiled code like .NET, the core issue is that fixing query errors involves a recompile and redeploy, which in most cases, is much more difficult than just fixing a completely disconnected (but not completely decoupled since there is a quasi-interface (the return result type and structure)) stored procedure.

For example, if a dataset today contains data from table A and tomorrow it needs to include data from table A and B (let's say they both contain the same elements, but one is used for archives), it would be easy to update the procedure to UNION the results from the two datasets without affecting the application layer.

This isn't the only scenario, for example, let's say the requirement changes and now the data needs to be filtered by another table.  It would be easy to add a new INNER JOIN to the query without affecting the application layer.  Not only that, it also allows for the recombination of fields (for example a user name field today only needs to show first and last name, but tomorrow, it may need to show the middle initial as well - this change can be done at the database level and not affect the application or UI layers).  It can also make it easy to change the underlying table structure so long as the return data isn't expected to change: it provides a layer of decoupling between the application layer and the raw data storage.

In addition, having a stored procedure allows for easier testing of the data layer without the added overhead of having to execute the application runtime and walk through the debugger line by line just to figure out if the return data is correct; it is much more efficient to simply execute the query and simulate the use case to find if the data that is returned is correct.  It becomes much easier and much less painful to simulate data access tests since they can be run, observed, and analyzed nearly instantly.

In larger organizations with dedicated DBAs, stored procedures have the added benefit of allowing SQL experts to add performance tuning to eek out extra performance without requiring the application to be rewritten or recompiled.  Again, we see this decoupling of the application layer from the data layer.  Of course you could always have templated SQL stored in XML files or something that would get rid of that recompile, but it is still likely to necessitate more redeployment if the application in question is distributed.  This key point is not to be taken lightly since -- as an example -- an error in string formatting may require the replacement of binaries and services across dozens of servers.  Not only that, testing in such a scenario still requires interaction with the application layer, adding to the possible failure points, time required, and general development pain.

My own conclusion is that using dynamic SQL (including LINQ) creates too tight of a coupling between the application layer binaries and the underlying data store; it's great for RAD and testing, but in any application of significance (especially in highly distributed environments), dynamic SQL at the application layer seems like it's a maintenance and testing disaster waiting to happen.

 Friday, August 17, 2007

Interesting Find

8/17/2007 9:32:30 AM (Eastern Daylight Time, UTC-04:00)

So this morning, I was contemplating signing up for Amazon Prime since the holiday shopping season is a mere three months away and because I just moved to a new house that's kind of out in the middle of nowhere.

In doing some research about the general satisfaction about the service, I came across an amazing find on the BusinessWeek website, of all places.

The article/post itself isn't so interesting or amazing in any sense, rather, it's the comments left by the readers that are amazing and utterly baffling.

Who would have thought that the readers of BusinessWeek are such giant retards who have no concept of how the Interwebs work?  Nay, who would have thought that they were so dense as to not have any concept of how customer service works?

As evidence, I present some of the comments to the article:

I had no idea that it would cost me 79 dollars to join the shipping club and would not have joined if I did. I don't order that much and don't care if it took longer.

I would like to cancel my membership and get a refund for that amount. The one purchase I made took longer then two day's anyway. It was not even paid for or shipped till 8 o2. I have not even receive any imformation about the club yet.

Than You
Donald TenEyck

Hey Donald, this is BusinessWeek, not the Amazon customer service page...get a clue!  I'm pretty sure that Jeff Bezos is not going to call you and apologize.

I have just received a charge on my Visa bill
for $79.00. I did not authorize this and will not
pay it. Please cancel "MY MEMBERSHIP" and I will expect confirmation from you immediately. I notice that I am not the only one that has unknowingly been charged. What a rip off. Please credit my account now. And then lose my email
address
Bonnie Smuffer

Is it really possible for people to be this dense and stupid and yet have access to the Internet?

And it goes on and on...the stupidity really reaches critical mass in this posting.  Of course, it's fully possible that it's just comment spam, but check out these two postings by "Jaques Laporte":

Hi AMAZON.COM

I have just received a charge on my Amex bill for $79.00.

>PRIME SHIPPING CLUB.
>Free Standard Shipping for eligible items shipped to P.O. boxes in the continental United States (excluding Alaska, Hawaii, and U.S. >territories, possessions and protectorates) and APO/FPO addresses with U.S. zip codes.

Note that I do not live in the continental USA.

I did not authorize this and will not pay it. AMEX will be notified.

Please cancel my so-called membership and refund me.
I don't want to spoil my relation with Amazon, but until total refund, YOU ARE NOT AUTHORIZED to bill directly my credit cards.
Take notice please that I currently have no payment methods in your system.

Best regards.
Jacques Laporte.

Posted by: Jacques Laporte at January 3, 2007 06:56 AM

And then:

I must confess that the refund was fast (2 days).
All's well that ends well.

Posted by: J. Laporte at January 5, 2007 12:38 PM

Amazing to think that there are such a large quantity of people with this level of stupidity...

 Thursday, August 09, 2007

I Hate Comcast

8/9/2007 2:52:02 PM (Eastern Daylight Time, UTC-04:00)

Over the past 4 years, I've been both a Comcast and Cablevision subscriber.  When I moved recently, I was "forced" into subscribing to Comcast due to whatever stupid regulations are in place limiting competition.

Let me list all the ways in which Comcast blows:

  1. Their customer service reps. are completely clueless.  It took me five call to three different people to figure out a stupid promotion that they were running that required you to be a customer already.  But clearly, on the ad, the terms and conditions state that it's available to new and existing customers.
  2. Their sales reps. are retards or just fuckups.  So to ensure that I got the exact package I wanted, just before I hung up with the rep., I specifically went over each line item to ensure that I was getting what I wanted.  Yet when the installation tech showed up - a day early, but an hour late - he didn't bring the right box!  I looked at the box to see if there were component cables and as I suspected, there weren't.  So I state "Hmmm...this doesn't look like an HD box" to which he responds "Yeah, I figured you wanted an HD box (gestures to TV)...those fuckups in sales...".  I check the order and indeed, not only was I not signed up for the package that I wanted, I was not signed up for HD either.
  3. Their customer service sucks.  So my wife goes to the local field office to tell them that they messed up their order.  Instead of offering for a tech to come out for free to claim the box and install an HD box, they hand her a new box and ask her to return the old one by Saturday.  That's quality service for ya' right there.
  4. The service sucks ass.  This deserves several sub-points:
    1. The digital voice comes with a shitty modem that has led to noticeably slower Internet performance (i.e. loading pages, sending files, etc).  You have no choice but to use this shitty modem.
    2. The digital cable is very unstable...many of the digital channels consistently lose integrity (blockiness, screen blacks out).
    3. The HD selection sucks.
    4. The fastest cable internet option costs $53 per month and it's only 8mbps downstream (and something like 1.5mbps up).  Contrast this with Cablevisions Optimum Online with Boost which offers 30mbps down/5mbps up for $44.95+$14.95.  Ho-lee shit.  There is no comparison.  To make matters worse, Boost comes with:
      1. A static IP (OH SCHNAPS!!)
      2. TOS allows outbound HTTP traffic over port 80!
      3. TOS allows outbound SMTP traffic over port 25!
  5. Comcast is expensive.  The Digital Preferred, which you need to get ESPN HD, costs $74/mo.  Plus, on top of the cable cost, you have to pay an extra $5 for HD channels and rental fee for the HD box.  Cablevision?  HD is included in all of the packages.  Plus, you can get most of the HD channels with the Silver Package @ $68/mo.

So in conclusion, Comcast sucks.  Cablevision has service to the zip code, but not to the street address...so what can I do but suffer :-(

 Wednesday, July 18, 2007

Mindblowing Job Candidate Trawling...

7/18/2007 3:20:24 PM (Eastern Daylight Time, UTC-04:00)

The following are the contents of an actual email that I received this afternoon:

Hi cchen

I have a very hot Java Developer rek with a direct-client in Manhattan. Please send me candidates with rates and contact-numbers.

Interviews this week.

Thanks
-Priya M.

I would be shocked if Priya lands any candidate this year.

Let's dissect all of the ways that this email is completely ineffective:

  1. I am a .Net developer.  I know Java and worked with it for 4 years on college, but I have not worked with it in any sort of significant capacity since then.  Assuming this email was generated based on my profile on either Dice or Monster, you'd think that the recruiter would attempt to filter the candidates...
  2. "Hi cchen"?  Why am I being addressed by my email account name?  I'm pretty sure that I have my name listed on Dice and Monster.  If this were to have come from some third party database somehow, wouldn't it make sense to just use something like "Good afternnon,..."?
  3. "rek"?  Take some time and type out the extra keystrokes and spell it out.  A little professionalism goes a long way.
  4. No.  I am not doing your job for you.

Contrast this with another email from a Bradley G.:

Dear Applicant,

My name is Bradley and I'm an IT recruiter at [XYZ]Technologies Corporation. Our records show that you are an experienced IT professional with experience in .NET. This experience is relevant to one of my current openings.

The opening requires ASP in addition to the above skills.

It is located in Pittsburgh, PA.

If you are qualified, available, interested, planning to make a change, or know of a friend who might have the required qualifications and interest, even if we have spoken recently about a different position. You may also send me an e-mail with a copy of your current resume. If you do respond via e-mail please include a daytime phone number so I can reach you.  In considering candidates, time is of the essence, so please respond ASAP.  Thank you.

Sincerely yours,
Bradley G.

If you are not currently seeking employment, or if you would prefer I contact you at some later date, please indicate your date of availability so that I may honor your request. In any event, I respectfully recommend you continue to avail yourself to the employment options and job market information we provide with our e-mail notices.

Much more professional and much more likely to actually attract a candidate, but it still gets a few items wrong:

  1. I did not apply for any job.  It would have been better to use use a simple greeting like "Good afternoon" or "Greetings" or perhaps even something creative like "Hello from IT land!"
  2. I would have added "please responsd ASAP"...seems like a bad sales pitch.

Just some random afternoon ranting...

 Sunday, July 15, 2007

Commitment Chains, GUIs, Frustration, And Other Ramblings...

7/15/2007 11:29:58 PM (Eastern Daylight Time, UTC-04:00)

Warning: massive brain dump ahead...

As I was laying down to sleep and having a discussion with my wife - much to her dismay - the topic of her current graduate class came up and she mentioned how much she enjoyed just sitting down and writing for 45 minutes each class.  I found it strange that she should put it in such a perspective.  I mean, there's nothing preventing her from taking the time to sit down and write for 45 minutes each day (and she did keep a journal up until maybe 3 or 4 years ago) as surely, countless minutes of her day (and any average person's day) is spent doing mindless things like watching television or eating or something else equally useless.

The idea of commitment chains occurred to me as I was using an analogy about exercise and trying to convince her that writing for 45 minutes each day is relatively trival compared to working out.  Think about it: in exercising, one starts a chain of commitments which can seem unconsciously daunting.  To exercise is to sweat, to sweat is to necessitate an immediate shower (well, unless you don't mind body odor or the salty stickiness of sweat), to exercise necessitates a larger load of laundry, and most importantly, in this proposition, is that it necessitates a healthy lifestyle lest that exercise went for naught. 

It is a relatively large commitment chain to make simply by exercising and perhaps this is why so many people find it so difficult to maintain a healthy lifestyle: the weight of this commitment chain is simply too heavy.  On the other hand, writing for pleasure carries little commitment of any kind.  You write if you want to, you don't if you are not in the mood.

What's the point?  No point, really :-D  I guess if there was a point, then perhaps it is that very often in life, we don't really take into consideration how little commitment it takes to do what we want to do and do what we enjoy.  We also fail to realize how these low commitment activities have a profound effect on our lives as they help us feel like we've done something.  Simple things like taking a stroll around the block, watering some flowers, laying down and watching the clouds pass, sitting with a cat on the grass, drinking a cup of lemonade on a hazy summer afternoon.  Perhaps that's the secret to finding balance in life: to have a healthy mixture of tasks with long commitment chains (work, family, health) mixed with activies of low commitment (I'm mixed on whether blogging is the former or the latter, but I do find it constructive to put thoughts to text some times).

Shifting gears now.

Prior to this discussion, we had another discussion about how we visualize dates.  I was thinking back to something that I had once read about how to interview tech candidates: propose that some object typically comes in a set of 14.  Now 5 additional elements are introduced...ask the candidate how he or she would organize the new elements.

Some people, like my wife, would tend to place the 5 elements "below" the 14 elements and line them up and start to form a multidimensional array - or a matrix, if you will.  Some people like me, would visualize it as a separate block of elements, but in a linear manner...more like containment where the first set contains 14 elements and the second set contains 5, but they are part of yet a larger set.  It is less of a repeating pattern and more of a general grouping.

This manifested itself clearly in the way in which we think about and visualize dates.  For her, as day of the week is important, she tends to organize her events and key dates in a typical calendar fashion and in fact, she can visualize it so well, that given one event in a month, she can probably tell you the day of the week of any other date in the month nearly instantly.  She views the set of 7 days in a week as a part of a matrix much as a calendar is typically visualized.

In my case, as day of the week is generally not that important, I visualize date and time as linear and quite abstract (I think the most natural way to think about it since it really is linear and absolute...it is only the incidental cyclical nature of our orbit around our Sun that defines constructs like seconds, minutes, hours, days, weeks, years and so on).  In my case, I am terrible at remembering dates and I am terrible at remembering order; I only roughly index that I have something to do some time in the future.  Ask me what I'll be doing two weeks from now, and it'll take me a good amount of time to figure that out whereas my wife's response will be nearly instantaneous.  I tend to think of time in blocks where I have commitments (meetings, errands, and so on) and blocks where I don't have commitments.

When you really think about it, time itself is completely abstract (what is it? will it end? when did it start? how much of it is there? what does it look like? what is the absolute unit of time?  can it really even be counted?), but the organization and demarcatinon of time into units seems...weird and useless to me; I am fine thinking about it in the abstract (i.e. "some time in the future, I need to do this") and not as an absolute (i.e. "on such and such date at such and such time, I need to do this" or "x units from now, I need to do this").  

There is a parallel in my profession: as a software developer, there is nothing tangible about the constructs that I build; the contructs that I build are purely abstract in nature: every GUI, every construct in software, is but an abstraction of numerous lines of code - or, is it the other way around?  Software is but one layer of abstraction on top of another...modern day software could not exist without the huge levels of abstractions that have been built to allow programs to be written efficiently.  Buttons are not buttons, they are rectagles.  Rectangles are not rectangles, they are arrangements of lines.  Lines are not lines, but merely a linear set of pixels.  But in essence, there is nothinig to grasp and to utilize to visualize proportion, all of it is purely hypothetical and kind of "uploaded" into my brain as a set of objects, relationships, and other abstract constructs when I sit down at my desk in the morning. 

In actuality, I find this process of uploading and unloading quite unpleasant (particularly the unloading part).  I have been told by my coworkers, wife, and family members that I can become quite unruly when I'm involved in my work.  The reality of it is that when I'm in my groove, unloading and then loading so much abstraction and so much data causes some sort of mental instability...I just get frustrated at the individual forcing the purge or I just lose my groove and have to kind of veg for the rest of the day...I simply cannot be constructive.

From an observer's perspective, I think this makes me seem like a loner or anti-social or if a colleague is coming to me with questions, it may seem like I'm impatient or uncooperative.  In reality, my bitter reaction is more of a defensive mechanism to kind of keep myself from having to go through these periods of derailment as in my case it's not a temporary derailment...it's like a long term derailment once it happens as there is simply too much data to store and reload that it's taxing on my mind.

For this reason, I think I've recently been in some hot water with some coworkers.  I simply don't take afternoon interruptions very well as that is the time when it is hardest to recover from derailment at that point.

Of course, the whole reason that this discussion and train of thought came up was the movie Stranger Than Fiction (it's an absolutely brilliant screenplay with an absolutely excellent performance by Will Ferrell (everytime you think he's going to break into his "normal" genres, he surprises you and keeps his acting true to the character...a brilliant perfomance)). 

This movie draws my attention on various levels: it is at once a deep inspection of what it means to live and to be alive, it asks what exactly is the scope of one life in the grander scheme of the universe, on some level it is a movie about religion (I haven't really fully formulated this part of it yet), and of course, it's a touching romantic comedy :-).

I also found the specials (and this isn't the first time) to contain some very insightful information on teamwork and project management that would apply to almost any field (but that's a discussion for another day).

What also caught my attention was how director Marc Forster and the visual effects team realized how Harold's thoughts were visualized with these planar "screens" with metrics, text, and data layered together.  It's much the same way I visualize data, code, structures, and tasks, all on virtual screens that I slide around, stack, layer, and intermingle.  I now realize that there is no organization to how I think about these constructs and abstractions...I simply see them in my mind as if before me was a stack of cards strewn about and yet I am able to reach out and pluck the ace of spades at will with no effort.

Maintaining such mental order requires a lot of effort and a lot of concentration.  I think it is because of the amount of effort required to work the way that I do, that I am so unpleasant when interrupted (much to the dismay of my wife, mother, and coworkers).  And believe me, it's not that I don't like to help others with the development issues or educate other developers and team members, rather such tasks are not my primary concern and shifting gears is extremely difficult when you have to maintain such large abstractions and structures in the mind.

So of course, the question is, what is the solution?  Well, perhaps I need to invest some time in some organizational books.  Perhaps I need a whiteboard to help unload some of the data and make it easier to reload as well.  Perhaps I need a bigger desk so I can scribble more and keep better notes.

Well, I think that about wraps this up.  Possibly not the most coherent or well organized entry, but it contained data would have kept me up all night if I didn't unload it :-)

 Tuesday, July 10, 2007

Software, Artistry, and Frustration

7/10/2007 10:52:46 PM (Eastern Daylight Time, UTC-04:00)

In describing my approach to software development, I like to use the term practical artistry.  What does this mean exactly?

Well, the practical part of it is that the class libraries, interfaces, and components have to work the way that tey were designed.  They should also be easy to use, easy to understand, easy to integrate with.

The artistry portion of this term is much harder to quantify.  Just what is artistry when used in the context of software development?

Art Tatum offers a very compelling definition:

Art is a method of communication which unifies surface details and form while taking both the intended meaning and aesthetics into account. This requires significant amounts of problem solving. The artist is constantly asking, "How can I best express this idea without ruining the proportions of the work as a whole."

This ties in with Fred Brooks' principle of conceptual integrity for it is the artist alone who sees the proportions of his work and the artist alone who shall ensure that the work abides by the guidelines of the orginal design intent.

Another term that I like to use is exemplary craftsmanship.  This concerns the little details that make code aesthetically pleasing, readable, and well crafted.  Many small details affect this quality of code such as consistent naming schemes, using extra keystrokes and not abbreviating non-standard terms, ensuring that spacing is consistent, formatting code in a consistent manner so as to make it more readable, and commenting public APIs.  On a higher level, it concerns the organization of code and the clear separation of domains (not in the Fowler sense, but in a more abstract sense) in a manner that enhances the extensibility and orthogonality of the code.

It's not just code, it's any trade.  A panel and house wired by a master electrician will certainly be different than a panel and house wired by an apprentice.  The cabling will be neat, the runs will be well thought out, the circuits will be well labled, the panel will be well organized, little details will have been considered, and the artistry of the finished work is apparent even to laymen.

With this in mind, I consider myself to still be an apprentice; I still seek to learn the trade from a master craftsman and I still seek to hone my skills and develop my artistry so that I may also craft software of a high caliber.  But I work hard to ensure that some sense of practical artistry and exemplary craftsmanship is apparent in everything I do.  From simple tasks like ensuring proper indentation in my source files, selecting the right margins in my documentation, and using the right fonts to more complex design issues like organizing libraries in proper dependency chains, achieving orthogonality in modules, and organizing objects in consistent and well defined fashions (i.e. utilizing design patterns).

Unfortunately, in my short career, my interactions with other developers have left me disappointed on this front for the most part except for three individuals whom I didn't have nearly enough time to interact with (this is not to say that I haven't worked with many fine developers, but three stand out as practicing these principles of craftsmanship).  These three were true craftsmen in the sense that the little details mattered to them.  Improving their skills as developers was an important aspect of everyday development.  Writing good code and following well known guidelines and principles meant something.  The naming of every class, of every variable, required at least some passing thought so as to ensure that each construct was congruent and aligned with the whole.

Of course, such discussion of artistry as it applies to software is not just frivolous academia, Maarten Boasson writes in The Artistry of Software Architecture:

Designing software is not very different from designing any other complex structure: Few people are good at it; no single recipe always produces a good product; and the more people involved, the smaller the probability of success.  On the other hand, a design produced by someone who is good at design provides an excellent basis for long, reliable service.

Software engineers consider the artistry of the design not only evaluating aesthetics but also the practical results of such a design such as orthogonality and added extensibility.  Boasson further comments:

In exceptional cases, a good software design is no less valuable than the great masterpieces that have been created throughout our rich history.  Examples of both bad and good designs can be found all around us, in almost every engineering field; practically everyone recognizes a piece of art when they see it.

So I often wonder why it is the case that I encounter and, of much greater concern, find high degrees of tolerance for bad design.  Not just bad design, but bad development practices like inconsistent usage of formatting elements (spacing, newlines, tabs), naming namespaces and classes against well established guidelines and practices, and other details like inconsistent casing.  Leadership just doesn't seem to care for the most part and it requires the rare and truly inspired individual project manager to understand the long term value of encouraging practical artistry and exemplary craftsmanship.

Bear in mind: it's not that I approach writing software with any sort of artistry or snobbery in mind...indeed, a good portion of it is the grunt work - simply putting the hammer to the nail, or putting the brush to the canvas, so to speak.  But at the end of the day, there is a personal satsifaction that is achieved from not just writing any code, but writing good code.  There is a satsifaction that comes from recognizing and implementing a superior system design.  Of course, it goes beyond personal satisfaction, good design, as Boasson writes, can provide long term value in the form of extensibility, maintainability, and reusability.

In almost all cases, as the majority of developers are not self motivated to write such code, it takes strong leadership, clear definitions or design guidelines, and enforcement of the policies to ensure that quality software - not just working software, but quality software - is crafted.

To me, it is the little details that go towards creating a better product.  There is certainly a time and place for prototyping and RAD - and certainly, I utilize these techniques all the time, but there is also a time to formalize the lessons learned from such exercises and to create a masterpiece...to write code that you would find have no qualms about showing to the world and exposing it to critique.

It is with this in mind that I find myself currently flustered.  Is it just me?  Am I being too uppity about all of this?  It's hard to say...I am truly conflicted about this as I cannot see how I can work productively and cooperatively in a team with people who do not honor the same sense of craftsmanship and artistry.  I awake and find that someone has scribbled on my canvas in a weak imitation of my style using colors that clash with the existing palette.  Analogously, I open the electrical panel to find that someone has stuffed some low grade wire haphazardly into the panel without any clear labeling.  I cannot help myself but cringe at the thought of integration - yes cringe.  I don't want to deal with ugly code and yet the leadership doesn't seem to care one way or the other and I am powerless to affect change (partly because I am a blunt edge and possess no sense of finesse whatsoever in dealing with these situations)...*sigh*

I can only hope that some of my desire to achieve practical artistry in code and design inspires others on my team, otherwise this will be painful to endure.

 Friday, June 29, 2007

Two Terrible Rulings

6/29/2007 11:36:46 AM (Eastern Daylight Time, UTC-04:00)

Opening up the newspaper this morning, I came across two terrible rulings recently passed down by the Supreme Court.

The first has to do with MSRP.  I'm guessing that the majority of American's who are not filthy rich often go bargain hunting for prices below MSRP in hopes of finding that killer deal.  I'm guessing that if you're like me, you're willing to sacrifice some service at the point of sale for a better price.  This is part of principle of what has driven online sales: for the retailer, they save by not needing to have sales people or point of sale service while for customers, they often get a chance to save a huge amount over retail outlets.

From USA Today (6/29/07):

The 5-4 decision overturned a 96-year-old law that prevented manufacturers from setting minmum retail prices.  The majority wrote that lifting the pricing ban could benefit consumers if retailers offered better service or selection.

Wait, what?  How about benefitting consumers by offering a better price?  Why not leave it up to the retailers to choose which model they choose to bring in customers?  If customers wanted service, they could choose the retailer that offered it.  If they wanted to find a bargain, they could choose the retailer with the best markdowns.  Instead, the court has decided that the consumer has no say in this...no freedom of choice when it comes to price shopping.

What I found was completely asinine was a statement by Richard Doherty:

Richard Doherty of technology market researcher The Envisioneering Group agrees, saying the price ruling could lead retailers to use more free products and better services as sales incentives.  "It's sure to be to consumers' benefit this summer and Christmas."

Is this guy on crack?  How about I just want to get a good deal on a HDTV...I don't want any "quote-unquote-FREE" stuff.  I don't care for point of sales services; I do my research on the products I buy and I already know what I want when I go into the store; I want to get the best deal that I can...why not leave that decision up to the individual consumer?

To make matters worse, the article offers commentary from a multi-billionaire:

Bill Gates, of golf equipment maker Ping, says, "Not every consumer is a bargain shopper.  Some consumers are looking for quality, innovation, personalization and customer service when they shop."

That's fine, Bill, if you have the money for all that jazz, but how about the rest of us who are just trying to put away some money for retirement, for our kids, for our families?  We just want to get by with a good deal.  How about this novel idea: let the consumers decide what they value - price or service.

Argh!

As if this wasn't bad enough, the Supreme Court also ruled to strike down school diversity programs on a national level:

The dramatic 5-4 decision throws into legal doubt programs that factor in race, including magnet schools that use race to draw students from different neighborhoods.

Accorind to Chief Justice Roberts,

Classifying and assigning schoolchildren according to...race is an extreme approach.

Well then, what would you suggest Mr. Roberts?  How do you overcome the economic hurdles that create defacto segregated schools?  How can you deny that integration - even if it is forced - is to the benefit of our racial diversity and our social fabric?

The fact is, if school children were simply assigned to schools based on location and district, it would in fact create further segregation and slowly reverse some of the progress that has been made.  I think it is true that we fear what we do not know or understand.  We view those who are different from us as outsiders because we do not understand their cultures, their languages, their traditions, and their ways; what better way to overcome the defacto segragation that occurs by township (and economic boundaries) they to help ensure a minimum level of integration?

Such an upsetting way to start the day...

 Tuesday, April 17, 2007

More Buffoonery From The Music Industry

4/17/2007 8:56:21 AM (Eastern Daylight Time, UTC-04:00)

Argh!

A panel of judges at the Copyright Royalty Board has denied a request from the NPR and a number of other webcasters to reconsider a March ruling that would force Internet radio services to pay crippling royalties. The panel's ruling reaffirmed the original CRB decision in every respect, with the exception of how the royalties will be calculated. Instead of charging a royalty for each time a song is heard by a listener online, Internet broadcasters will be able pay royalties based on average listening hours through the end of 2008.

Why does the music industry insist on treating the consumer like an enemy?

It doesn't make sense to me and I don't see how it can make sense to the artists.  Many of these internet radio sites play tracks which otherwise would get no airtime on radio, which otherwise I would never have heard, and from artists which I would have never discovered.  Zero 7, Air, Bebel Gilberto, Susumu Yokota, and on and on...

I know this isn't the case for everyone, but at least in my case, internet radio has led to the purchase of almost all of my CDs in the last 4 years and these are albums which otherwise would have little mainstream exposure at all.

I just don't get the logic that the music industry is operating on...

 Wednesday, April 11, 2007

Random Developer Rant

4/11/2007 8:52:37 PM (Eastern Daylight Time, UTC-04:00)

I have to say, one of my big pet peeves in terms of software development is evangelizing by "senior architects" who haven't written code in years.

I'm not against evangelizing a product or technology in general, but it seems so much less like bullshit when it comes from people who are in the trenches, dog fooding the technology they are evangelizing.

The real problem arises when some developer who's been out of the loop evangelizes without having used the product in a real world development environment. Or even worse, sometimes, you get guys that evangelize about practices, architecture, and design. Bleh! These guys are the absolute worst! Lots of things make sense in an ideal world where software deadlines are always met ahead of time :-) (I love those Walgreen's commercial) but the reality is, no one lives in that world.

But in any case, one of our codebases at Zorch recently underwent a code review. As a part of this code review, the team at our counterpart ran some code analysis tools against our codebase. Of course, I have no objection to this in principle, but what did bother me was that they didn't really even analyze the results of the code analysis output. They skimmed for "reds" and basically just took a look at the count of warnings. To make matters even more asinine, the individuals performing the code review took at least 3 minutes trying to find out where the code analysis utility was located.

Okay, I must at least give a disclaimer: in general, I like the idea of code analysis and good practices. I like the idea of general guidelines in terms of coding conventions. But I'm not a Nazi when it comes to policing code - I was at one point in my career, but these days, I figure that as long as it's not terrible (i.e. lots of misspellings, inconsistent patterns, well known bad practices like massive string concatenation), I'm not one to harp on these small details.

Just as each writer has a different style, just as directors have different styles, and just as a painters have different styles, I realize that developers have different styles as well. I try to point out obvious bad mistakes when I see them (like using DataSets without really using DataSets), but I think otherwise being too anal about this stuff is counter productive because, you know, this stuff can always be cleaned up later - it's way more important to get the product to work first.

Now with that out of the way, what kind of bothered me the most about this whole event was that after the output of the code analysis was generated, we didn't review a single warning that was generated to validate whether there were flaws with our codebase. I suspect that even if we did, these guys wouldn't have known what half of the warnings meant. What a waste of time. I could have ran the review and emailed it to them if that was it.

So, that's my rant for the week :-)

 Tuesday, March 27, 2007

Am I Just Being Compulsive?

3/27/2007 10:48:37 AM (Eastern Daylight Time, UTC-04:00)

I waked up this morning with some notices that new code has gone into our main repository.

So of course, I perform a checkout to get the new code and what do I get?  Argh!  The commiter added the _svn directories (possibly from a previous working copy) into the repository!  So of course, the checkout fails and I need to revert to a previous version.

Now I'm a little annoyed.

Then I go in and check the directories to confirm that indeed, the _svn directories were checked in like any other directory.  I also find that our naming rules, which I had thought we all agreed on, are not being followed.  In addition, we had discussed building all projects off of the main solution and building sub-solutions from that main solution trunk.

Nope.  We have a solution sitting in it's own directory.  To top it off, it's a solution file that doesn't follow naming conventions.

Then there are other various little things which just add to the morning frustration; the use of "Sharepoint" instead of "SharePoint", "Engr" instead of "Engineering", inconsistent naming of image files, the existence of an App_Data directory (why is there an App_Data directory in this solution???????), referencing obsolete libraries (I could swear we discussed this)...

These little things all matter in the end in creating a sense of professionalism and coherence in the final product when DLLs and classes are named according to guidelines, file naming is consistent, and spelling (misspelled words in code annoy me to no end) is correct.

Man, these things just grate me so much.  I dunno, am I just OCD about my code?  This kind of stuff just drives me batty.  It literally freezes my brain until the issue is resolved.  Nobody seems to care much about the artistry and craftsmanship that goes into their code...that is until we need to do a customer or partner facing code review, then the only code they want to put up for review is mine...

*Sigh*