Wow.

and

Wow.

I think that's all I need to comment on these

As I was laying down to sleep, I started to think about the next generation Gameboy (GBX, Gameboy Next).  Honestly, I don't remember the train of thought that lead me to thinking about it, but I was sooo engrossed, that I had to get out of bed to jot down ideas and what not.

The first thing that came to mind is what type of media would Nintendo choose to use?  I think that any sort of optical or magnetic disk type media would be way too inefficient from a power and loading time perspective.  Clearly, Nintendo has always placed a big emphasis on quick load times, which are essential for portable gaming systems.  In addition, Gameboys have a rich tradition of looong battery life.  Disk based media require spinup time, which negatively affect load times.  So the only thing that comes to my mind is flash media (or small format hard drives, if they're cheap enough and sufficiently durable).  It will likely be a proprietary format (for reasons that will be discussed below). 

But to distribute flash media with each game is inefficient and costly (as was always the main issue with cartridge based systems aside from the size limitation).  We have already been told by Nintendo that the Revolution will offer games for download.  It would seem like this would also be the obvious choice for the GBX, with one radical difference: the games will be download only. 

Yes, download only.  This may sound bad for stores that sell games, but consider the facts: 1) stores will have an advantage in that they can distribute game related materials (manuals, freebies, etc.), 2) not everyone will have access to an internet connection, so stores will still need to have download kiosks, 3) stores will allow users to validate copies of existing software titles so that users can download.  That last point is of particular interest as it means that the GBX will have backwards compatability by allowing users to download copies of their old games.  We have a precedence for this as Revolution will allow users to download old NES and SNES games (and who knows what else, maybe even Sega Genesis games?!).  On point 1, all manuals will be made available online in PDF format for download.  On point 2, an internet connection will not be required to play the game, only to download the game.

All downloaded games are portable across units, but not across media.  What this means is that you can download a game to a particular media and you can then use that media in another unit to play the game, but you cannot copy the media.

Flash memory is relativley cheap nowadays, with retail prices for 1GB of memory ranging from $40-50.  For comparisons sake, the Gamecube disks are 1.5GB in capacity.  Keeping in mind that this is a portable system meant to be played on a small screen and the fact that flash memory prices will drop significantly in the next 1.5 to 2 years (the timeline for the GBX), we can postulate that a 2-4GB flash unit at $40-50 could hold a good number of games considering that the current DS memory cards are only supported up to 128MB.  Of course, the games themselves will be cheaper as the overhead of distributing the games is significantly reduced.  The cost of printing the games is completely eliminated.

The advantages of using flash media and downloads is easily apparent in the cost savings for Nintendo and the convenience for the user.  Using solid state memory allows for significant power savings and reduced loading times compared to magnetic and optical media.  For game saves, the GBX can either reserve game save space on the download media (for example, if the game is 120MB, 10MB may be reserved for the game saves for a total footprint of 130MB) or perhaps use a seperate, more conventional (non-proprietary) media, for game saves.

So why is a proprietary media required for the downloaded games?  The reason is that it must support certain measures to ensure that games are not duplicated (or at least not easily duplicated) and/or pirated.  More specifically, it must contain a write only section that cannot be altered. How does this all work out?  I'm glad you asked

• Each media will have a unique identifier (UID_Media)
• Each media will have a private key (K_V,Media) and a public key (K_U,Media)
• Each GBX unit will have a global public key (K_U,Global)
• Nintendo servers will have a master database that contains the unique ID (UID_Media) for every media manufactured along with the public key for the media (K_U,Media)
• Nintendo servers will also have a private key (K_V,Global)

Certainly, there will be some sort of handshake procedure and what not to setup the connection for browsing game catalogs and initiating the download to ensure that only registered hardware (registered when manufactured) can connect to the servers, but I'm only going to cover how a theoretical download scenario could work after the handshake.

(I aplogize for the unconventional notation, as I'm too lazy to go in and format the HTML properly, so just follow along.  Also bear in mind that this is a very high level overview.)

1. <Unit> M₀ = Encrypt(K_U,Global(UID_Media)).  The first step is to create a message by encrypting the unique ID of the media using the public key of the Nintendo servers.  This ensures that only Nintendo servers, which have the private key, can decrypt the message and map the unique ID of the media to the public key of the media.  The message is sent to a Nintendo server.
2. <Server> UID_Media = Decrypt(K_V,Global(M₀)).  The server decrypts the message from the unit using the server's private key.  This results in the unique ID of the media.  The Nintendo servers contain a key map of media unique ID to media public key.
3. <Server> M₁ = Encrypt(K_U,Media(K_Shared)).  Using the public key of the media, a shared key is encrypted to create one part of a message.
4. <Server> M₂ = Encrypt(K_Shared(File_Game)).  The game binaries are then encrypted using the shared key.
5. <Server> M_F = M₁ + M₂.  A final message is created by encapsulating the encrypted shared key and the encrypted game file.  This composite message is then returned to the GBX unit.
6. <Unit> K_Shared = Decrypt(K_V,Media(M₁)).  The GBX unit obtains the shared key by decrypting the first part of the return message using the private key of the media (remember, it was encrypted using the public key of the media which is stored at the server).  The shared key is never stored in an unencrypted form.  Each time a player loads a game, the shared key is decrypted again.  Only the encrypted form of the shared key is stored (perhaps the unique ID of the media is also stored in the message as an added measure).  Because the shared key is encrypted with the public key of the media, only the private key of the media, contained in a read only region of the media, can be used to obtain the shared key.
7. <Unit> File_Game = Decrypt(K_Shared(M₂)).  The game file is read using the shared key.  Decryption is done in real time using hardware level decryption for performance reasons.

Essentially, this would be a form of DRM where the rights are associated with the media, not with the unit.

Bill Gates was straight on in commenting that the HD-DVD vs Blu-ray format war is insignificant due to the fact that this will be the last significant physical media (from a distribution perspective) for quite a while (at least when it comes to consumer electronics; holographic storage will eventually become the standard in ultra high capacity data storage).  Nintendo, I think, will be the first gaming company to move away from distributing physical media altogether by switching to a download only type of service for its next gen portable console.

Other random thoughts on the console are:

• The DS screen resolution is currently 256x192 (for each screen).  PSP is 480x272.  I expect that, with the improvements in LCD and processor technology in the next two years, the GBX will have a resolution higher than the PSP (although we all know that Nintendo has a habit of undervaluing graphics capabilities).
• It will have a 6 button design similar to the DS.  The current GBA has a 4 button design (A,B,L,R).  I picture a setup more like the GCN's, however, in that it will be three smaller buttons surrounding one large action button.
• The unit will have a built in gyroscope.  This ties into the Revolution and some of the experimental games on the GBA which have built in motion sensors (WarioWare Twisted!).  Racing games, flight sims, etc. will be totally sick on this machine.  In addition, it may also connect to the Revolution as a wireless controller.
• Following in the vein of the SP and the Micro, it will be slick.  It will be sexy.  I picture it somewhat like an iPod Nano in terms of finish (except it'll be more resistent to scratches).
• It will have built in wireless capabilities.  We see that Nintendo is finally coming around to all of this 'Net gaming and really embracing it (Mario Kart DS).
• To enforce a kid-safe environment, as each unit will have a unique ID, Nintendo can create an architecture whereby each conversation and each exchange of text is logged and scanned in an asynchronous fashion.  Other users in a conversation may also choose to explicitly tag a conversation as breaching the terms of service.  Essentially, it would require a massive grid of computers to scrub recorded voice and text data for abuse.  In turn, Nintendo can punish those users by disabling voice and text capabilities (on the Nintendo network) for an increasing period of time with each infraction.
• There is a very distinct possibility that we will be seeing an emergence of large capacity, small format hard drives in the next year.  This is related to the recent developments in storage design.  Specifically, perpendicular storage technology, which promises to increase disk density significantly.  Anywhere I've used "flash memory", it may very well be replaced with a micro harddrive boasting 20-40GB.
• I think it'll look like the OQO ultraportable in terms of layout (the screen slides up to reveal the input buttons), except not as wide.  This would be inline with the design of the Gameboy Advance SP "clamshell" and would be great for viewing media when not gaming.  Which leads me to...
• The GBX, contrary to Nintendo's typical stance on building pure gaming machines, will be a multimedia platform as well.  With the emergence of cheap, large capacity storage and the competition (Sony), it will be hard for Nintendo to ignore this functionality.
• And finally, this being Nintendo, we know that there is going to be some sort of innovation that hasn't been done before on a handheld gaming system.  I predict that this will be stereoscopic 3D.  Yes, you read that correctly.  Sharp has already developed an LCD for cellphones which has this technology.  What's great is that the effect can be turned off in case it causes headaches and what not for certain users.

Okay, that's enough babbling and speculation from me.  Time to sleep damnit!  I dunno, I've somehow managed to hype myself up over my totally fabricated speculation

But mark my words, I think what I've outlined here will come to be in the form of the next generation "Gameboy".

I came across a good article on prototyping as I was reading about some of the reactions to the rumors that the Nintendo Revolution hardware will not be capable of much more than the current Gamecube hardware.

In my senior level software engineering course, my professor discussed that one of the pitfalls that was associated with building prototypes is that different audiences have vastly different expectations and draw different conclusions from prototypes.  For example, an architect that builds a prototype to demonstrate a new portal architecture doesn't necessarily have to make the UI look pretty.  Other developers can appreciate the architectural design elements such as how requests are handled, how state is handled, how communication is handled between different components and so on.  But show it marketing or sales in the same state, and the project could be dead in the water before it even starts, even though it may be the best portal architecture ever designed.

I think Nintendo took a big risk by showing demos of the new controllers in actions so early.  While it was necessary to prove that the new control schemes could work and work well, there is a (mis)perception around the gaming community that the early prototypes represent the final product in terms of graphical quality, even though we currently have no idea what kind of hardware the demos were running on.  The average Joe looks at the graphics of the current gen. hardware and then looks at the demos and will promptly proclaim that the Revolution sucks without realizing the true level of innovation and potential of the Revolution.

The article raises the question: "Is the Prototype the Production System’s Ugly Sister?"  Most of the time, yes.  But in some very special cases, the visual elements of the prototype, while they may have no significant impact from a software engineering perspective, become deal breakers:

The prototype does tend to be pretty ugly, and deliberately so. Absolutely zero time should be spent trying to make it look appealing. There are, however, a few notable exceptions to this:

• The prototype’s goal is to prove that a particular technology can be made to look good (e.g. writing a GUI client with Java Swing). In this case, the prototype would be more of a proof-of-concept, and would consist primarily of GUI tricks and best practices to get the best out of the platform and widget toolkit being used. In fact, you would spend less time concentrating on actual functionality, because that’s not the point of the prototype!
• The customer has difficulty distinguishing between form and function. If this is the case, it sometimes pays dividends to spend a little extra time making the prototype look good. Don’t go overboard though—remember what you’re creating is going to be abandoned pretty soon, in favor of a brand spanking new, shiny version with all mistakes learnt (at least that’s the plan!)
• The prototype will also be used by Sales & Marketing, as an early sales pitch for your upcoming new product. In this case, looks and shininess (the “ooh” factor) probably take priority over real demonstrable functionality. Showing the customer your impressive 20-megabyte auto-generated comma-delimited invoice file might bore them faster than you think.

These are the exceptions, though. Most of the time, the prototype is intended primarily as a means of gaining insight into the ideal architecture, and is written in tandem with the design—before any production code gets written.

I've bolded the portions that I found to apply to Nintendo's current situation.  While I think that most people can understand that those demos were really designed to show off the controller, they have, nonetheless, inadvertantly relegated the Revolution to "has been" in the minds of many gamers that have associated improved visuals with "Next Gen".  So Nintendo has made the unfortunate mistake of not making the demonstrations "pretty" enough, even though they were mainly meant to demonstrate that the control scheme could work.  Of course, Nintendo can still blow us all away with XBox 360 level graphics, but they've already lost the attention of a percentage of hardcore gamers for whom graphical prowess is king.

The one thing to remember is that Nintendo has been working on this next gen. hardware for quite some time now.  There will be a new CPU from IBM and a new GPU from ATI.  I think the reasonable conclusion is that the graphics for the Revolution will be significantly better than those on the Gamecube and first gen. games will likely be on par with first gen. XBox 360 games.

I came across an editorial on CNNSI.com on, among other things, the play of Baron Davis and how he's helped turn the perpetually bad Warriors around and get them off to their best start in the franchise's history.

While the editorial relates to basketball, I think it serves as a great analogy to anything in life that requires leadership and teamwork.

"No disrespect to the point guards who were here in the past, but Baron's the type of point guard that makes everybody better," says backcourt mate Jason Richardson. "He gets in the lane and just finds guys. I been waiting for a guy like him to come and he took my game to another level."

This is not, incidentally, the type of quote one reads about Marbury, despite his career averages of 20 points per game and 8 assists. But this type of appraisal, I would argue, is a simple, but fail-safe way to assess any point guard: do his teammates like playing with him?

So how, exactly, does one make teammates speak of you with the enthusiasm of Rex Reed reviewing a big studio film likely to quote him in its print ads? "Everybody needs special attention, especially from a point guard," Davis explained. "It's about learning my teammates and their personalities. Being a point guard is about making the guys around you confident to where they feel like they want to play with you because they know that you're looking out for them. I study guys and know where they like the ball and where they are effective. When I played with David Wesley and P.J. Brown, I knew where they liked it and tried to get it to them in those areas."

Great lesson to learn on leadership, I think.

Yes, while I have absolutely no time for gaming these days (especially console gaming),  I still constantly trawl through the 'Net looking for news and info on the next gen. consoles.  I don't know why, as I don't seem to have much time to play console games anymore.

But in any case, today marks the official start of the Next Generation in the console wars.  Microsoft's XBox 360 officially releases today.  In addition, Sony announces that the price of the PS3 will debut at about $300-400, contrary to the earlier rumors that the pricepoint would be closer to $500+.  At $500+, I definitely wouldn't get one.  At $300-400, for a Blu-ray player, I'd probably pick one up.

I've been debating on whether it's worth it to get an XBox 360.  I didn't get the first XBox, even though there were some great games on it.  But I've got a nice, big HD set at home just yearning for the Next Gen. systems.  Oh who am I kidding, I don't have the funds for it in any case :P.

Meanwhile, I've also been reading about the poor manufacturing quality control for the first batch of XBox 360's.  Apparently, there have already been some "casualties" with the XBox 360's in circulation.  From what I've read, it's primarily a heat issue.  This makes me worry somewhat as the Nintendo Revolution's CPU is also being produced by IBM and the Rev. will be in a much smaller form factor.  If it's using a similar architecture, then it could be an issue.

Speaking of the Revolution, news on that front has been pretty mum.  My wife was playing Monkey Ball on the GC last night (I dunno what would compel her to suddenly want to play it, but she hooked up the GC and sat there basically all night playing it).  I sat there watching her twisting and turning her whole body (to no avail) as she guided here little monkey around the course.  Man, Rev. is going to rock sooooooooooo hard.

Wow, I had a lot more fun in Philly than I thought I would have.

Back up a bit.  I had originally registered for the Microsoft launch event in Philly as a participant a few weeks back.  Last week, Rich, my new manager, got 4 of us together, the only 4 that had really played around with SQL Server 2005, and asked if we could come up with a demo for the show as, apparently, we had a presence at the show. 

Long story short, I ended up taking up part of the responsibility of building the demo and also working the booth at the show.  Somehow, we ended up building a database mirroring demo which featured an ASP.Net 2.0 and Atlas frontend.

I also ended up with the responsibility of creating some posters and datasheets that we could hand out along with our demo.  Keep in mind that all of this came up last Thursday.  Having technical and graphical skills is both a curse and a blessing I guess.

Well, in any case, the show was a huge success.  We had people lining up to look at the demo and to talk to Igor and myself.  For an app that was put together in 10-12 hours, it was surprisingly stable.  We only had one major issue and that was a networking issue (dropped IP addresses).  It was so good that we had people coming back to the booth after seeing the datasheet from other people.  We had people coming back with their friends because they were so impressed.  It was quite amazing.  We ran out of the datasheets for the demo in the first 90 minutes of the show (maybe the first 60).  Fortunately, we also had it in poster form (which we didn't give away).  It drew quite a bit of attention and there were a lot of people who just stopped in their tracks to check it out.  The funny story regarding that is I almost didn't have the poster printed.  I was originally only going to print 8.5x11 copies, but Frank, another consultant, suggested that I go with the poster.

Believe it or not, I put it together in ~3-4 hours Tuesday night after I got back from a client site.  Everything on this sheet was created from scratch (Except for the computer graphic, which came from Visio).  I swear I was half asleep when I put it all together

I actually heard a lot of comments on the poster, even one from a professor who said she really loved the layout and colors.  Even my wife was impressed that I put this together when I did/in the amount of time that I did it in.

So all in all, a fun but tiring day.  Man, I just love building cool stuff and I love connecting with other developers.  Hopefully, I'll have more opportunities like this in the future.

I'll be in Philly next week for the Microsoft release event for SQL Server 2005, Visual Studio.Net 2005, and BizTalk 2006 next week (11/17).

I'm not only going as a participant, but a small group of us will be there representing EMC, one of Microsoft's partners for the event.  Rich Millman, my current MC, will be on some speaking panel and a couple of us will be manning a booth on the floor.

As there were only 4 of us in the group at INS Piscataway that have played around with and read up on SQL Server 2005, I was invited into a brainstorming session on what type of demo that we could put together that would draw people in and hopefully get some new contacts.

My first thought was to create a failover cluster using 4 spare PCs that we had.  Not that it's a great demo of the new features of SQL Server 2005, but I figured that it would draw people's attention and since most developers probably very rarely interact with failover clusters.  It would have been cool to let people walk up and plug/unplug indivitual nodes and watch it failover automatically.  Alas, we didn't have any spare hardware sitting around to build the disk array and it was probably too late to borrow anything from EMC.

My second idea was a little better.  Even though database mirroring isn't officially supported in this release of SQL Server 2005, I figured it would be cool to demonstrate it as it's much more likely to be used than failover clustering due to the low cost of implementation.  And so, myself and Igor went about building a demo setup for database mirroring.

One of the first challenges we had to overcome was to figure out why it's not officially supported in this release.  Obviously, it would have sucked to spend hours working on the architecture and UI only to realize that mirroring was buggy and unstable.  After some research, it turns out that the primary reason for not supporting it in this release is because of the fact that Microsoft couldn't find enough beta testers to fully test the new feature.  With that in mind, we decided we could probably pull it off and Igor and I started to dig in.

It's actually fairly cool and takes advantage of a lot of the new features of the 2005 suite.  The UI is an ASP.Net application that utilizes Atlas to retrieve data from a web service.  Another web service was written to interact with the host machine services to stop and start the individual SQL Server instances.  Very cool.  I'm hoping I can talk them into letting me post the code and walkthrough here for anyone that wants to try to set up mirroing.

So if you're going to be in Philly at the launch event, look for INS there!  The demo is very cool.

Yesterday, news spread that the Kansas school board passed legislation to include intelligent design into the school curriculum.

As expected, there was some colorful discussion on Fark.

In an opinion piece in Time this week (11/14/2005), Eric Cornell, the Nobel Prize winner for Physics in 2001, brings up some good points with regards to an unrelated case in Dover, PA.  He states that:

"The central idea of intelligent design is that nature is the way it is because God wants it to be that way.  This is not an assertion that can be tested in a scientific way, but studied in the right context, it is an interesting notion.  As a theological idea, intelligent design is exciting."

That's the key right there.  At the core of all sciences is a process of exhaustive, systematic testing to draw a conclusion based on what can be observed.  No such systematic testing can be applied to the notion of intelligent design.  How can you test the idea that, because some beings are so complex, they must be born of some higher order being?  At best, the discussion should be relegated to some philosophy or theological course and have nothing to do with the sciences.

Notice how Cornell uses the term "God".  While ID activists will attempt to convince us that this isn't about Christianity and creationism, it is quite clear, based on the main leaders of this movement, that this is simply a cloak for injecting the creationist agenda into out public school systems.  Otherwise, we may as well teach Flying Spaghetti Monster as well, right?  It's frightening to think that certain parts of the coutnry are really not that far off from the fundamentalists and extremists that we so detest.

I think that the most important point that Cornell makes is that to use "The Will of God" to answer questions that science has no solution for yet, is dangerous to the progress of mankind.  It's dangerous in the sense that all science is driven by the knowledge to understand that which still remains a mystery. 

"The thrill is that our ignorance exceeds our knowledge; the exciting part is what we don't understand yet."

To use "The Will of God" as a blanket statement to answer questions that which we do not know the scientific answers to, is to say:

"Everything outside this box we can only explain only by invoking God's Will."

It creates an artificial constraint on the growth of our collective knowledge.  It hinders a generation of scientists and discovery by drawing a bounds.  In a sense, it's no different than those that claimed (and believed) that the world was flat.  If no man had the audacity to challenge this thought, if we had accepted such "facts", then it would surely be a very, very different world today.

I haven't been following the riots in France with any particular interest, but a blog post by a journalist in France caught my attention.   A little snippet, if I may:

The rebellion is spreading spontaneously -- driven especially by racist police conduct that is the daily lot of these youths. It's incredible the level of police racism -- these young are arrested or controlled by the police, shaken down, pushed around, and have their papers checked simply because they have dark sins, and the police are verbally brutal, calling them 'bougnoules' [a racist insult, something like the American "towel-heads", only worse], 'dirty Arabs' and more. The police bark, 'Lower your eyes! Lower your eyes!' as if they had no right even to look a policeman in the face. It's utterly dehumanizing. No wonder these kids feel so divorced from authority.

Wow.  Certainly, there are better ways to voice your dissatisfaction with goverment policy, but I can't blame the youth if the picture is really as dark as the blog paints it to be.

For me, I think the most distrubing thing is that I don't see how an resolution can be reached easily as the core issues are not ones that can be changed overnight or through words alone.  I simply cannot answer what the proper course of action is on behalf of the goverment as there is no central figure with whom to negotiate terms of peace.  Further martial action will surely only be met by more resistence and increased unrest amoung the youth.

Could we be seeing the early stages of a modern day revolution?

Perhaps what's most frightening is that, for the most part, none of the current activity has been organized on any large scale.  I think everyone would fear the invovlment of an organized Islamic uprising which may draw upon the vast network of European Islamic extremists.  Aye, this is a Jihadist's dream in the making.

I hate to just link to pages, but there's a lot of stuff to catch up on today, so allow me to indulge myself by placing a little reminder here

Dino Esposito has a nice little write-up on authoring custom controls in .Net 2.0. Hop on over and check it out.

Jesse Ezell has a post on why Visual Studio Team System is just craziness (in a bad way).

Lance offers a small take on the ever present challenge of staying relevant and educated.

Wally links to a post listing various AJAX frameworks.

That's it for now; got lots of reading to do tonight.

Just to wrap up my Austin trip, here's a few random, slightly organized thoughts on the city of Austin and my trip:

1. There's lots of good, cheap Tex-Mex food in Austin (but I guess that this is true for most of the southwest).  Damn, I had some of the best Mexican food I've had in my life and it was ridiculously cheap.
2. PT Cruiser...what an abomination of a car.  If you didn't know already, the rear window controls are at the base of the center console.  Double-U, Tee, Eff?  I didn't drive one, personally, but damn, it was ugly, slow, uncomfortable, and the antithesis of ergonomic.  Not that I had it much better; I had a Cavalier.  No wonder the American automakers are tanking so hard...I'm surprised they've lasted this long with crap like this.
3. The capitol building in Austin has some very nice floorwork and craftsmanship in general.  Very nicely architected and designed.  It's kind of cool that there's so much concentrated in such a small area (two universities right around the capitol).
   
4. Austin (and probably Texas in general) is very open compared to NJ (the most densely populated state in the country).  I like it.  No traffic jams, wide roads, well designed traffic patterns...I wouldn't mind living there, to tell the truth.
5. TownePlace Suites is so-so.  For $89/night, it was a decent deal as I had my own kitchen, queen size bed, and a nicely sized living room.  The bed was very comfy, but the showerhead was weak and the "light continentel breakfast" really meant "bagel in a plastic bag".  Homewood Suites, on the other hand, was much better in terms of the food and accomodations (they had a basketball court in the back!).
6. Being from the north, I tend to look at people oddly if they're walking around with a cowboy hat on, which seems to be common practice in the southwest.

All in all, Austin is a very nice place.  Not nearly as congested as most of NJ.

As I had a lot of down time, I made a bit of progress with The Mythical Man Month.  Without writing an entire essay, I'd just like to share some passages that caught my attention.

"Most organizations spend considerable effort in finding and cultivating the management prospects; I know of none that spends equal effort in finding and developing great designers upon whom the technical excellence of the products will ultimately depend."

"My first proposal is that each software organization must determine and procliam that great designers are as important to the success as great managers are, and that they can be expected to be similarly nurtured and rewarded.  Not only salary, but the perquisites of recognition-office size, furnishings, personal technical equipment, travel funds, staff support-must be fully equivalent."

I've never reall understood why so much emphasis was placed on middle management in most organizations.  I do agree that having exceptional managers can help increase organization and productivity dramatically, but the fact of the matter is that there are few exceptional managers.  My coworker, Igor, offered that instead, the emphasis should be placed on teams.  Small, self governing teams organized by technical function (database team, UI team, data objects team, etc).  The benefits of such a system of organization is clear, for no manager overseeing 20-30 employees can truly understand the strengths, weaknesses, and capabilities of each of the employees.  But such quandaries are easily sorted out in a small team where the team must bear the burden of the responsibilities and thus the team becomes accountable for understanding the function and capacity of each of its members.

One point, in particular, that I'd like to share is one of Brooks' bullet points on how to grow great designers:

"Systematically identify top designers as early as possible.  The best are often not the most experienced. [Emphasis mine]."

Too often in the software industry, the emphasis is placed on years of experience, and not on the actual merits and capabilities of the individual.  As the art and science of software engineering continues to expand and evolve, the best designers will expand their minds and evolve their techniques in parallel.  Software engineering is not a field in which the information and knowledge base remain static.  Certainly, there are core principles that never seem to change, but there are also many different new perspectives, practices, and patterns that emerge with each iteration of tools and environments.  I guess my point is that organizations must find ways to identify and then consequently nuture potential and not merely take the easy road by measuring years of experience.

There were also parts of the text that relate to a project I'm currently working on where the client continually makes change and feature requests, which, by themselves, are not necessarily bad.  But on some levels, the change affects the fundamental design principles of the application, which is time consuming and prone to introduce bugs without massive re-architecting and regression testing.  On this, Brooks says:

"The hardest single part of building a software system is deciding precisely what to build.  No other part of the conceptual work is so difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems.  No other part of the work so cripples the resulting system if done wrong.  No other part is more difficult to rectify later."

"Therefore, the most important function that software builders do for their clients is the iterative extraction and refinement of the product requirements.  For the truth is, the clients do not know what they want.  They usually do not know what questions must be answered, and they almost never have thought of the problem in the detail that must be specified.  Even the simple answer-'Make the new software system work like our old manual information processing system'-is in fact too simple.  Clients never want exactly that."

How true this is.  That last point is particularly interesting.  It's something that I've never understood; shouldn't the idea be to make it better than what you already have?

As I was driving home, I was thinking about how, I would approach a software design project knowing what I know now.  Sitting in on a hardware infrastrucure design session, it's clear how different the art of designing hardware solutions is compared to software solutions.  This is not to belittle the work done by the EMC guys, not at all.  But the fact of the matter is that the constraints, requirements, and features are so well defined by the cost and physical limitations of hardware that architecting a hardware infrastructure is a far less torturous exercise than designing a software system architecture.

I've been asked before, during interviews, how I approach system design.  I think that the right answer is that there is no answer.  Anyone that dares give "an answer" is like a voter that votes strictly Republican or Democrat, regardless of the particular issues at hand; it's a foolish and dangerous approach to think that there is one method or methodology that can be applied to every system.

The secret, as I've discovered on one of my recent projects that I deem to be my finest work to date, is that software must grow in an almost organic fashion instead of being built.

Of this, Brooks' says:

"Must of the present-day software acquisition procedures rest upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it.  I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy."

"Let us turn to nature and study complexity in living things, instead of just the dead works of man.  Here we find constructs whose complexities thrill us with awe.  The brain alone is intricate beyond mapping, powerful beyond imagination, rich in diversity, self-protecting, and self-renewing.  The secret is that it is grown, not built."

"I find that teams can grow much more complex entities in four months than they can build."

And so it was on my last project; the current (and I shan't call it final) design grew, piece by piece, line by line, class by class from a conceptual vision that I had in my head.  At every step, I considered where I could refactor the code, where I could make generalizations, where I could consolidate code, and where I could improve the interfaces and classes.  Where I spotted room for improvment, I did so.  With no interference, the very design itself grew as I learned more about the system and the dependent systems.

I can see that many of the practices and ideas behind agile and XP are not entirely new developments, but rather a natural evolution of the ideas that have been present in software engineering for decades.  Perhaps what's bothersome to me is that, even with this much time, the practice of engineering software is still very immature and it seems as if companies have not learned from the incidents and experience of the past.  We still see an over-emphasis, in many organizations, on management and not enough on discovering, developing, and rewarding top designers and programmers.

Okay, so maybe I've run on a little bit longer than I expected But I'm done now, I swear!  Enjoy your weekend!

Well, as it turns out, the client here in Austin wasn't really that interested in software related professional services at the moment.

I've spent the last two days working with two awesome consultants, Hillary and Bob, from EMC.  All I can say is that I'm extremely impressed with their amazing professionalism, knowledge, and approachability.  They both had an amazing breadth and depth of knowledge about their practice areas that I was just blown away.  These guys are simply amazing and I wouldn't hesitate to recommend them for architecting any sort of high availability, high reliability infrastructure for database systems.

I think that this type of konwledge comes from two sources:

1. Extensive real world experience in implementing solutions.
2. A company that really puts an emphasis on continuing educaiton for their employees on the different packages, products, and their capabilities.

On the first point, Bob and Hillary are both have an astounding amount of experience working with large network and storage systems.  It was clear by the understanding of the different product, their specs, and how it stacked up against competitor's products.  Not only that, there was an incredible understanding of how the products worked, down to how the very bits and bytes were being moved around on disk.  Amazing.  It's the type of experience and knowledge I have with some of the larger software systems I've built, where I know every in and out. 

On the second point, I think one of the coolest things that they had in their tool chest was a true, searchable knowledge base of their products that detailed exactly what their products do, their limitations, and the specs.  The client asked if it was possible to do this, and, within minutes, Bob was digging into the design documents for a product that did what the client described to make sure that it would work in their environment.  Amazing.  But it was very apparent that EMC kept their employees very well educated and up-to-date.

The client was great.  Very friendly and accomodating and incredibly well versed in a variety of different areas.  Unfortunately, my services weren't really needed, but I was certainly drooling over the prospect of helping them design and implement the system that they had already designed, spec'd out, and partially implemented in test.  I love working with large distributed systems, something we don't really get to do much nowadays.  While a lot of the tools and languages that I use everyday were new to them, like T-SQL and C#, I have no doubt that these guys will certainly do a pretty decent job with the system.

Well, it wasn't a totally wasted trip.  I learned a lot about some of our partners and had a chance to meet some great guys and gals out here; I only wish that my services could have been utilized to a greater degree (I hate the feeling of being paid to do nothing), but there was clearly an unwritten agenda on the client's mind and software related professional services were not on that agenda.  I helped where I could and offered what knowledge was relevant.  But I can't complain too much, I did eat pretty well.  I've got to figure out how to get these pictures off of my cell phone

I originally wrote this post as I was sitting in the back of a 737 to Austin.

In the software industry, there is an interesting Catch-22 situation that many young developers will find themselves in: you generally aren't afforded the opportunity to architect any meaningful system since you lack the experience to do so, but you really can't gain the experience unless you do it yourself.

No number of classes, books, and training material can prepare you for architecting a real world solution; only real world experience and trial and error can. So you may be asking, then where to all of the architects come from? Well, in my view, most architects come from three different paths:

1. Promotion. This situation arises by the natural progression of careers. You typically start out as a low level code monkey and work closely with a lead develpoer and architect. You learn their way of thought by viewing their designs. You learn the software and environments by implmenting their designs. One day, as your career progresses, someone will say: "Hey, Chuck's been here a while, he understands the business model, he understands our environments, and he understands the tools, why don't we let him architect this solution?" This can take considerable time as managers are reluctant to use unproven resources to architect anything of significance.
2. Training. Microsoft has a host of acrhitecture and solutions experts certification programs. Many times, companies, especially the big name ones, will hire based on this qualification alone. To me, this type of architect is the most useless (and dangerous) architect (if not combined with one of the other two). Why? Well, this type of architect typically has very little real world experience digging into the tools and environments necessary to build the solution. As such, any design that arises may make sense on an academic level, but will end up being difficult, impossible, costly, or messy to implement. To me, any sort of sterile classroom training will only take you so far; there is no substitute for experience and knowledge of the tools, environments, and business problem. My ex-coworker, Kent Brown, was really against this type of architecting.
3. Accidental. I think that this happens the most in the consulting world. I've been an accidental architect at most of the places where I've worked, none-more-so than at ITT, where I basically redesigned their existing intranet applications and rebuilt them all using a new, common codebase and paradigm. I literally did everything, from the database design to the UI design to the actual implementation and deployment. In general, this can be a dangerous situation as you never know what you get when you "accidentally" put someone into an architecture position. But I think that ITT lucked out inthat I was much more experienced then they initially thought for a guy coming straight out of college.

So why do I bring this up? Well, I find myself as an accidental architect at the moment as I ride in the back of a 737 to Austin. Without really thinking about it, I responded "Yes" to a request by one of the managing consultants (MC) at my company, INS, to head out to Austin to help with, what was described to me as an Access to SQL Server upsizing project. While I'm primrarily a developer, this is not outside the bounds of my experience and skillset as it was one of my primary responsibilities at ITT. As I would later learn, this is not the real role that they need to fill, but rather one of an architect.

As I accepted, my cross-divider fellow cube-dweller, Dan, chimed in and asked, quite bluntly, "Are you nuts?" For I had accepted a gig in Austin for two weeks without any actual details. To be honest, I really didn't think about it that deeply. I mean, I had accepted other offsite gigs several times, none of which ever panned out (at least two). I was thinking that this would be the case as well. Plus, I figured that if my MC asked me to fill he had this role, then confidence that I could/would succeed at the task and I was assured that there would be more information forthcoming.

Hoooooo boy. I accepted late Friday afternoon and got "more" details, if you consider "more" one network infrastructure diagram with no legend or supporting documentation. Damn, I shoulda seen this coming. To make a long story short, I am currently heading to Austin with little knowledge of what I'm supposed to be doing or what the goal/scope of the assignment is.

Step back a couple of hours. Around 2 PM on Monday, I think it really started to sink in that I could be in a situation where I had no possibility of success (I hate that feeling). I had already booked my flight on Saturday ($1000+!!!) but i still didn't know what exactly I was supposed to be doing. By 4, I was jetting home to pack my bags. I had spent most of the afternoon trying to set up accomodations and discussing what little details of the project were known among the other consultants and thinking to myself: "Brilliant Chuck, juuuuust brilliant." One interesting tidbit is that my billing rate is actually a very fortunate number in Asian cultures, especially those that speak mandarin.

Bad news. As I'm driving home, my "service engine soon" light came on. Doh! I was just hoping to make it home in one piece. At the same time, my wife was enroute from work to help me pack and send me off (thanks hun!). We were cutting it close. We ran around and I prepped 7 days worth of clothing and other random stuff (like my Gameboy). We were packed and out of the house in under 30 minutes! Time: 5:00 PM.

Knowing Jersey, I really should have planned more time into this No sooner had we passed exit 11 on the Turnpike did the traffic come to a standstill. It was literally a giant parking lot as we creeped at < 5 mph. Good gracious, we still had ~5-7 miles to go and baggage check-in. I don't think I've ever hyperventillated, but I was starting to as I was sitting there, not moving. Scanning the XM stations revealed that there was a stalled truck on the Goethals bridge at exit 13...unfortunately, we needed the exit right past that. Fortunately, the reports were that the truck was moved and traffic was slowly movinig again. Yay!

So for now, my thinking is that I'll try to make the most of it. Certainly, there are less qualified individuals that have been placed in my role in the past, right? Plus, at the least, it's a learning experience on working with total strangers If it's what I think it is, then I may have a good chance to come out on top. Wish me luck!