WTB [Roundup] or [Weed-B-Gon] PST!

Pardon the MMO parlance there.  Yowza been busy.  Still busy but trying to pull my head out of the sand for a while.  Just caught up with about a month’s worth of unread blogs.  Many thanks as usual to Jason Haley for recapping me with almost everything I needed to know in that time.  Dude, you moved to Seattle?  WTF?  (been giving that some thought myself lately…neither here nor there).

So I got to thinking about patterns in software development.  Not design patterns, but project patterns.  Patterns I have seen over and over.  I would love to at some point write more in-depth about this current project that is damn near killing me, but can’t yet.  Maybe later this summer.  There’s a lot of lessons to be learned.  Instead, I want to talk in generalities about what I’ve supposedly learned in my time as a professional software developer.

Let’s talk about project stats here.  Scott was talking about foregoing the cert list (I don’t hold any certs, don’t really believe in them) for the project stats.  Here’s mine:

5 major (as in mission-critical, enterprise-level, everyone-is-counting-on-this-platform) projects.  4 failed.  1 in progress.  Prospective employers:  flee for the hills, this guy is a black cat breaking a mirror while walking under a ladder.  There have been a lot of small and even big successes in those 4 failures, but I still count them in the loss column.  Every one of them was left unfinished, only one at my choosing.  Two of them were at companies no-longer-in-existence (and I was there on the last day for each).  1 of them was never fully supported by management and when I was fired (I was a week away from quitting) the project was abandoned.  The other was mired in one of the worst development environments I’ve seen when I left, with 50 different project teams all pulling in different directions, and if it did ever get finished it a)probably involved an Act of God, and 2) is a good bet that it’s not in use.

That’s not to say I’ve never done anything good in my career.  I’ve had a lot of small victories.  I’ve had some medium victories.  I’ve even had some solid victories amidst those 4 major failings.  But the bottom line is that I have not FINISHED (that magical state) a major project and been able to sit back for a few weeks and admire my work before moving on to v2.  One of the big differences between IT development and commercial development is that you don’t “ship”.  It’s hard to find that finish day.  That V-P (victory over project) day.  I’m working on reaching that day right now.  I’m hoping it’s in July sometime.  I’d like to go 1–4 and start working my way out of the cellar toward .500 and maybe one day to a winning record.  But in another sense, I feel like if I get to V-P day on this project that I almost wipe the slate clean on record and go up 1–0.  Ever have that feeling?  Like just one win would bring you back from the brink?

Anyway, let’s talk about some project patterns, or rules even, that I’ve picked up in these failures, and that keep resurfacing if you drop your guard against them even for a minute.

Not controlling *any* of the top-line factors that determine delivery date:

This is one of the worst, and it has happened over and over again.  You cannot let someone “at the top” tell you both the level of functionality and the release date.  It doesn’t work that way.  You try to make it work that way, but it doesn’t.  I’ve “learned” this lesson time and again.  Here’s the thing – sometimes it happens without you even knowing.  It happens because of a very sinister form of scope-creep.  You specify the software, get it all planned out, and then figure for a timeline.  Everyone likes it.  You have buy-in of the major department heads and other “stake holders”.  You start to cruise.  Time goes by, and new information comes to light.  Some manager didn’t accurately describe the process of his people, and didn’t pay enough attention even though you had 30 meetings about it during the spec process and now, 8 months in, major changes.  Some of them architectural.  They’re gonna cost.  Time, money, whatever.  Upper management understands this in theory, and timelines are adjusted accordingly, but they are only adjusted on paper, that day.  As the original projected completion date draws near, management seems to have erased the file in their collective brain that had the new date.  They only remember the old one.  Now you’re late.  This happens a few times over the course of the project, and you are dead in the water.

How to fix it:  I obviously don’t know.  New people is rarely an option.  Even if the budget allows once you are at some point in the timeline, the schwartzshild radius of the due date, you reach a place where it’s harder to bring someone new on than it is to just work extra.  Incremental development *may* do it, but that’s not always a reality.  (as a side note, I’d love to have a dinner conversation with someone who is truly Agile or whatever and learn how you apply such a beast to the type of systems development I do).  There are ways to mitigate the damage by creating frameworks and architecting your applications so that they are fairly change-tolerant, but that only goes so far.  Granted, the more times you experience this, the better you get at anticipating it and handling it, but it still sucks.

Not dealing with personnel issues when they come up:

This is another biggie, and one I have struggled with time and again.  Most recently this week.  You get underperformers on a project, people that either won’t or can’t hack it, and you let them sit.  These people are bugs in the project.  You have to deal with them early, just like bugs in code, before the cost to fix becomes greater than the cost to leave it alone.  For whatever reason, you may have people on the team that just don’t perform.  You try to lead them, you try to mentor them, you try to motivate them, and nothing seems to work.  Or they will act motivated enough to be left alone for a few weeks and start the process over.  Sometimes they don’t get it, sometimes they don’t care.  While it would be nice to weed these people out in the interview process, sometimes that’s not the case.  Sometimes you go to work and they are already there.  You inherit them.  Sometimes you hire them because they seemed like they could do the job with some prodding and you’re in a small market so you figure you will make do (that is a huge mistake, I’ve done it, I don’t plan to do it again).  Sometimes they interview well, have good resumes, good references, and turn out to be complete psychopathic fucktards.  Either way you have to deal with the real problems, and if necessary, you have to make the tough decisions.  I’m not saying you have to be Chainsaw Al Dunlap, but if you need to pull the trigger and cut someone loose, you need to do it.  Because at some point you end up letting the problem go on long enough that you can convince yourself that a)what little contribution this person makes is better than no contribution at all and it’s too late in the project to hire a replacement, or 2) that you can cover long enough and do something when it’s not such a critical time so that you don’t look like a bad manager.  That’s a pretty big fear in and of itself.  If you hired someone it’s damn hard to fire them because you feel like a fuckup.  Believe me, I know.  But you have to deal with personnel issues because they will kill a project, kill team morale, make sub-standard contributions, and otherwise make your life a living hell.

How to fix it:  Grow some balls.  Try to mentor, try to coach, try to motivate.  When you get to the point where you feel like you have honestly done all that you can, you have to pull the trigger.  It will only get worse.

Not knowing what you are developing:

Yikes.  Sounds ridiculous?  I bet I’m not the only one out there.  You are given some ethereal vision, something great usually, and told to run with it.  Maybe it’s the next great platform that your company will run its business on.  Maybe it’s some super sweet software idea that will sell millions.  There’s no spec.  There’s no solid idea of what it has to do, that’s all it is is an idea.  On a small project, or something that fills a niche role here or there, this is fine.  You can build it fast and in increments.  But something big?  Something…”mission critical”?  Negative.  Don’t go there.  It’s a death trap.  You people who don’t like BDUF…that’s fine.  But you better get a fucking spec that’s all I’m saying.  You don’t write a line of code until you understand what you are writing at least.  Maybe not every little detail down to the button clicks, but you gotta know what you are writing.  Because if you don’t know what the product is, the project never ends.  This happened to me on major failure #1.  We had no idea what we were writing.  What started as an idea to serve a couple of customers and took one weekend to develop as a small ASP application turned into a 2 year never-ending feature orgy that completely transformed everything we did on a daily basis.  The parts of it that got finished did well, and were in fact critical to our mission, but when our mission failed, the project died the death of an unfinished loosely defined hodgepodge of spaghetti code and feature-sets held together with breadsticks and shellack.  This is not a fun thing to be a part of.  Even worse, sometimes this one crops up in the middle of a project that you thought you had pretty well defined.  See above.

How to fix it:  This is a tough one.  You have to manage expectations in a very micro way here.  You have to lock down a specific set of features and call it a release.  You have to push those new epiphanic feature requests to the table for the next version.  This is the only way to maintain sanity and even hope to turn out something decent.  If you can’t exert this level of control over the process, you gotta bail out.  I’m totally cereal.

Not knowing what and when to cut:

This can be a tricky one, but it’s an absolute necessity that you have this in your tool-belt.  You have to know what features you can cut, when you will need to cut them, and in what order.  This involves playing a game of what level of functionality can be deployed in V1 that makes the users and management so happy that they can wait for the next release to get what you cut.  You have to know the business and your audience very well, and you have to know the software you are building very well.  What can it live without for a month, a quarter, a year after initial release.  These things you keep in your bag when you start running up against the deadline, and if you are good, you cut them silently and nobody notices.  Why not postpone them up front?  Couple of reasons.  The first is that you want to provide as much as you can.  You want to deliver the best and most featureful software ever created.  That’s a good goal.  Unrealistic but good.  The second is that if you cut them up front, publicly, you will be expected to adjust the timeline down.  That’s bad.  If you do that, then in 6 months you are going to find yourself in the same position for whatever reason and you won’t have anything to cut to make up time.  You have to keep the bullet in the chamber and pull the trigger when the time is right.  The two big mistakes are not ever pulling that trigger, and not having the bullet in the first place.  It sounds cheap, and maybe to some of you it sounds unprofessional, but the reality is you have to have this ready to go to survive any large project.

How to fix it:  This is all about experience.  It is one of those things you have to learn as you go.  So pay attention.

Anyway, these are just a few of those things that pop up over and again in large projects that fail.  You have to learn from your mistakes and get better every time, but even then it will happen to you if you aren’t vigilant, and you have to put your experience to work to fix it, or know when something is unfixable (it happens) and that it’s time to bounce.  As for me…I’m working on batting .200