Developing software for corporates- the necessary obfuscation layer

[darrenb]

If you put your car in at the garage you want an accurate estimate of how long things will take, what is wrong and what needs to be fixed so you can try and get a handle on whether they are trying to overcharge and are competent and you want it to work at the end.

Software development is a completely different ballgame to auto repair. A big part of the problem is that people try to squeeze technological innovation into an old industrial economic model through bogus comparisons with blue collar work (and this is surely how offshoring started).

The more complex and high budget things become they less you can just trust someone and the more it needs to be tracked.

I'd like to modify that statement in a couple of ways.

Firstly, the more complex and innovative things become the less you can track it. Of course, you can put in lots of pretend tracking in there, but all it's going to do is interfere with the actual process, and I think the OP made that point very well. I would add that a key goal of tracking is to find and use generic metrics, but what is actually important within an innovative process is completely dependent on context, so this is just the Don Quixote of Managerialism flailing at windmills again.

Secondly, the more complex things become, the more you have to rely on trust. That can be tough to establish -- impossible, even, if you yourself are not a trustworthy person. But ultimately the only way to get something as complex as software done is to spread around good will and hope it sticks. If it turns out that you can't trust your people, your best course is simply to admit failure at the start and save the shareholders/investors/taxpayers from wasting millions. Now I know no exec would ever do that, but just imagine how helpful it would be if they did and cleared the way for projects that actually have a snowball's chance in hell of succeeding.

If you put a fleet of a thousand limousines in to a garage for some upgrade and every day one is out of business costs you a grand you're going to want see planning and progress.

If a heart surgeon is performing on you, you're going to want to see progress too. But you can't. Why not? Sorry, you're simply not qualified, and you wouldn't understand what is going on even if you could observe and weren't blissfully unconscious (of all the little mistakes and corrections going on).

So at some point you just have to trust the professionals. If you've managed to successfully de-professionalize IT, then once again, sorry, you're out of luck.

[minestrone]

Software development is a completely different ballgame to auto repair. A big part of the problem is that people try to squeeze technological innovation into an old industrial economic model through bogus comparisons with blue collar work (and this is surely how offshoring started).



I'd like to modify that statement in a couple of ways.

Firstly, the more complex and innovative things become the less you can track it. Of course, you can put in lots of pretend tracking in there, but all it's going to do is interfere with the actual process, and I think the OP made that point very well. I would add that a key goal of tracking is to find and use generic metrics, but what is actually important within an innovative process is completely dependent on context, so this is just the Don Quixote of Managerialism flailing at windmills again.

Secondly, the more complex things become, the more you have to rely on trust. That can be tough to establish -- impossible, even, if you yourself are not a trustworthy person. But ultimately the only way to get something as complex as software done is to spread around good will and hope it sticks. If it turns out that you can't trust your people, your best course is simply to admit failure at the start and save the shareholders/investors/taxpayers from wasting millions. Now I know no exec would ever do that, but just imagine how helpful it would be if they did and cleared the way for projects that actually have a snowball's chance in hell of succeeding.



If a heart surgeon is performing on you, you're going to want to see progress too. But you can't. Why not? Sorry, you're simply not qualified, and you wouldn't understand what is going on even if you could observe and weren't blissfully unconscious (of all the little mistakes and corrections going on).

So at some point you just have to trust the professionals. If you've managed to successfully de-professionalize IT, then once again, sorry, you're out of luck.

I have to say, that is an outstanding post.

Just a shame it is a complete pile of absolute tripe.

[aussielong]

I have some strong opinions on this one. Every company is different. Processes differ, but in all cases there is a process. It is undeniable. I fully understand creative types don't like doctrine and I have worked in a lot of tulipe places where managers pomp about arranging pointless meetings and get all shouty when the don't like what they hear. I've also seen the other side of the coin where hands off management is killing a project also, I believe there was a thread on here recently from NWPTC about how soul destroying that is.

Without process there is chaos. The level of process a team needs for optimum performance is subjective, and is generally based on the quality, or indeed competency of said team. Extreme examples include local government, who generally employ low grade staff and layer on the red tape. Or the trade union mentality. Or off-shore teams needing micromanaging. Thick with process, inefficient.

For those who understand these things, it's like we need some power factor correction between the stuff doers and the management. With unity power factor being the holy grail. For those that don't understand PFC, it's essentially a way of getting the most efficient work out of an electrical device. If you have a factory with lots of electrical motors then due to the nature of the motors, they actually impede the flow of electricity because they have lots of windings, and this causes a lag. This lag causes overheating and this heat that is lost is energy that could have been spent doing work. [Very oversimplified explanation of PFC].

Or a bit like when I was in swimming classes, you get your stroke wrong and you make lots of foam but don't get much movement. Get it right and you are putting in lots of laps.

So for me getting the process right is about how much process as well as what the process is.

What I didn't want to see in this thread was the petulant rantings of the self obsessed "creative types" carrying on as if those who hired them have no right to formulate how they keep tabs on where there money goes.

Don't disagree with you about needing some process.

What I'm saying is the real process of developing does not match the management requirements of a process.

I have multiple aspects of the program on the go at once, I build end to end and keep iterating until I'm done, at certain points I stand back and check I've not gone mad-refactor

I'm going over requirements all the time to make sure nothng drops down the cracks

The "design" is complete when I'm finished building

At the end, I apply a few design patterns where necessary... This is like tidying up my hand writing (in the real world)

Always thinking about testing and test ability

This is my process and it works for me

[cojak]

I used to be like Suity. I'm not anymore.

I am a BA, I no longer believe in process. I say Adaptive Case Management is the future. Many otherwise competent professionals (a bit of a stretch i know but work with me on this) say they hate process. There's something in there if you care to look.

There's nothing wrong with process for routine, repetitive tasks.

But if you ask people doing emergent, unpredictable, non-repeatable work if they follow process, 80% of the time they'll say 'no'. They are knowledge workers and current process analysis and management doesn't really work for them. This is where ACM comes into play.

I'm not going into ACM here (because it's a holiday and I can't be arsed), but essentially it ensures that those closest to the 'process' can adapt it.

[doodab]

The key to successful collaboration is to understand that the structure a PM will try to impose is driven by their needs. The more enlightened ones will show an interest in helping you do your job as well but it's by no means a given.

It used to be the case that I thought I didn't have a process, but the more I thought about it the more I realised I did, it just wasn't the process that the MS project jockeys had written down. Now I'm older and wiser I generally try and work with the PM and either restructure the plan to reflect reality a little better or at the very least de risk it by bringing obviously high risk unknowns forwards (for some reason human nature is to tack these on the end after all of the stuff we already know how to do) and in the worst case just bulltulip them to ensure that there are as many days as I think I need while accommodating various uncertainties.

This usually goes along the lines of putting a couple of client demos in the development phase which pleases them no end as I usually commit to demoing certain functionality at those points and it gives them a clear opportunity to track progress. That commitment will buy you some trust and flexibility elsewhere.

[BlasterBates]

If you don't break your project down into small doable tasks you are going to end up with an amorphous large piece of software that doesn't work.

For example get the bit working that reads from the database, then get the bit working that aggregates the data. Next get the bit working that sends the data to the client etc etc.

You ought to be able to plan that.

1) read from the database - 2 weeks
2) aggregate data - 1 week

Also a good idea to do an overall design and identify the components you're going to develop, so you can map the components to your implementation plan. I always do a detailed design before I start any siginificant development, yup it might change somewhat but it should be pretty firm. Then you need milestones.

I mean if you 're going to read from the database you know you need a class that connects to the database and some Data reader don't you. Then for aggregation you know you need something that sits on top and puts stuff in vectors adding everything up etc etc.

One problem is that requirements aren't understood first, sometimes too vague, maybe that's the problem.

If there are questions on the requirements, good idea to write an acceptance test specification, that is a great way of clarifying everything. Describe all the test cases and then there is no question that what you've done is as they "want it".

[d000hg]

PMs by definition don't work very hard and don't understand what the people they take credit for are actually doing

No, not by definition. Bad PMs may be like that, but bad developers are no better.

one of the keys is when the project manager realises he is not the boss, but the enabler, the oil between the wheels, the servant.

Hit the proverbial nail there EO. Of course this requires the PM to trust his developers are good at their jobs, if he has resources foisted upon him this all goes wrong.

[BlasterBates]

I used to be like Suity. I'm not anymore.

I am a BA, I no longer believe in process. I say Adaptive Case Management is the future. Many otherwise competent professionals (a bit of a stretch i know but work with me on this) say they hate process. There's something in there if you care to look.

There's nothing wrong with process for routine, repetitive tasks.

But if you ask people doing emergent, unpredictable, non-repeatable work if they follow process, 80% of the time they'll say 'no'. They are knowledge workers and current process analysis and management doesn't really work for them. This is where ACM comes into play.

I'm not going into ACM here (because it's a holiday and I can't be arsed), but essentially it ensures that those closest to the 'process' can adapt it.

aha this reminds me of the notion of "contingency", if I maybe academically pompous for a minute or two.

In Organisational theory there are different ways of management ranging from a "power based" culture to a bureacratic. If it is unstructured then you need a manager who is akin to God, basically behaving in a slightly bullying way to get things done but no procedures, otherwise you have unstructured chaos, this is what you need in a small start up company...f*** the procedures roll your sleeves and get it done, but with someone in charge who knows where he's going. Then there's move towards large orgs like banks where you need procedures design and all those boring things that get on developer's nerves but if you don't do it you could end up like Barings or Lehmans .....on the rocks.

[cojak]

Well the 'why' and the 'what' are important (goals, legislative rules etc).

But the 'how'... leave it to the specialists to work that out. They're usually better at it than non-specialists.