A blog post by Rod Hamilton - Unico Senior Architect. Reposted from his blog site Omitted for  Clarity.

Why is it, that delivering reliable software seems to have become a bit of a ‘black art’? We’ve been doing it for so long, surely we should be able to get it right by now? I was recently asked to give a presentation to an IT department who seemed to have perfected the process of starting projects that would either be canned well before ever making it into production, or if they did go live, they would end up disappointing and under whelming users.

It reminds me of recent experiences my friends and I have had building new homes with one of Australia’s biggest residential developers. In some ways you’d consider them the IBM’s of the building world based on the sheer volume of construction and number of builder’s and sub-contractors they keep in business each year (one difference – they don’t charge like IBM, I couldn’t afford those guys building my house). Their staff were extremely efficient in the sales process. They’d walk us through the display home, highlight the features, recommend variations from their catalogue and help us visualise life in a new home with “alfresco entertaining and expansive spaces”. It would be our very own slice of the good life. Their experience in having built these houses so many times and then their efficiency in throwing it together was evident – in my case the house was delivered 3 months ahead of schedule.

Personally, I was pretty happy with the result, however, there were some minor (and some more major) problems that we and our friends experienced. Once our houses were planned, variations were particularly difficult to make happen. They resulted in schedule delays, cost increases, even when the change meant less work for the builders. Some friends wanted alternative materials used which not only added cost, but as the builders were inexperienced with the materials, they were implemented poorly. Once in the house, poor design of some features lead to increased operating costs. For example, heating in one of the homes didn’t allow for zoning off specific rooms which lead to extreme inefficiency – every room was heated all of the time when only a couple of rooms were to be used.

These sorts problems parallel the software world closely. The business starts with what they believe to be a well defined set of requirements. Developers design and work against these requirements but as requirements change, inflexible delivery approaches lead to scope, cost and schedule blowouts. Poor technology selection which may be inappropriate for the problem being solved, or may not be well understood by the implementer can lead to performance, reliability or maintainability issues down the track. Rigid architectures that don’t consider how the business will actually use the software to meet immediate, and just as importantly, future needs lead to unnecessary expense in extending the solution.

So whether we’re building a house, or building a software solution, here’s 3 principles worth a look:

1. Be adaptable during delivery: The saying goes something like “the only thing that stays the same is that everything changes” and this is no different when talking about business requirements. We’re kidding ourselves if we think the business won’t change their mind or won’t need to introduce features as they learn more about their problem domain and the right solution. Business moves so quickly now that businesses need to change their mind just to keep up. It’s no longer practical to deliver using waterfall methodologies or manage scope with an iron first for projects susceptible to significant change. Agile methodologies such as SCRUM lend themselves well to dealing with this, but we need to be great negotiators and clearly communicate the impacts of changing requirements late in the delivery cycle to stakeholders. Quite often dropping some scope to make way for the ‘must have’ new requirements is achievable when put into context for the customer.
2. Architect the necessary flex points: It’s easy to focus just on the current requirements, but the architecture usually needs to be cost effective when meeting future requirements. The other extreme almost as bad. Often we aim for ‘infinite extensibility’, trying to satisfy every possible future enhancement. This is common in SOA environments where it manifests itself in the form of massive service catalogues and the needless creation of services that have a low likelihood of reuse. The impact of such an approach is twofold - governance of so many individual services slows delivery, and the number of individual hops necessary to complete a transaction slows runtime performance. The focus should be simplicity - design the ‘necessary flex points’, build them in and forget the rest. To quote a colleague of mine “Creating good software is about keeping it simple. If its simple, it gets done quickly and can be maintained more readily”
3. Prove the selected technology: Selecting the right technology needs to consider more than just “will it work”. Does the market have the skills to support it? How will it impact TCO? Is it malleable enough for developers to respond quickly enough to changes in requirements? Use technologies you are familiar with and build a prototype up front to prove the executable architecture. A little investment up front can save a lot of pain later on.