The Difficulty of Managing Software

The Need For a Better Way

It's a real problem. It happens in governments, non-profits, and in the powerful enterprises that control our private industries. Every day, authority figures are forced to make decisions that directly impact software development. These authorities include leaders, managers, and business people, many of whom are clearly not qualified to understand how computer software works.

To make things worse, many workers rush in purporting to understand software, yet have, at best, mediocre competence in it. How can non-developers, or mediocre or poor developers, successfully make good decisions that impact development?

Many workers with a mediocre or poor understanding of software even end up holding positions directly responsible for software development. An admitted non-developer may look to a purported developer to help inform their decision. This includes decisions about what worker should be in what role. But how can the non-developer tell which of those advising "developers" are really qualified, if the non-developers themselves don't understand the underlying subject in the first place?

The consequence is that many organizations make bad decisions about software. And that, is an understatement. We are talking about decisions which have the potential to impact millions of dollars, loss of human life, and threats to national security. The risks are difficult to exaggerate.

Previous Solutions

Consultants and Freelancers

With so much at stake, it is quite a surprise that many have done so little to help. Yet there has been some help. Specialized consulting firms employ various levels of vetting to recruit developers and contract them out, as individuals or teams, to organizations. Freelancing portals, often including ratings systems, facilitate connections between independent developers of varying levels of competence and clients willing to use these portals to obtain development work. Though consulting firms and freelancing portals may be helpful, they do not entirely resolve how the development is done.

Agile

The agile frameworks, under the Agile Manifesto and principles, include a few ceremonies and artifacts attempting to resolve these issues. Unfortunately, the agile frameworks are highly controversial. Possibly because it does not solve the root problem, many organizations have implemented agile in such a way that some feel it makes things worse.

Agile has gotten a bad reputation because many organizations are implementing a fake version of agile, and that fake agile is making things worse. The organizations are adopting certain external elements such as terminology and activities from agile, without an underlying value transformation. Therefore, they are simply adding useless activities and applying misleading terminology, while continuing to do things in a way that is opposite to the true principles of agile. It is no wonder many have left these environments with a bad taste in their mouth for agile.

Startups

It seems the best current possible solution to have an opportunity to have a software development process of reasonable quality, is to leave traditional organizations and start a startup company. Many have done just that, and some startups have achieved astronomical success (often, it seems, directly related to great software). In such a startup there is less of a problem because the decision makers and implementers are one and the same.

Yet honestly, software startups are not a very good option. Such startups also have an astronomical failure rate. It is very hard to run a successful business, and, especially to start a brand new one from nothing. Successful startups require not only great software development, but also market research, sales, accounting, legal support, and much more.

Once again, many companies have arisen to help startup companies with their challenges. But, the root problem remains. Once the rare startup becomes highly successful, it soon starts looking more like the traditional organization, including manifesting the same software development problems.

How can we resolve the root problem? Please comment below.