Author Archives: admin

Technical debt: like financial debt, yet not quite

Debts arise when you spend more than you can afford. Furthermore, debts grow by itself: they come with an interest rate. Debts do not have to be a problem, as long as they are temporary, and as long as you have the means to pay them back. Because this is the number 1 characteristic of debts: they need to be paid back, one day or another. If you can’t pay them back at the moment when you have to, this leads to serious consequences: you might lose your house, or you might get blacklisted by banks.

Most debts are financial: you owe money to a bank for your mortgage, or for the new car you bought last year. These debts can be expressed in clear financial numbers, which makes them easy to identify, analyze and manage.

In the world of software development, debts exist that cannot be easily expressed in numbers. It’s called technical debt.

In this article I will explain what it is, compare it to financial debt and describe some guidelines on how to deal with it.

What Is Technical Debt?

Technical debts are introduced by sub-optimal technical decisions. That’s rather abstract, so let me give an example. Let’s say your current software application works with the euro currency only. But business is expanding to areas outside of the euro zone, and now an important potential customer requires your system to also support the American dollar. What do you do? Basically, you have two options.
The first option you have is: you implement the requirement in such way that your system simply supports both euro and dollar. Your software programmers explain that it is relatively easy, although it requires a lot of code that they call ugly: a lot of ‘if currency = EUR then …’ constructions, for instance, something which makes testing and debugging more difficult.
The other option is: create the possibility to use any kind of currency in the system. Your software programmers explain that this is harder to implement, it will take more time, but the resulting system will be cleaner and easier to maintain.

The example is not 100% technical, but I hope you get the idea: there are situations where you can choose quick wins, or go for more solid (yet more expensive) technical solutions.

Short Term Versus Long Term Profit

And so we arrive at the crucial question: do you choose a short term profit (please your new customer as fast as you can), or do you choose to invest in the future (create a solid solution that will support a broad range of future requirements, including the urgent one of your new customer)?
Choosing the short term route generates fast and measurable profits (revenue because of a new customer). However, chances are you create technical debt here, debt that is not visible immediately.
Choosing the long term, you may have to tell the new customer that he has to wait longer before your system can comply with his requirements. You will have to invest more than in the short term solution, which will affect immediate profit negatively. On the other hand, your system will become more future proof, and future new customers with yet another new currency can be pulled onboard without any effort.

The Interest Rate: High Maintenance And Low Production

And this is a very important reason why technical debts arise: technical decisions that support short term profits, instead of long term profits. The more this happens, the larger will become the amount of technical debt. But remember: it comes with an interest rate! So the bigger the technical debt becomes, the more interest you pay. Or expressed in software development terms: you will spend more and more time on basic maintenance tasks (bug fixes, implement small features), and less and less time on what you would need to do (develop your system according to the new requirements of your markets: i.e. create value for your customers).

Guidelines For Dealing With Technical Debt

It’s OK to have some debts (probably inevitable in software development), it would be even OK to have a considerable amount of debt.
However, as a technical manager you cannot just let it grow, that would be very irresponsible, not to mention bad management practice. Maintenance costs could literally kill your company. So what to do?

As technical debt is not very visible, make it visible: identify the moments when technical debts arise, and document it. Ways to do this: talk to your technical specialists, actively discuss the consequences of certain technical solutions for your technical debt. If technical debt is already there, have your system be reviewed by independent software quality experts.
Subsequently create a plan for payback. Make an estimate (i.e. risk analysis) of the interest rate (how does the technical debt affect your productivity), and decide when and how you are going to pay back. Remember that the exact moment that you urgently need to payback (for instance when the technical debt would block the development of a new required feature) is quite unpredictable. In general it is good practice to pay back your debts before you are urged to pack back.

When Technical Debt Has Become Very Big

When I’m contracted by a company, usually the technical debt is already way too high. It’s sometimes so high that investments of millions of euros would be necessary to pay back all the debt. The problem is: money itself is not making the debt disappear. Money needs to finance the right activities that will pay back the debts. When the debt is very big, it has also become very hard to identify which activities are needed. When the house is almost collapsing, should you try to renovate it or should you just build a new house?
In one of my next blogs I will address this situation.

Differences With Financial Debts

As you may have noticed by now, technical debts share some characteristics with financial debts. To conclude this article I will summarize the most important differences previously mentioned:

Technical debt is not very visible and can not be easily put in numbers. This makes it harder to manage, especially by managers that do not completely understand the concept of technical debt. This is one of the reasons why software departments are very hard to manage by managers without a firm background in software development.

It’s hard to predict when it’s payback time. For your mortgage, your bank will tell you exactly when and how much to pay. In the case of technical debt, most of the times you don’t know when it’s payback time. It might take years before you need to pay something back, it could also be tomorrow.

Resolving technical debt requires investments. Financial debts you resolve with plain money. But to resolve technical debts, you need to invest! Invest in the right people, right activities. And, like all investments, they come at their own risk. Another reason why it’s so hard to get rid of technical debt: activities that are supposed to get rid of it might even increase the technical debt!

More IT expertise in the boardroom, please!

In my work as a freelance IT consultant I work for companies or institutions that are not IT companies themselves. Mostly they are consultancy firms or research institutes. Knowledge intensive organizations, I call them myself.

Digitization process in organizations

So did I work for a consultancy firm in renewable energy. Their business is acquiring knowledge regarding solar energy, wind energy, energy savings, energy systems and use this knowledge to help their clients implementing energy saving measures or renewable energy systems.
That doesn’t sound like a company with many IT activities. But energy is quantifiable, meaning that it can be measured and calculated. And that’s where IT enters the room: data needs to be stored, calculations need to be programmed. Employees (energy consultants) started doing that by themselves in Excel. After a while the Excel sheets turned too big, too complicated, and where possible, the shift to more professional databases was made. A database administrator was needed, and a programmer was hired to code calculation applications.

IT activities and IT jobs grow fast

What is going on here exactly? A whole range of job functions (information technology jobs) shows up, sometimes within a few years, that has no direct relation with the main expertise of the company (energy in the example).
One might say, every company has job functions that are not directly related to the main expertise of the company: a secretary, a cleaning person, a driver, etc.. What’s the big deal? And that’s true, every company has a variety of job functions, and the bigger the company, the bigger the variety in functions.

The board room can’t keep up

But the point I’d like to make in this blog post: the IT activities as well as the number of IT job functions are increasing rapidly in many companies, due to the increasing digital demands of the market or the company itself. The energy company I mentioned was just one example, but the same phenomenon is happening in many other kinds of organizations.
And what I notice in ALL those organizations: the board room struggles managing the IT activities and the IT employees. The board members have no affinity, education or experience with IT, and it shows.

“Board rooms struggle managing IT activities and IT employees”

Where board rooms fall short

Many board rooms lack IT knowledge and experience. That’s the point of this article. But how does this show exactly? Some examples (and there are many more, unfortunately):

The wrong IT people are hired: lacking skills or experience. Management has no clue how to recognize a good programmer, a good network engineer.
IT activities are outsourced for the wrong reasons, or vice versa: kept within the company while outsourcing would make more sense.
Lack of communication with IT department or employees: IT is treated as a technical facility, not as an important business partner. They pass the fact that IT is very much related to business processes.
Management by incident: only when problems reach the surface (a server breaks down, a system is hacked and data is stolen), the management acts. A long term IT policy is absent.
Underestimation of IT projects: projects are led by people without any IT background, resulting in poor quality, out of hand budgets and lead time, resulting in a lot of frustration among employees.

So what to do?

My advice to organizations where IT is becoming a crucial part of their business: make sure you get someone in the board with an IT background. It doesn’t have to be a 100% IT person, but at least someone with some technical background and experience in leading IT projects. And yes, for many board rooms that means incorporating someone that is 10 years younger than the average board member!
Some organizations already appointed a CTO (Chief Technology Officer) or a CIO (Chief Information Officer). It doesn’t really matter how the position is called though, but put some IT background in the board, please! A smart board room will do it, it will save them a lot of money and frustration, the business will run smoother and it will also be more responsive to the future digital needs of the market.

The misfit between computers and humans: blaming the bit

Computers in the year 2015 are not much different from the first computers built in the seventies. Many will disagree on this with me, but let me refine myself: I think computers have not fundamentally changed. Sure, they’re faster, have more capacity and they’re much better connected to each other than they were before. They also have become much cheaper, what makes them much more available, and therefore, the amount of applications of the computer has exploded.

However, there’s one important thing that has not changed: the foundation of the computer, called the bit.

“Computers have not fundamentally changed over the years.”

Computers work, and have always worked, with bits. Zero or one, true or false. Furthermore, computers basically do two things only. One: they store bits on devices like a hard drive or other type of memory, and they can retrieve them from these devices. Two: computers can do (advanced) calculations (and comparisons) with bits, according to logical instructions. Current storage capacities and processor speeds make computers very powerful when it comes to calculations (think of weather prediction calculation models) and storing/retrieving bits (think of databases, or the Internet).

Although the bit definitely contributes to the computer’s power, at the same time it’s its limitation. All information you’d like to store on computers, or have computers work with (even the commands you’d like to give to a computer), needs somehow be converted to bits, or alternatively formulated: the information needs to be encoded according to some encoding mechanism (this mechanism I will get back to later on). When retrieved from the storage device, the bits need to be decoded to obtain the original information.

When it comes to numbers, this is quite easy, a bit already is numerical, and all possible numbers can be easily encoded to bits. But when it comes to other kinds of information, like language, images, speech, there’s a mismatch between the nature of the storage (bits) and the nature of the information, and storing it in bits does not seem so easy or efficient any more. Another difficulty with storing information in bits, is the fact that for humans, information always is related to other information. To make myself clear, let’s consider a piece of information: the word ‘phone’, for instance. For humans, this word is related to the image of a phone, the language English, the history of telephony, and many more things. But how to store all this using just 0’s and 1’s?

And there the problem arises: there’s a big misfit between humans and computers. When we look at the human brain, although we still don’t know exactly how it works, we see that it does not work with bits. Information is stored without any conversion or encoding, at least, not in the way a computer does this, using bits. And also an important difference: information is not stored in local packages, like a computer does on a storage device. When new information is added to the brain, it is being integrated in the brain, not simply added (like computers do). All kinds of (neurological) relations are created in the brain the moment the new information arrives. Completely different from what computers do.

I’m not a brain expert, and it would go too far to go into the exact differences between the human brain and computers, but I think I can safely state that our brain works quite different from a computer. And personally I think this is the reason why it’s sometimes so hard to work with computers.

But let’s go one step back here. How DO we manage to work with a computer? The answer is: through an intermediary called software. Software is what we use to bridge the fundamental gap between computers and humans. Databases exist, because of software. Your monitor shows images, because of software. Your iPod plays music, because of software.

“Software is what we use to bridge the fundamental gap between computers and humans.”

Software also translates human commands to machine code. However, this does not mean interaction between humans and computers is always smoothly. In fact, the communication between humans and computers is still very framed and limited. The intermediary software has its limitations. Not because it is not a good enough translator, no, it’s just that not all human communication can be translated to computer code, there’s a fundamental limitation. When we type or click very specific commands, then yes, software can translate this easily. But when we start talking to our computer, well, that’s a whole different story. Maybe one day there will be software that will perfectly be able to catch what we’re saying (i.e. make letters, words and sentences out of the sounds we produce with our voice), but then again, how to translate this to commands a computer will be able to understand and act upon? Don’t forget a computer just takes 0’s, 1’s and calculation commands.

Let me give an example. Let’s say I’d like to tell (verbally) my computer: ‘Book me the first flight to New York’. I can imagine there will be software that understands this spoken sentence, and translate it to computer commands. I can imagine it in this particular case, because it is a very well defined command. Assuming the computer has access to flight booking systems, the computer will just search these systems for flights to New York, find the first one, book it, even executes the payment.

Now, let’s change the sentence slightly: ‘Book me the first cheap flight to New York’. I added just one word: ‘cheap’. For a computer, this changes everything! Because of the word ‘cheap’ the command is not well defined any more, at least not for a computer, and cannot be translated to machine code. What is cheap for a flight to New York? 500 dollars? 1000 dollars? Exact instructions please!

This is just a simple example, regarding a relatively simple task of booking a flight. I hope it illustrates how big the misfit is between humans and computers, and how limited the possibilities for software are to close this misfit. Software will become more advanced in the future, no doubt about that. But without any fundamental change in computers, I don’t think the misfit will ever disappear. If we really want to get along well with computers, or drastically increase the capabilities of computers (to get to advanced artificial intelligence for instance), we have to make them more human. And that would mean getting rid of the bit.

“Without any fundamental change in computers, I don’t think the misfit will ever disappear.”

Communication with specialists, or how to recognize a good manager

Working in IT means working with specialists. The IT world is one with a large amount of specializations (technical, methodological, business-wise), that is still growing every day. The same can be said of many other lines of business and it is even true for non-business area’s like science: it’s a scientist’s job to know a lot about very little.

Apparently, the world needs specialists. I would even take it one step further: specialization is probably inherent to economic activity. Every company has specialists employed, and as the company grows, the rate of specialization of the individual employees increases. In a small company, one person can act as a management assistant, sales assistant and takes care of the company purchases. As the company grows, these tasks will be distributed over several persons, meaning they will specialize in a certain activity.

This is a normal development, but together with the creation of specialists, another kind of employee is created, the one that is generally called the manager. The manager does not have a specific specialization, at least not at the level of his/her subordinates. The manager’s job is basically about ensuring the several specialists work well together, and contribute efficiently to the objectives of the overall organization.

So, seen from this perspective, what would be the number 1 skill for a manager? Communication with specialists. Or more specifically defined, communication with people that know more about a certain ‘area’ than the manager does. This ‘area’ could be pure knowledge (like in science), it could also be certain tasks or activities (like communication with customers).

Communication is a two way process, and in this case there’s the way from the manager to the specialist, and there’s the other way, from the specialist to the manager. A good manager tries to optimize both ways. How? Well, basically by doing this:

The manager keeps up with the specialist’s skills and knowledge, to a certain level. For good communication, a manager must at least know the basics of the area his/her specialized subordinates works in. By experience or by education. Important to add: this could be a continuous process, meaning also a manager may need to attend courses, seminars or other ways to get informed about new developments in specialized areas. A good manager can be recognized by doing this. The opposite is a manager that doesn’t, and eventually he/she will lose contact with his/her subordinates: he just doesn’t know or understand anymore what the specialist is doing exactly.
The other thing a manager should do, in order to maintain good communication, is encourage the specialists to communicate about their work with non-specialists. Specialists tend to communicate well with other specialists (especially within the same area of expertise), but in a multidisciplinary company it is essential that they are also able to convey certain information about their activities to non-specialists (like managers or employees of other departments). You may call this collaborating, but basically I’m talking about a certain condition that is needed even before actually collaborating: communication skills. A good manager can be recognized by promoting this. The promotion can be done in several ways, one of them is organizing sessions where specialists tell about their work, about what they do, or about new developments in their field of specialization. The opposite is a manager that doesn’t do this: isolated employees are the result, and it is not difficult to imagine that all kinds of problems will arise: inefficient collaboration in general, problematic projects and miscommunication between management and specialists more particularly.

Of course, the above I write based on my personal experience. I try to do both things, whenever I work as a (project) manager or team leader, and it works. I work mostly in IT, and in there I see a lot of miscommunication going on between managers and specialists. Personally, I rarely encounter IT managers that actively try to improve communication, by doing the two things I mentioned above. But the ones that I’ve seen doing it, were quite successful in their jobs. And the ones I’ve seen not doing it at all, well, they basically caused a lot of problems and did not hold their position for a long time.

Top 3 mistakes made by IT clients

In previous posts, I explained why software development is complicated and susceptible to errors. Subsequently, I tried to find out out why especially large governmental IT projects fail.

In this article, I point my arrows towards the clients of IT. It’s my opinion that a lot of problems in IT projects start at the client side. Clients are usually very passionate about their own business (not being IT), have all kinds of ideas about how IT should support their business, but do not put much effort in trying to acquire at least some basic knowledge of IT (‘I don’t want to understand it, just use it’).

So what does this attitude lead to? Here’s my list of top mistakes made by clients in IT projects:

Mistake number 1: Insufficient IT knowledge, no IT policy nor strategy to lead the way

This can hardly be called a mistake, rather is it a fundamental problem among many IT clients. Like I stated above, very often a client’s attitude towards IT is rather egocentric: they are interested only in what IT can do for them when they need it (short term vision), they don’t show much interest in IT itself. Fair enough, but if IT is important for your business, this attitude makes collaboration with IT suppliers very difficult. IT suppliers put effort in getting to know your business (if they’re any good), as an IT client you should also put effort in getting to know the IT business. This can be done several ways: have certain employees follow some basic IT courses, attend IT seminars, the bottom line is: get sufficient IT knowledge inside your company. Use this knowledge to be in control of your IT and IT projects (and not leave the control to your IT suppliers). Establish an IT policy or strategy, and follow it in the execution of your IT projects.
Altogether it will make the collaboration with IT suppliers much more efficient, resulting in better IT for your organization.

Mistake number 2: Make inconsistent or overly complex demands

Many IT projects start by collecting requirements. All involved departments, managers or employees are consulted, and they’re asked, what does the new system need to be like? All wishes and requirements are collected by a project manager (many times without much knowledge of system design or development), written down in a document and presented to potential IT suppliers.
The problem with this kind of ‘democratic’ collection of requirements, is that many times it lacks direction by a process analyst, system analyst or anybody else that is able to point out things like inconsistencies or overly complex requirements.
IT suppliers are usually not in the position to tell a potential client that their requirements are inconsistent or lack structure. They’d basically have to say: this is not smart, you shouldn’t do it this way. Of course, they don’t always say this, instead they say (because they want the project): OK, if you want it that way, we’ll do it that way. It’s very likely that a rather difficult project will follow, and all the activities that are needed to measure up to the complex requirements need to be paid for sooner or later. By the IT client.

Mistake number 3: Tendency towards custom made solutions

Related to the previous issue, IT clients (especially the ignorant ones) tend to go for solutions that perfectly meet their wishes. Apart from the fact that these wishes are not always valid, useful or consistent, many times they require custom made technology (or at the best customization of some standard technology). Again, in many cases this is a big mistake. Customizations are expensive, difficult to maintain, and create dependency of the IT supplier that created the customizations (in general it’s dangerous to depend on one supplier). Sooner or later (expensive) drawbacks will show up.
Especially if you don’t know much about IT, go for standard solutions that are based on proven technology and are widely used. You’ll find many IT suppliers that can help you with these kinds of solutions (so no dependency on a specific IT supplier).
A simple example is the creation of a new website. Websites nowadays are built within a CMS (Content Management System). Being a company that does not know much about IT, always go for a website based upon a widely used CMS like WordPress, Joomla or Drupal. This way you’ll have a large pool of IT suppliers that can create, maintain or extend your website. Some web design companies offer their own CMS, creating a potentially very lucrative technical dependency, because they’ll be the only ones that have knowledge of the technology behind your website.

Of course, these 3 mistakes are very much related to each other. And needless to say, IT clients make many more mistakes, but these are the ones I encounter most frequently. Other experiences? Feel free to leave a comment.

Why Dutch governmental ICT projects fail

Quite frequently, articles appear in the Dutch press about problematic ICT projects of Dutch governmental bodies. Budgets that get seriously out of hand, time schedules that are exceeded by years, products that do not meet expectations. Recent examples of projects that can be characterized as very problematic are the SPEER project for the Dutch ministry of Defense (estimates of expenses on this project go as high as 1 billion euros, the project should be finished in 2009, is still going on, and there are still no deliverables that meet expectations), and the OV chipcard (a system for electronic registration and payment for all public transport in the Netherlands, the project needed a very long time to release the actual product and get people to use it, it still suffers from security issues and a low customer satisfaction level).

In 2014 a parliamentarian inquiry about Dutch governmental ICT projects has been carried out. I’d like to highlight two things that can be read in the final report (you can download the whole report here):

Estimates (by IT experts) of yearly losses due to failing governmental ICT projects in the Netherlands reach as high as several billions of euros. This would correspond to up to 1% of Dutch GDP, or several hundreds of euros per year per Dutch inhabitant. According to these estimates, the problem could be called considerable. It has to be mentioned that the estimates of the losses vary a lot, apparently it is not easy to make these estimates.
International research shows that for governmental ICT projects (world wide) with a budget higher than 10 million dollar the success rate (successful being defined as within budget and time schedule) is 7%. 36% of these projects fails completely (system never released). Apparently, it does not seem to be an exclusive Dutch problem.

So why would this be? In the report of the parliamentarian inquiry (219 pages) many reasons, problems, and suggestions for improvement are described. But what is the bottom line? I thought about it, and came to these points:

Large ICT projects are difficult in general

In a previous post I explained why software development (habitually an important component in ICT projects) is often underestimated regarding complexity and risk for failure. Complexity and risks correlate strongly with project size. Governmental ICT projects are among the largest ICT projects you’ll find. The amount of end users is huge, as is the amount of processes the project should serve, resulting in large amounts of requirements for the ICT project. Governmental organizational, technical and financial structures are also very complex, which makes it very difficult to carry out ICT projects: they usually involve several governmental departments or organizational units.

Governmental ICT projects tend to be custom made

An important characteristic of especially the ambitious governmental ICT projects (I mentioned a few in the introduction) is the fact that the system that should result from it, is quite unique. It’s not a ready-to-go out-of-the-box type of solution. The project has to deal with some very specific requirements, that apparently cannot be addressed by proven or existing technology. So at least part of the project needs a custom made solution. New software or technology is developed especially for this project. However, this creates two risks:

The risk that the newly developed technology is not (at least initially) adequate. This is not unusual for new technologies in general: they tend to have many teething problems, and it takes a long time before they become stable and sufficiently advanced to meet user’s expectations and needs.
Dependency on the supplier of the newly developed technology. Especially for knowledge intensive technology like software, this is a major risk. In case the supplier does not deliver up to expectations, there’s no easy alternative. In ICT it’s not unusual that suppliers take advantage of this dependency, resulting in large profits for the supplier, and out of control budgets and product quality for the client.

Governmental ICT projects are subject to political decisions and influences

In my opinion, ICT projects, to become successful, need a stable environment, well defined working procedures and project organization, together with firm leadership to defend it. The world of politics is not exactly an environment that can be characterized as such. In politics decisions (including policy making) are not always based on sound knowledge, insights or vision. Many times, the result of a political discussion is some kind of compromise, satisfying all stakeholders, but not necessarily the best decision for a pragmatic project like an ICT project.

The Dutch situation

So far my reasons for failing governmental ICT projects anywhere on earth. However, it’s my personal opinion that in the Netherlands, with a very fragmented political landscape (no less than 17 political parties are represented in the Dutch parliament, senate or European parliament!), and a political culture of participation and deliberation, there is a high risk of starting overly ambitious ICT projects, based on complicated political compromises.
Besides, the Netherlands is a small country with a dense population, with an intrinsic need for organizing and regulating things, not rarely choosing (still to be developed…) IT technology as a solution for complicated problems.

Altogether many risk factors for governmental ICT projects. They’re not easy to avoid, so, despite the recommendations of the 2014 parliamentarian commission, I expect governmental ICT projects to keep on failing in the future, unfortunately.

Why software development is complicated and susceptible to errors

Software projects are often problematic. They take a long time, are expensive, need a lot of maintenance, tend to have bugs in it, and quite often the final result does not work according to the original expectations: intuitive to work with, without any failures or bugs, and fast.

In a previous post I described why software development can be compared to building a house. In a way, this article will be about why it can not be compared to building a house: software development is much more difficult than building a house. But why exactly is it so difficult to develop good software? In this article I’ll summarize the most important reasons for this phenomenon.

The elements of software are small and many

Software consists of lines of code. A software package like Microsoft Office contains millions and millions of lines of code. Not all of it is handwritten (some code can be generated by computers), but a lot is. To compare with: a book of 300 pages contains around 10 thousand lines of text. So, to write a software application like Microsoft Word, you need to write at least a thousand books in computer language. Not only is this a lot of work, every line has to be correct as well, a failure in a line could potentially disable the whole software program.

Software depends on hardware

Software needs a computer or a server. It uses computer memory, disk space, network connections. Any failure here will stop the software from working, at least partially. Good practice is to have backup hardware. The complication here is that you need software that manages an automatic transition to backup hardware in case of a hardware failure. So software that makes sure the principal software will still run in case of hardware failures. Complicated, right? Yes, and therefore sensitive to errors.
Another factor that relates to the hardware dependency is the fact that hardware developments go fast. It is not unusual to replace hardware every 4 years with more powerful and faster hardware. Of course, this has consequences for the software as well.

Software depends on other software

I just mentioned this point already: most software is not standalone: it depends on other software. Most commonly it depends on an operating system. An operating system is a software package that operates the hardware the software will run on. Windows is an operating system, and Linux as well. Operating systems are among the largest software packages you will find. It consists of billions of lines of code.
Failures in the operating system could have consequences for the functioning of a software program that depends on it. Upgrades of the operating system (all operating systems need frequent upgrades for security or other reasons) also might affect the working of a software program that relies on it.
Similar to developments in hardware, software developments go quite fast as well. Operating systems are replaced about every 5 years, having consequences for all software that relies on it.

Software applications are prone to functional changes

Software is very much related to social and work processes: for working processes you can think of applications like Enterprise Resource Planning (ERP) or Customer Relationship Management (CRM), for social processes world wide known examples are LinkedIn and Facebook. Any kind of social or work process is dynamic, and therefore software will need to keep up with this dynamics. New requirements for software appear frequently, which causes most software to be in a continuous state of change. What does this mean? It means software should always be built in such way that it can be adjusted relatively easy. What makes this a very difficult task is the fact that it is unknown what the future adjustments will be like exactly.
Building a good software application with fixed requirements is already quite difficult, building it in a way that adjustments are easily possible is even more difficult.

95% of a software program is invisible, and people don’t realize this

The visible part of a software program is called the user interface. However, most of the program is not visible: it’s behind the screen. Complex architecture and an enormous amount of code lines are hidden from the user of the program. At first sight, a program may look simple, however, behind the screens it can be very complex. People don’t realize this, even managers of software projects don’t always have a clue about the complexity of a software program.
This becomes a problem when decisions or estimates have to be made regarding a software project. Budget estimates, project schedule, skilled people to be hired. If one does not understand the complexity of a software program, it is easy to make judgmental errors, resulting in problems of several kind (projects getting out of budget, instable software, etc.).
Many of the problems in software projects are related to this: decision making by people with insufficient knowledge or insight.

Anyone can develop software, very few are able to develop good software

Anyone with a computer is able to do software development. Software development tools are available on the Internet, most of them for free. Nowadays it’s quite easy to build your own website for instance.
However, as the size of the software application increases, the learning curve and the skills needed to be a successful software developer, do rapidly grow. To be able to work within an environment of millions of lines of code, you need to know about software architecture and design patterns, and know how to work with testing frameworks, for instance.
Not all software developers have this knowledge and these skills, and it shows, efficiency and productivity among software developers and architects differ by a factor 20.
Good developers are hard to recognize by people that do not have software development knowledge themselves (like most recruiters), so it happens quite frequently that software developers with insufficient skills are assigned to ambitious software projects, resulting in problematic software development.

All together, many factors why software development is difficult and can become problematic. To a certain extent, things can always be improved, for instance by improving IT knowledge of people in and around the software development process. Another option would be standardization. However, this contradicts in a way the power of software development: its flexibility. There are no physical boundaries, possibilities are endless: software can be developed totally tailor made.

The counter side of this flexibility may be clear by now: low level of standardization and intransparent complexity. My tip on how to control this should also be clear: skills and knowledge!

Why software development compares to building a house

Most of my clients, when discussing software development, they justify themselves by telling me that they don’t understand much about it. I always respond that, in order for their software development projects to be successful, they have to understand some basic concepts at least. However, they’re usually not very motivated to learn about software development. “Not my piece of cake”, “I just want to use the software, not understand it”, “Too complicated for me”, or even “It doesn’t interest me”.

Yet, I try to obtain their interest anyway. If you want to have your own software built (like many of my clients), I’m very sorry, but you just have to understand some things about it. One of the strategies I use in my communication, is to compare software development to something people do know about. For instance, development of software can be compared to building a car. In this article I will use another analogy, by comparing software development to the construction of a house.

Here are four similarities. Useful for anyone that struggles in communicating with software builders or software sellers. Interesting for everyone else!

Think twice before you build your own house

Let’s start with an observation. Most people don’t build their own house (or have it built). They buy a house that’s already out there. Sometimes it’s brand new, most of the times it’s been used before. Why? Because these houses are just there: they’ve proven themselves to be useful houses. Why bother building a house from scratch if you can find an existing one that fits your needs and desires? Building a house is difficult, even if you don’t build it yourself, you’ll need to know at least something about architecture and construction.
The same thing applies to software: use existing and proven software if you can, and if not, have your own software built only in case:

you have very specific wishes
you know at least the basics of software development yourself
you have a very good budget to spend

Design is everything

Let’s say you do want to have your own house be constructed, and you can’t do it yourself. How do you start? Well, you need a design, right? A good design! Therefore you hire an architect who will do basically two things:

make a design that reflects how you would like the house to look like: exterior, rooms, floors, materials etc.
make a design that proves how the house should be built: what are the important walls, what are the exact dimensions, what parts are reinforced, what quality of materials is to be used, etc.

The first one can be called a functional design, and this is where you, the client, should (and probably wants to!) play an active role. The second one is the technical design. As a client that does not know much about construction, you’ll probably not interfere much. Good practice however would be to have the technical design be made by an independent architect, i.e. not someone that is involved in the construction itself. Why? Let’s assume you have the construction worker make the technical design (so not an independent architect). He’ll probably does it in such way that it serves his own interest: maybe he’ll implement very expensive materials you don’t really need (but he’ll make good money out of it) or he leaves out security components he thinks are too much work to implement.
Now back to software development: it works exactly the same! You need a functional and a technical design. And you want the designer to be somebody else than the software developer, if your aim is to have long lasting and price-quality balanced software. Needless to mention: bad design = bad house.

Building is composing

As the preceding paragraph pointed out, a good house is built according to a good plan. However, it is not necessary to completely construct it from scratch. For a great part, the constructor uses components and puts them together. The design leaves a hole in the wall, a window (=component) is put in there. In case the window ever needs to be replaced, it can be easily done so. This is characteristic of components: they’re rather independent parts of a construction. The constructor’s effort is this: integrate the component in the bigger whole.
Good software is built, and characterized, in the same way. If possible, and preferably, existing components are used (the implementation requires some effort) within a certain architecture. Only in case no useful component exists for a certain feature or requirement, it is good practice to create a new component.
Example: say your software application needs to be able to exchange information with your bank. A component is implemented (existing or newly created), that does just this. In case you move to another bank, only this component needs to be replaced (or adjusted), the rest of the architecture will not need any adjustments.

You’ll need maintenance

A house needs maintenance, not much explanation necessary here. You’ll need to paint wooden parts once in a while, especially the ones exposed to wind and weather. A roof does not have eternal life for the same reason.
Of course, software is not exposed to wind and weather, still, it needs maintenance. Basically this is because software is almost never standalone or independent. It always depends on hardware (a PC, smart phone or server) and other software (a software system that operates the hardware that is used: the operating system, like Windows or Linux). These two, hardware and operating system tend to come with new versions every few years, having consequences for all software applications that depend on them. Here’s a longer list of common reasons for maintenance of software:

hardware or operating system upgrade (just explained)
security reasons (especially when the software communicates with the world outside, sometimes you just have to make things more secure)
bugs: unfortunately software is, and will never be, failure free

One last observation: when you notice that your house needs an extraordinary amount of maintenance, something more fundamental is wrong. Most probably there’s a serious construction error (usage of some inadequate components if you’re lucky, fairly easy to replace), or much worse, the house has been built according to an inadequate architecture, which is very difficult, if not impossible to fix. Needless to say, the exact same thing applies to software!

Self SEO

Being a modest person, I don’t like to be in the spotlights very much. I’m not too fond of presentations, sketches at wedding parties or karaoke. At birthday parties (if I go at all), I don’t feel the need to tell the most amazing story you’ve ever heard, or the funniest joke you can imagine. I guess, well no, I’m quite sure, I’m more an observer than a performer.

Nothing wrong with that, I’d say, the world needs performers as well as observers. However, all observers need to perform sometimes, and all performers need to observe every now and then. As a freelance worker I also have to stand out somehow, for potential clients to learn about me, or recruiters to offer me assignments.

Not so easy, as there are thousands of consultants like me, how do I show the world that I’m the one you need? My own area of expertise is IT, so standing out on the Internet should be the least to strive for. I’ve got my own website since 6 years, renewed it completely recently. According to LinkedIn my profile strength is almost All-Star like. I don’t know what that means exactly, I thought All Star is for basketball players, but at least I can associate the word “star” with standing out, so it must be a good thing.

Google’s search results for ‘Arnaud de Klerk’

Next thing, let me google myself, see how I’m ranked by world’s leading search engine. First hit: Twitter account of someone that goes by the exact name as I do (as far as I know, there are only two persons in the world that go by my exact name). Google’s description line of this hit: “Nerd met een voorliefde voor muziek, concerten, (uit) eten en borsten!” (translate this). Not me. Not good. Not good at all!

All right then, clear. Time for some serious self marketing, and more specifically, spoken in terms of Internet marketing: time for some serious self SEO (for those who don’t do self marketing: Search Engine Optimization)! Within a few weeks, I’ll have to be at the top of the ‘Arnaud de Klerk’ Google’s search action, with a serious line of information about me, that’s for sure!