Implementing an Effective Software Development Process: Agile, Scrum, CMMI and Where to Start
The first problem when reading through the literature about agile and scrum: it seems overwhelming. Reading about CMMI sends you even further into the bush. The large IT providers easily assign some people to study, master and implement such complex processes. But the biggest part of the software industry consists of small and medium sized firms. In Europe, the biggest part of the software firms have somewhere between 10 and 50 employees. If you work at such firm, you probably want to get going, produce and get your projects ready. The thing on your mind is to get that new product release out on the market, to speed up your project to make a deadline. You don't have the time to learn a new process, think everything through, document it and implement the change.
The common way of organizing software projects in most software firms is (or was) the waterfall method. Most firms don't choose this process deliberately, but grow into it, because it's a natural human way to get organized. It has its parallel to the construction world: you have an idea of your dream house; you hire an architect who captures all your ideas and puts it into a drawing; the drawing gives a very clear picture of the house and matches with the picture in your mind; you accept the drawing; the drawing goes to the construction firm who tells you what it'll cost to build the house; and so the project starts. In most cases, the construction firm gets it right: the house is like the picture. What they frequently don't get right is the timing, but as we've grown accustomed to that, we accept the delay and we're happy t o move into the new house.
I think there are 2 crucial differences between construction and software development, which agile and scrum address:
It is very hard (maybe even impossible) to clearly picture the software application before it exists
It is much harder to organize, because the thought process and the execution don't follow linear patterns
Whatever process you want to implement, however you call it, there are two things that can improve a software development process drastically (those two are also central parts of scrum and agile):
1. Create the picture in steps instead of upfront.
For smaller projects, this might be over-organization, because the picture can be made clear upfront. For bigger projects, it brings substantial improvement. Define the high level requirements of the project (create a rough picture), then cut out a part that can be completed within two weeks. Divide that part into clearly defined tasks and start building. Anything that has been built in those two weeks has to be tested and reviewed thoroughly and based on the results, the part and tasks for the next two weeks are defined. This gives a more flexible approach to building the picture and gives an opportunity to 'steer' the project in the right direction.
2. Organize communication
In the waterfall method (and the natural human organization method), the requirements are supposed to be clear, so we let the programmers do their work and expect to see the result when the deadline is due. Whatever happens in between, we expect them to fix. The problem in this organization is communication or the lack of it. Scrum and Agile have daily meetings as its cornerstone. Every day at a fixed location, with fixed people on a fixed time, the team discusses three questions: 1. What have you done yesterday?; 2. What will you do today?; 3. Where are you stuck?. Especially this last question is what brings a leap in productivity. Programmers often bang with their heads against a closed door. If you don't hear them banging, they sometimes keep doing it for a week, believing that they'll catch up the time they lose later o n, trusting that they'll get through that door. Daily meetings ensure that everyone hears the banging and puts the brains of the team into opening the door.
From these two, the second has the biggest impact and is the simplest to implement. If you are not ready to implement a whole new process, start with organizing communication. While building a house, the supervisor walks around daily and will see it when someone can't get the door in place. He will walk to the person and ask what's happening and will organize help to fix the door naturally. In software development, the door is invisible, the solution to fix the door is not straightforward and programmers find pride in solving complex problems and will try anything (for days in a row) to solve it. Communicate, ask the three questions every day and your projects will start moving faster and more fluid.
