How to transform an organization

To me, the big theme of this year’s Scan-Agile conference was that transforming individual teams to agile is not enough – the entire organization needs to change. Converting a team to, say, Scrum is a waste of time if the increased output and flexibility cannot be used by the organization. It can be even harmful if it puts more pressure on other places of the organization. (And of course, it may be impossible for the team to change if the organization doesn’t.)

Based on the teachings of the conference, if I had to transform an organization, how would I do it? Here’s how:

1. Can you draw the value stream map? If not, simplify your product offerings until you can.
2. Draw the value stream map. Find out where the bottlenecks are.
3. Slow down the value stream to the speed of the bottleneck.
4. Find ways to speed up the stream at the bottleneck.
5. Go to 2.

Any process changes, e.g. the introduction of agile development practices or beyond-budgeting principles, should have the goal of improving the value stream. If the change cannot be justified by looking at the value stream, it should not be made.

I don’t know much about value stream mapping or theory of constraints, but I think I ought to. I hear The Goal is the book to read.

I have only tinkered with process changes on the team level. You with more experience, please tell me, did I understand things correctly?

Let’s change hankintalaki

I’ve heard from many mouths that hankintalaki (the Finnish law governing publicly funded purchases) prevents effective software development for the Finnish public sector. The parliament is going to make changes to the law this autumn. Now would be a good time to tell our representatives what we think.

To start with, I would like to understand the exact problems with the law. How does it inhibit success of software projects? How should the law be changed?

Also welcome are any ideas about how to express our concerns effectively to lawmakers. I am trying to get the parliaments’s tietoyhteiskuntaryhmä (Information Society Group) hear one of us, but as far as I know, nothing else has been planned so far.

I also sent this question to the Agile Finland mailing list. Please let me know if you write about this on your blog. I’ll summarize the answers in a later post.

Learning and doing

Question

Think about your last project. How large portion of the effort was spent on writing actually useful features for the customer?

We are talking about actually useful here, from the customer’s point of view. Do not include the time spent on testing, meetings, waiting for the code to compile, duplicate code that could be refactored away, etc.

Do you have an answer? How much is it?

My answer

Let me do a ball-park estimate of a project from my past.

The codebase consisted of 80,000 lines of code. Roughly half of those were unit tests, developer scripts, and other non-production code so that brings the number down to 40,000 lines. At least half of the code could have been refactored away: 20,000 lines left. And about half of those lines were for features with negligible purpose, such as unnecessary user configuration options. So, I would estimate that 10,000 lines of code were actually useful.

Including the entire project team – developers, testers, project managers, customer representatives – the project took about 10 person-years. That is roughly 2,300 person-days, 17,000 person-hours, or 1,000,000 person-minutes.

Now, if we generously assume that each line of code takes 1 minute to type, it took about 10,000 minutes to type in the actually useful code. That is 1% of the 1,000,000 person-minute total.

So my answer is: about 1% of the time was spent on writing actually useful features for the customer.

Now, I know what you must be thinking: what a crappy project. But no, not at all. The project received high praise from the customer and spawned more project orders. Both the customer organization and the development organization started an effort to spread some of this project’s methods to other projects. This project succeeded.

Even in a successful software project only 1% of time is spent on writing actually useful features.

Learning cycle

The other 99% goes to finding out what are we trying to do and how we can do it.

We are learning about the product we are building, the interfaces we are using, the customer domain, and so on. We even learn about the code we wrote ourselves. Every bug report is feedback and every bug fix a start of another iteration on that minicycle.

To be more productive, we should focus on the 99%, not on the 1%. How can we learn faster?

Pareto and headache

Raganwald asks:

1. 20% of the features are responsible for 80% of the headaches of software development, and;
2. 20% of the features are responsible for 80% of the value of the software to its users.

Are those the same 20% of the features on your project? If not, why not?

I cannot decide if that is a rhetorical question or not, so here’s my answer:

At the beginning of a project, the two are unrelated. What happens to them during the project largely determines its success.

A good team quickly notices which features cause more headaches. They will ask if the expected value of these features is great enough to justify the pain. If not, they throw those features away. The longer the project lasts, the more aligned the values of the features are to the pain they cause.

There are at least two ways to fail in this. First, the team may not notice early enough that a feature is painful. When they do notice, the pain of an alternate path may have grown beyond the expected pain of finishing and maintaining the feature.

Secondly, the team may be unable or unwilling to choose an alternate path. They have a contract or a boss or a mindset that tells them to finish the software the way it was originally planned.

Unfortunately, in most projects I have participated in, there has been little overlap between the painful and the useful features. I believe the same holds true for the industry in general.

A word of warning. If you think this is an argument for agile development, think again. If you think this is an argument for waterfall development, think again, too.

(Not) coping with multiple projects

I haven’t posted in a long time. The reason is that my mind has been elsewhere. I’ve had a lot to do on other projects and I could not put enough effort on blogging. (There is another reason, but I’ll deal with that in a later post.)

I am not a very efficient multitasker (although, who is?) so I instinctively shun away from multiple projects. Through experiment and experience, I’ve developed an understanding of my internal project portfolio rules. Here they are, put into words for the very first time:

• I can have one primary project.
• Concurrently to the primary project I can have one secondary project.
• There are no tertiary projects.
• Secondary project cannot have a deadline.

The rules are descriptive, not prescriptive. That is, this is my understanding of how I work, internally.

Sometimes I don’t listen to my internal voice and try to disobey the rules. What happens then in my head is a redefinition of projects until the rules are satisfied again. This often happens implicitly, with bad consequences.

Say, for instance, that I am working on a primary project and a secondary project within the limits of my internal rules. A third project comes along that I feel is more important than either of the two. I put my energy into it, and magically, one of the other projects drops away, without me even noticing it.

A dropped project is no longer a project — it is just a thing I am supposed to work on, a thing I have lost my passion to do. Automatically, I try to avoid it, and if I absolutely have to work on it, my mind is elsewhere. Whatever the result, I will not be proud of it, since it is not my child anymore.

My best strategy to deal with these rules is to have one long project (primary or secondary) overlapping a lot of short projects (of the other variety). The long project keeps utilization high, and the short projects ensure that many projects finish. Incidentally, it makes sure I have time to write the blog as well.

Ruby Training at Helsinki in December

Rapid Ruby -noutopöytä is a Ruby course that let’s you choose the bits that interest you the most. I am giving it (in Finnish) together with Jarkko Laine on December 1-5 at Helsinki. We are focusing on actual programming, so you will get plenty of practice.

Persuade your employer to pay the bill, and it will be worth every penny. (To your employer too.)

Play your moves

If I had to play an even game of go against Takemiya Masaki, I would lose. I could have three hours of thinking time per move, he five seconds, and he would still beat me.

Takemiya has a master’s intuition and I don’t. His moves just work, and it looks like magic to me. No amount of elaborate scheming could turn the game to my victory.

Asked what amateurs should do to get stronger, Takemiya said: “It’s very important to play what you feel. It’s best not to think of the difficult parts of go or to worry about it too much.”

You cannot train your intuition if you worry about what your teacher would play. When confronted with a choice between the “right” move and the move that you like the best, choose your move.

When you play your move, you are excited and your heart pounds quicker. The worst that can happen is that you lose the game. The worst that can happen? No, the best! You have learnt something, and your hormones help you to remember it. Your intuition is a little closer to perfection.

If you only play other people’s moves, you will never know why (and if) they are the correct moves. Worse yet, you learn to ignore intuition, your most important tool.

Software design is not unlike go.

Every methodology I’ve come across has, at its kernel, a very small section labelled “do magic here”. — Katie Lucas

The magic is using your intuition. You can create a first-class design if your intuition is good. Otherwise it is like playing an even game of go against Takemiya Masaki. No amount of time will be enough to win you the game.

No amount of time? Well, obviously, if you are five years old and have a knack for go, by training for twenty years you may become as strong as Takemiya.

Similarly, to design great software, you should train your intuition first. Read books, learn from others and above all, play your moves.

Go computers, go!

The MoGo team is optimistic:

“The current result forecasts that before 2020 a computer program will defeat the best human Go player on a 19×19 Go board in a regular match under normal tournament conditions. This is remarkable, since around 2000 it was generally believed that the game of Go was safe to any attack by a computer program. The 9-stones handicap victory casts severe doubts on this belief.”

Perhaps.

(game record)		(game record)
USGC, August 1998		USGC, August 2008
Black:	Many Faces of Go, computer program	Black:	MoGo Titan, computer program
White:	Martin Müller, 6-dan amateur	White:	Kim Myung-Wan, 8-dan professional
Handicap:	29 stones	Handicap:	9 stones
Result:	White wins	Result:	Black wins

Love of abstractions

Once, when I was seven, my teacher gave the class an assignment to draw a picture of a bird. We used wax crayons and the teacher told us: “Be sure to use colors on every part of the picture. There should be no white on the paper when you finish.” I set out to draw the most beautiful bird I could, but I soon found the task Herculean. I pressed as hard as I could, but tiny spots of white always remained within the strokes of my crayon.

At the end of the lesson, I had barely managed to fill a dozen square centimeters with satisfactory saturation. The only color I had used was the blue of sky, as I had had no time to shape the bird yet.

I found it deeply unjust when the teacher disapproved my attempt, and congratulated pupils who had completed their drawings with a lighter hand.

Fast-forward twenty-two years.

I spent the last two weeks of my summer holiday at the European Go Congress. This was my first tournament in over two years, so I didn’t have high expectations. I ended up winning six games out of ten, which I was very happy about.

I adore this game. The way high-level concepts emerge from simple rules is very similar to how numbers form beautiful patterns in mathematics.

Come to think of it, this is almost the definition of fascinating: complex behavior arising from simple systems.

This is what draws people to computer science, and this is what draws them to go.

As I did twenty-two years ago, I still find it natural to start thoughts from a blank slate, with no presuppositions. When I am given a set of rules, I intrinsically consider them to be complete – even though I don’t follow all rules quite as literally as I did back then.

I am thankful that I have a hobby and a job where this trait of mine is a strength instead of a weakness.

Agile Web Development course report

I wrote a report about the course I taught at Tampere University of Technology in April: Experiences from Agile Web Development course, spring 2008.