Dan Bergh Johnsson's Blog
Thoughts on Programming; what we do, and how we do it
Updated: 19 min 56 sec ago
In Agile, the Practitioners Own the Process
Dear Junior
The term "process management" might be one of the driest around in system development, but in the shift to agile, the term hides a secret both surprising and delightful.
Demystifying "process", is nothing but our daily habits, what we do when creating systems. Examples of process include how we decide when we commit stuff, how we manage our requirements, whether we do pair programming, when to do code reviews, how we decide something is finished , etc. It is really about our every-day habits of life when working.
Put that way "process management" becomes how we decide what habits we should have, or how we change them, like if we should introduce code review, or take them away, if we should be more rigorous on "no failing tests" when committing, etc.
Put that way "process management" is simply how we get better at development over time.
Process Management Traditional Style
In traditional process management each organisation has some kind of authority that decides what process we should follow. It might come in the form of a "Chief Process Officer" or as a "Project Office", but the point is that somewhere we decide the process. The groups that do the daily work of creating and maintaining systems are then to follow this process. These groups can be projects, maintenance groups or have some other organisational structure. For simplicity let us call them "practitioners" for short. So, project office decide the process, practitioners follow the process.
In many cases "project office decide, practitioners follow" is an illusion. There actually exist (take my word for it) project offices that write a process behind locked doors and then hand out a binder to the rest of the org, while believing the process will be followed. Truth is that those documents will never be read outside the project office, and will obviously not be followed.
As an example, the project office might have decided that all code should be reviewed, and then fully believe that so is done — because it is in the process. The truth might be that no code is ever reviewed at all and the practitioners might not even know code review is part of the formal process. This "alienated project office" is of course a process management anti-pattern.
A good process office traditional style will of course behave differently. It will stay in touch with the practitioners and check whether the process is followed. If it is not, they will question why it is not followed, and explain the rationale of why they process look the way it does. It will also gather feedback and change the process to better fit changed of discovered needs.
However, even if the project offices adapt the process upon request, it is still in charge. If a practitioner what to change the process (e g "continuous, automated deployment to test environment"), she will suggest that change to the project office that will evaluate it. If the project office thinks it is a good idea, they will change the process, and the practitioner can start following the improved process. However, the practitioner cannot single-handed change the process just because she thought it was a "nice idea, worthy to give a try".
Process Management Agile Style
Agile development on the other hand relies heavily on the idea of continuous improvement. The basic idea is that that the process you have today is just the best you have come up with this far. Further more, the challenge of today is to find out what you should do different tomorrow. Agile development also honours doing experiments — if you get an interesting idea, then try it out for some time. At next retrospective, you evaluate the change and keep the it if it proved beneficial — or throw it away and try something else.
Experimentation is not only a right in agile, it can also be claimed to be a responsibility. Taking it to an extreme, it can even be claimed that if you have not changed your process in some significant way the last year, you are probably no longer agile. It is probably a sign you no longer do experimentation coming up with no ideas to try. That you have found "the perfect process" is just to improbable to be likely.
Here comes the mind-shift: for the practitioners to do this experimentation, they should have a mandate to do these continuous changes to the process. In other words, they should be empowered to change the process (ever so slightly) at their own will.They should not need to ask project office for permission. We want to keep the threshold for new ideas as low as possible.
Rephrased, the practitioners are actually unilaterally allowed to change the process. They now own the process they follow themselves, it is no longer owned by the project office.
Project Office Still Valuable
This does not mean that the project office becomes obsolete. Quite the opposite, it might even become more important. When all practitioners can form their process, it becomes even more interesting to have some group that pays attention to all these changes, gather the changes that where successful, and can promote cross learning between practitioners of different groups.
A team might for example struggles with that some pieces of the code is only understood by one single developer. Independently another team has solved a similar situation by using pair programming whenever such code is touched. The project office might then recognise the situation and suggest that the struggling team try that method to see if it will help them.
At this point of time the Project Office might change its name to enforce their new role. Some have re-branded themselves as the Agile Office, which I think is nice turn. Mike Cohn has reported that at one of his clients they called this group the "Agile Enablement Team" to emphasise that their goal is to enable other to become more agile.
What about Standard Process
In agile process management you can still have a standard process. The difference is that the standard process will not be pre-defined, but rather emerge. The project office will gather ideas that have worked at some team, e g "pair program on baldy understood code", and suggested it to some other team. If it worked at the second team as well it will promote the idea a little more active, and if it fits the organisation as a whole, it will soon enough be a standard among most or all teams. At this point "pair program on badly understood code" has become part of a de facto standard.
Over time, more a more practices will emerge and spread to all teams and a standard process will be formed. But, it will only do so if those practices actually work well throughout the organisation. And, it will not be cut in stone, but open to be challenged and evolved in the same dynamical manner.
So, the standard process will not become a standard process because someone say so, instead it will become the de facto standard process because a majority of the teams have tried and found the same set of ideas valuable and fit within the organisation. Here the project office plays a key role in facilitating and guiding this evolution so that the good ideas are spread and that the practises have a chance to converge instead of diverging.
The Shift
In traditional process management, the process is owned by the process office and followed by the practitioners.
In agile process management, the process is owned by the practitioners and facilitated by the process office.
Small mind shift in theory, huge difference in practice.
Yours
Dan
P.s. The situation of the Project Office in an agile environment resembles very much the situation of the architect who is unable to enforce architecture, but instead must work with the practitioners to explain and evolve it.
The term "process management" might be one of the driest around in system development, but in the shift to agile, the term hides a secret both surprising and delightful.
Demystifying "process", is nothing but our daily habits, what we do when creating systems. Examples of process include how we decide when we commit stuff, how we manage our requirements, whether we do pair programming, when to do code reviews, how we decide something is finished , etc. It is really about our every-day habits of life when working.
Put that way "process management" becomes how we decide what habits we should have, or how we change them, like if we should introduce code review, or take them away, if we should be more rigorous on "no failing tests" when committing, etc.
Put that way "process management" is simply how we get better at development over time.
Process Management Traditional Style
In traditional process management each organisation has some kind of authority that decides what process we should follow. It might come in the form of a "Chief Process Officer" or as a "Project Office", but the point is that somewhere we decide the process. The groups that do the daily work of creating and maintaining systems are then to follow this process. These groups can be projects, maintenance groups or have some other organisational structure. For simplicity let us call them "practitioners" for short. So, project office decide the process, practitioners follow the process.
In many cases "project office decide, practitioners follow" is an illusion. There actually exist (take my word for it) project offices that write a process behind locked doors and then hand out a binder to the rest of the org, while believing the process will be followed. Truth is that those documents will never be read outside the project office, and will obviously not be followed.
As an example, the project office might have decided that all code should be reviewed, and then fully believe that so is done — because it is in the process. The truth might be that no code is ever reviewed at all and the practitioners might not even know code review is part of the formal process. This "alienated project office" is of course a process management anti-pattern.
A good process office traditional style will of course behave differently. It will stay in touch with the practitioners and check whether the process is followed. If it is not, they will question why it is not followed, and explain the rationale of why they process look the way it does. It will also gather feedback and change the process to better fit changed of discovered needs.
However, even if the project offices adapt the process upon request, it is still in charge. If a practitioner what to change the process (e g "continuous, automated deployment to test environment"), she will suggest that change to the project office that will evaluate it. If the project office thinks it is a good idea, they will change the process, and the practitioner can start following the improved process. However, the practitioner cannot single-handed change the process just because she thought it was a "nice idea, worthy to give a try".
Process Management Agile Style
Agile development on the other hand relies heavily on the idea of continuous improvement. The basic idea is that that the process you have today is just the best you have come up with this far. Further more, the challenge of today is to find out what you should do different tomorrow. Agile development also honours doing experiments — if you get an interesting idea, then try it out for some time. At next retrospective, you evaluate the change and keep the it if it proved beneficial — or throw it away and try something else.
Experimentation is not only a right in agile, it can also be claimed to be a responsibility. Taking it to an extreme, it can even be claimed that if you have not changed your process in some significant way the last year, you are probably no longer agile. It is probably a sign you no longer do experimentation coming up with no ideas to try. That you have found "the perfect process" is just to improbable to be likely.
Here comes the mind-shift: for the practitioners to do this experimentation, they should have a mandate to do these continuous changes to the process. In other words, they should be empowered to change the process (ever so slightly) at their own will.They should not need to ask project office for permission. We want to keep the threshold for new ideas as low as possible.
Rephrased, the practitioners are actually unilaterally allowed to change the process. They now own the process they follow themselves, it is no longer owned by the project office.
Project Office Still Valuable
This does not mean that the project office becomes obsolete. Quite the opposite, it might even become more important. When all practitioners can form their process, it becomes even more interesting to have some group that pays attention to all these changes, gather the changes that where successful, and can promote cross learning between practitioners of different groups.
A team might for example struggles with that some pieces of the code is only understood by one single developer. Independently another team has solved a similar situation by using pair programming whenever such code is touched. The project office might then recognise the situation and suggest that the struggling team try that method to see if it will help them.
At this point of time the Project Office might change its name to enforce their new role. Some have re-branded themselves as the Agile Office, which I think is nice turn. Mike Cohn has reported that at one of his clients they called this group the "Agile Enablement Team" to emphasise that their goal is to enable other to become more agile.
What about Standard Process
In agile process management you can still have a standard process. The difference is that the standard process will not be pre-defined, but rather emerge. The project office will gather ideas that have worked at some team, e g "pair program on baldy understood code", and suggested it to some other team. If it worked at the second team as well it will promote the idea a little more active, and if it fits the organisation as a whole, it will soon enough be a standard among most or all teams. At this point "pair program on badly understood code" has become part of a de facto standard.
Over time, more a more practices will emerge and spread to all teams and a standard process will be formed. But, it will only do so if those practices actually work well throughout the organisation. And, it will not be cut in stone, but open to be challenged and evolved in the same dynamical manner.
So, the standard process will not become a standard process because someone say so, instead it will become the de facto standard process because a majority of the teams have tried and found the same set of ideas valuable and fit within the organisation. Here the project office plays a key role in facilitating and guiding this evolution so that the good ideas are spread and that the practises have a chance to converge instead of diverging.
The Shift
In traditional process management, the process is owned by the process office and followed by the practitioners.
In agile process management, the process is owned by the practitioners and facilitated by the process office.
Small mind shift in theory, huge difference in practice.
Yours
Dan
P.s. The situation of the Project Office in an agile environment resembles very much the situation of the architect who is unable to enforce architecture, but instead must work with the practitioners to explain and evolve it.
Categories: Discussion
Refactoring as Time Machine
Dear Junior
If I had a time machine, I would not need to worry very much on how to invest my retirement pension plan. I could just pick any random fund, and if that one turns bad, I could just travel back in time and pick one that will succeed instead.
Now, who said that time travel should be free? If nothing else, the power to run it should send your electricity bill to unusual heights. So each time travel will certainly come with a fee. However, if it enables me to "have made" a better choice of pension plan thirty years ago, it would probably be worth it.
The Existential Angst of Making Decisions
How wonderful would that be, to not have to spend effort and energy on finding what hopefully is the best, and then have to worry about having made the wrong decision.
Unfortunately, I have no time machine. And chances are I will not have one either when time comes for me to retire, about thirty years from now.
So, boring and uninteresting it might be, I have to spend time and energy on investigating different choices to make a good one. And live with the angst.
In System Development We also make Decisions
When developing software, I constantly have to make a lot of decisions, small or big. It might be just what to name a method, or it might be how to split the system in tiers; still decisions, decisions, decisions. And, all those choices affect how easy the system will be to work with in the future - picking the wrong choice, and I will have a hard time.
Sometimes I find that I have earlier made a design- or architecture decision that "seemed like a good idea at the moment", but now feels like crap. Perhaps I should have spent a little more time and effort thinking over all possible futures?
In System Development We Have a Time Machine
To my delight, I have found that in system development I have the time machine I lack in financial investments. Often some not-too-complicated refactorings can "take me back in time" and change that template method solution (that does not suite me well any longer) to a shiny state and strategy pattern (that does suit my current needs).
Of course, the refactoring time travel is not for free - there is a fee to be paid in the work I have to do and the testing to ensure it went well. However, that fee is much lower than paying the price of continue working with a code-base that is not well aligned with my best-idea-at-the-moment.
Obviously, I still have to spend enough time and energy to find a decent solution when making a choice. The difference is that I do not have to spend the enormous effort to cover all options. The possibility of making future refactorings simply shrews the economics of making a decision. Refactorings make it possible to settle for a good solution now, keep the speed up, and pay the small fee for changing your mind in the future. And I can skip the "angst" part.
In that way refactoring is a wonderful little time machine.
Yours Dan
Ps The ability to do refactorings in the future makes it possible to keep options open longer, thus deferring decision-commitment. This is what I see as the central message of the Real Options-movement.
If I had a time machine, I would not need to worry very much on how to invest my retirement pension plan. I could just pick any random fund, and if that one turns bad, I could just travel back in time and pick one that will succeed instead.
Now, who said that time travel should be free? If nothing else, the power to run it should send your electricity bill to unusual heights. So each time travel will certainly come with a fee. However, if it enables me to "have made" a better choice of pension plan thirty years ago, it would probably be worth it.
The Existential Angst of Making Decisions
How wonderful would that be, to not have to spend effort and energy on finding what hopefully is the best, and then have to worry about having made the wrong decision.
Unfortunately, I have no time machine. And chances are I will not have one either when time comes for me to retire, about thirty years from now.
So, boring and uninteresting it might be, I have to spend time and energy on investigating different choices to make a good one. And live with the angst.
In System Development We also make Decisions
When developing software, I constantly have to make a lot of decisions, small or big. It might be just what to name a method, or it might be how to split the system in tiers; still decisions, decisions, decisions. And, all those choices affect how easy the system will be to work with in the future - picking the wrong choice, and I will have a hard time.
Sometimes I find that I have earlier made a design- or architecture decision that "seemed like a good idea at the moment", but now feels like crap. Perhaps I should have spent a little more time and effort thinking over all possible futures?
In System Development We Have a Time Machine
To my delight, I have found that in system development I have the time machine I lack in financial investments. Often some not-too-complicated refactorings can "take me back in time" and change that template method solution (that does not suite me well any longer) to a shiny state and strategy pattern (that does suit my current needs).
Of course, the refactoring time travel is not for free - there is a fee to be paid in the work I have to do and the testing to ensure it went well. However, that fee is much lower than paying the price of continue working with a code-base that is not well aligned with my best-idea-at-the-moment.
Obviously, I still have to spend enough time and energy to find a decent solution when making a choice. The difference is that I do not have to spend the enormous effort to cover all options. The possibility of making future refactorings simply shrews the economics of making a decision. Refactorings make it possible to settle for a good solution now, keep the speed up, and pay the small fee for changing your mind in the future. And I can skip the "angst" part.
In that way refactoring is a wonderful little time machine.
Yours Dan
Ps The ability to do refactorings in the future makes it possible to keep options open longer, thus deferring decision-commitment. This is what I see as the central message of the Real Options-movement.
Categories: Discussion
Complexity Drives Effort
Dear Junior
There has been a recent discussion on story point estimation (or sizing) that is so common in Scrum. Especially the discussion has been about how sizes should relate to complexity and to effort needed to complete story. In a blog post Mike Cohn made a good example between effort and complexity by comparing "licking a thousand stamps" on one side, and "performing a simple brain surgery" on the other side. These where assumed to take approximately the same effort, but had drastically different complexity. Point was that they should have same amount of story points.
It is Effort that Makes a Difference
I think there is a lot of merit to the point Mike makes that it is amount of effort that is the usable parameter - we need it to make rough cost estimates and to predict what functionality can plausibly be released when. Complexity is not usable in the same way.
However, even if "effort" is what we primary care about, to me complexity plays a very central role - more than I have seen mentioned in the discussion. To clarify I would like to dive into the thousand-stamps-example.
Story is Effect, not Task
To start with, the team should not get a story saying: "lick a thousand stamps"; because that prescribes a specific solution. A good story should only point out the desirable effect, and leave to the team to pick a good solution. I guess the story could instead have read: "ensure a thousand stamps get licked", or something along that line. The difference is subtle but important. In the first case, the team is given a task to perform. In the second case, the team is given a problem to be solved.
Bulk Driving Effort
If you are to lick ten or a hundred stamps, you will certainly just do it. If you are to lick a thousand you will get mighty bored and tired - with ten seconds per stamp that amounts to three hours of work. However, it is still manageble, so you just get onto it. So here the effort will be driven by the bulk of the job - more stamps, more effort.
However, if you are to lick ten thousand stamps, or a million, then you are facing another magnitude of effort. To lick a million stamps would take 10 000 000 seconds, which is 70 weeks of full-time work - the better part of two years. Facing that task I would probably start building a stamp-licking machine.
Interesting enough this changes the game. The effort of building such a machine is independent of the amount of stamps to be licked. It is roughly as hard to build such a machine whether it will lick thousand or ten million stamps. So, bulk does no longer drive the effort.
Complexity Driving Effort
So, how hard is it to build a stamp-licking machine. Well, the actions involved is pretty straight forward, there are only one set of maneuvers, and no decisions to make - so, not-too-hard is a fair guess.
And here comes the other half of the interesting change of the game. The effort is decided by how hard it is to build such a machine - i e the complexity of the task. So, bulk does not drive effort any longer, instead it is the complexity of the task that drives the effort.
Software Development Teams do not do Bulk
Going back to software domain, the equivalent of licking lots of stamps (10 seconds each) is probably to send a pre-formatted mail to a long list of email addresses found in an excel sheet ("new mail", CnP mail content from template, CnP email from next line in sheet, "send").
A team of developers given the story "send mail to top thousand customers" would certainly not open Outlook and start at top of list. Instead they would write some script that generates a mail to mailer daemon given the to-addr - effort 15 min. Then they would write another script that loops over a text-file with email addresses and sends same mail to all of them using first script - another 15 min. Last, they would export the mail addresses from the excel sheet to a file, feed it to the looping script, and off we go - another 10 min. OK, let us allow for extensive testing and coffee break but we are still around two hours.
Complexity Drives Effort
The important point here is not whether the script-solution would be faster or not. The important point is that the effort needed is not determined by the size of the batch. The effort needed is determined by how hard the problem is to solve - how much time must be spent discussing design, how tricky it is to get right in coding, how much testing is needed to feel confidence in the solution. In other words, its complexity.
The effort which we need to put in to solve a problem in software development is very much a function of the complexity of the problem. The complexity can be essential to the problem, i e if we are working in a knowledge-intensive domain. This is of course the call for Domain-Driven Design. The complexity can also be accidental, e g if we have an over-complex architecture or a codebase in bad shape. This is of course the call for software craftsmanship.
Disregarding the reason, overcoming said complexity will be what decides whether we need to spend a lot of effort or not.
Sizing, or estimating, in story points should definitely reflect the amount of effort we think we need to put in. However, in most cases it is the underlying complexity - essential or accidental - that drives that effort.
Yours
Dan
Ps On of the reasons why complexity of problem is so central is that software development to a large degree is a knowledge journey.
Categories: Discussion
The Problem with XSS
Dear JuniorI guess that one of the first things you have to do to address security risks like Cross-Site Scripting (XSS) is to learn what they are about. Here I must admin that even though I have been interested in security, I have lived in ignorance about several of the most common attack techniques. OK, I had some vague idea what a XSS attack was like, but would not be able to explain it. Really I have to thank my colleagues Michael Boman and John Wilander for finally setting the details for me.Deep down XSS is really about making someone run a program that they did not intended to run. That program might be a harmless pop-up “You have been XSS”, it might be a local port-scan, or it might be installing a keylogger.The road to make this happen is to mix up the distinction between code and data. This is typically in a web browser which is fooled to accept something that looks like content (data), but actually is JavaScript (code) that gets executed in the browser.There are several ways to make a XSS happen but the one I have found most easy to understand is what is known as the “stored” attack.Imagine that you have a community site with some discussion forum on – say – how to raise children. Member parents can enter posts and comments to a discussion. When you go to the discussion page, all posts and comments are sent to you as an HTML page and rendered onto your screen by your browser. Let us imagine what happens if some evil pretend-to-be-parent enters a comment saying “I think you should <script>alert(‘Spank them!’)</script>be nice to kids”; what would happen? When you, the unsuspecting victim, browse the discussion you get back an HTML page. To start with, the page you get back is valid HTML – it is allowed to drop in a javascript almost wherever you want to. Your browser will render the text “I think you should be nice to kids”, and the script will be executed popping up an alert saying “Spank them!”. Of course, a yet more evil pretend-to-be-parent would substitute ‘alert’ for less pleasant, and now the computers of all unsuspecting parents are infected with keyloggers stealing their credit card numbers and uploading them to some obscure server in unknown jurisdiction.This variant is called “stored” because the attacking script is stored on the forum server. There are other variants for getting the javascript into the poor browser, but I have found this the easiest to understand.So, what is the problem? A simple way out is to blame it on bad indata validation. The forum comment “I think you should <script>alert(‘Spank them!’)</script>be nice to kids” should simply not have been accepted – and in this case that might be true. If it had been a discussion on JavaScripts, it might have been a little trickier.Another way to view the problem is that the browser has mixed up data and code. What was data (the forum comment) was interpreted as code and executed. No good. But that would be a case of blame-shifting, because we (the programmers developing the forum site) are perfectly aware about how browsers work.So really, the problem is in the forum application that mixes up domains. It takes what is a comment in the in-data domain (text field and http request), and happily let it go over to the presentation domain in which it is interpreted not as content, but as code. How come? Well – if we want the string “<script>alert(Spank them!)</script>” to be interpreted as content in a browser we must encode it properly. So, the long-and-winding answer is that we have missed to apply the proper encoding when transforming from the comment domain to the html-presentation domain.Now, here we have formulated XSS as a problem in Domain Driven Design terminology – then we can see what Domain Driven Design would do to address the problem.But that must be the subject of another letterYours Dan
Categories: Discussion
Joint Venture, Rather than Customer-Provider
Dear Junior
When a customer wants something from a provider, she places an order — and after a while there shows up a delivery. This works wonderfully well for a lot of situations. For example I needed a new laptop, so I went to the on-line Apple store. There I could configure how I wanted my MacBook (fast processor, no extra memory, and a solid state disk, please), and push the "order" button. About a week later there showed up a package at the office; enclosed was the computer - to specification. So the customer-provider system with order and delivery works in a lot of situations. Unfortunately, system development is not one of them.
The Customer-Provider Role-Play
Nevertheless, it is quite popular to let developers and business engage in a "role-play" where the business people are denoted the role of the Customer - often "because they pay". And as a contrast, the IT department take on the role as the Provider. I understand that these roles can come into play when business and IT actually are different companies.
However, it is really interesting to see this role play when both sides actually are departments within the same juridical entity. This makes the "they pay" argument kind of suspicious. In those situations we are not customer and provider. Rather we have chosen taken on those parts of the role-play, as a metaphor.
Hopefully we have made a careful analysis when deciding to use this metaphor. I think that in most situation, this is not the case.
Hopefully we have decided that this metaphor will help us to promote and guide valuable behaviour. I think that in most situations, it does not.
Customer Metaphor not Helpful
To me it seems there are two main reasons that the customer metaphor does not help us much.
First, the metaphor is so heavily loaded with associations and "wisdom" that our thoughts and behaviour is led astray. For example, we have all heard "the customer is always right". Unfortunately, this often creates a mental barrier to go and ask the business people.
On a similar tone, I have on several occasions heard programmers defend doing a bad job by "I only give the customer what they asked for" - when the proper thing to do would have been to stand up, walk away 25 meters and ask the business representative if the effect we have realised, but they probably not anticipated, really is intended or desirable.
Customer-Provider Misses Information Exchange Intensity
The second reason that the customer-provider metaphor does not help us is that I think it is fundamentally broken. If we study the information exchange in the metaphor, there is first one information transfer from the customer to the provider - i e the order. Then nothing happens, until delivery. Then there is a bulk of information going in the opposite direction - the delivery with all the details of the solution. In my experience, the information exchange of a highly successful system development initiative does not follow this pattern at all.
In my experience the key aspect of system development is knowledge generation. This journey is the joint endeavour where we combining the tactic ("able-to-do") domain knowledge of the domain experts with the expertise of creating mathematically precise logic systems of the programmers. In system development those two competences combine to create something new, that did not exist before. Together they create a precise understanding of a selected aspect of the problem at hand. In Domain-Driven Design terms, this distilled and precise domain expertise is captured in the domain model.
The information exchange when distilling the model is very intense, and is far from suitably described by one "order" followed later by one "delivery". I think the Domain-Driven Design focus on creating a shared, precise understanding of the problem domain very well captures the feeling of how much cooperation it takes.
Of course, we can "patch" the Customer-Provider by allowing (and encouraging) the provider to repeatedly ask the customer for clarifications and explanations. The other way around we can let the provider make delivery in parts - incrementally and iteratively show results en route. This is better, but I think it still misses a central point, the feeling of solving a problem together.
Customer-Provider Misses Mutual Initiative
Another way to view the problem with the customer-provider metaphor is to look at where the initiative lies at any given point of time. In the metaphor, the initiative is first at the customer. The customer will do whatever preparations needed to complete the order. During this process, the customer might interact with the provider, but the initiative to drive the process forward remains at the customer. This remains until the order is sent over.
After the order, the initiative goes to the provider, who will start working. At this point of time, the provider might reach out to the customer to discuss issues and ambiguities, however the initiative remains at the provider.
Finally, upon delivery, the initiative is once again transferred to the customer, who will verify the delivery, accept it, or file deficiency reports.
Characteristic for the metaphor is that at any given point, it is clear who has the initiative. During this time, the other part can comfortable sit back and rest assured that it is "the other's" responsibility to pick up the phone.
System Development Requires Mutual Proactive Involvement
In system development, I want both sides to be equivalently engaged and involved in the work all the time. Creating a strict model a la Domain Driven Design is a creative and collaborative work that requires both sides (business and IT) to feel the responsibility to take the initiative whenever they are struck with an idea (often late at night) and ensure that every new insight or idea is brought into the common work. At no time can either side lean back and rest assured that "I am not responsible at the moment".
End of Customer-Provider
So, can we please put the Customer-Provider metaphor behind us? It does not really help us in guiding our ideas, thoughts, or behaviour in any direction that is well aligned with the agile values of modern system development. If we absolutely need a metaphor, we must look for another one.
On a side note, Swedish agile front figure Marcus Ahnve pushes the idea that it is time for us as a line of industry to stop working by metaphors. Instead of saying "system development is like construction" or "is like product manufacturing" we should start talking about system development on its own merits. I agree that it is about time - actually my grandfather could have been a programmer, and I could have been the father of some young developer in the profession today. On a similar tone, Eric Evans has also mused on the subject under the tittle "Software is (Not) Like That". I second both of these thinkers that we need to be careful with metaphors, and we would be better of without metaphors that with bad ones.
Joint Venture as a Better Metaphor
However, if we necessarily must metaphors at all, and if we insist on picking them from the business world, then I would suggest another one. I would suggest one where the information exchange is more intense, where both sides have the initiative all the time and where booth sides care about the result they create in common. I would suggest using Joint Venture as a metaphor instead.
In a joint venture two parties decide that they should cooperate over some common interest and work towards some goal that is out of reach for either of them. A good example is the cooperation between Sony and Ericsson to create a mobile phone that was also a nice piece of consumer electronics.
Sony and Ericsson did not solve this by Sony putting in an order to Ericsson to build a mobile phone according to their consumer-electronics specification. Neither did Ericsson put in an order the other way around. Instead they started a new company, SonyEricsson, where each part owed 50% of the stock of shares. In this shared company, both parties invested by sending some of their best engineers, and both parties committed to supporting the joint venture so that it would bear fruit. Which, by the way it did. I think this situation better resembles the interactions and caring that we experience in a well functioning system development initiative.
Bear in mind; we still use "joint venture" just as a metaphor. If someone wants to actually do business this way I will be really interested to sit by and watch. But I will probably not line up to invest. However, used as a metaphor, I think joint-venture would be more helpful that customer-provider.
And being helpful is all I ask from a metaphor.
So, at Monday morning, we might as well stop referring to the business people as our Customers, and instead refer to them as our Partner. It might actually make a difference.
Yours
Dan
ps I did present a blitz presentation on this idea at the Agile Sverige 2010 conference at which the use of language and metaphors was a theme that emerged.
When a customer wants something from a provider, she places an order — and after a while there shows up a delivery. This works wonderfully well for a lot of situations. For example I needed a new laptop, so I went to the on-line Apple store. There I could configure how I wanted my MacBook (fast processor, no extra memory, and a solid state disk, please), and push the "order" button. About a week later there showed up a package at the office; enclosed was the computer - to specification. So the customer-provider system with order and delivery works in a lot of situations. Unfortunately, system development is not one of them.
The Customer-Provider Role-Play
Nevertheless, it is quite popular to let developers and business engage in a "role-play" where the business people are denoted the role of the Customer - often "because they pay". And as a contrast, the IT department take on the role as the Provider. I understand that these roles can come into play when business and IT actually are different companies.
However, it is really interesting to see this role play when both sides actually are departments within the same juridical entity. This makes the "they pay" argument kind of suspicious. In those situations we are not customer and provider. Rather we have chosen taken on those parts of the role-play, as a metaphor.
Hopefully we have made a careful analysis when deciding to use this metaphor. I think that in most situation, this is not the case.
Hopefully we have decided that this metaphor will help us to promote and guide valuable behaviour. I think that in most situations, it does not.
Customer Metaphor not Helpful
To me it seems there are two main reasons that the customer metaphor does not help us much.
First, the metaphor is so heavily loaded with associations and "wisdom" that our thoughts and behaviour is led astray. For example, we have all heard "the customer is always right". Unfortunately, this often creates a mental barrier to go and ask the business people.
On a similar tone, I have on several occasions heard programmers defend doing a bad job by "I only give the customer what they asked for" - when the proper thing to do would have been to stand up, walk away 25 meters and ask the business representative if the effect we have realised, but they probably not anticipated, really is intended or desirable.
Customer-Provider Misses Information Exchange Intensity
The second reason that the customer-provider metaphor does not help us is that I think it is fundamentally broken. If we study the information exchange in the metaphor, there is first one information transfer from the customer to the provider - i e the order. Then nothing happens, until delivery. Then there is a bulk of information going in the opposite direction - the delivery with all the details of the solution. In my experience, the information exchange of a highly successful system development initiative does not follow this pattern at all.
In my experience the key aspect of system development is knowledge generation. This journey is the joint endeavour where we combining the tactic ("able-to-do") domain knowledge of the domain experts with the expertise of creating mathematically precise logic systems of the programmers. In system development those two competences combine to create something new, that did not exist before. Together they create a precise understanding of a selected aspect of the problem at hand. In Domain-Driven Design terms, this distilled and precise domain expertise is captured in the domain model.
The information exchange when distilling the model is very intense, and is far from suitably described by one "order" followed later by one "delivery". I think the Domain-Driven Design focus on creating a shared, precise understanding of the problem domain very well captures the feeling of how much cooperation it takes.
Of course, we can "patch" the Customer-Provider by allowing (and encouraging) the provider to repeatedly ask the customer for clarifications and explanations. The other way around we can let the provider make delivery in parts - incrementally and iteratively show results en route. This is better, but I think it still misses a central point, the feeling of solving a problem together.
Customer-Provider Misses Mutual Initiative
Another way to view the problem with the customer-provider metaphor is to look at where the initiative lies at any given point of time. In the metaphor, the initiative is first at the customer. The customer will do whatever preparations needed to complete the order. During this process, the customer might interact with the provider, but the initiative to drive the process forward remains at the customer. This remains until the order is sent over.
After the order, the initiative goes to the provider, who will start working. At this point of time, the provider might reach out to the customer to discuss issues and ambiguities, however the initiative remains at the provider.
Finally, upon delivery, the initiative is once again transferred to the customer, who will verify the delivery, accept it, or file deficiency reports.
Characteristic for the metaphor is that at any given point, it is clear who has the initiative. During this time, the other part can comfortable sit back and rest assured that it is "the other's" responsibility to pick up the phone.
System Development Requires Mutual Proactive Involvement
In system development, I want both sides to be equivalently engaged and involved in the work all the time. Creating a strict model a la Domain Driven Design is a creative and collaborative work that requires both sides (business and IT) to feel the responsibility to take the initiative whenever they are struck with an idea (often late at night) and ensure that every new insight or idea is brought into the common work. At no time can either side lean back and rest assured that "I am not responsible at the moment".
End of Customer-Provider
So, can we please put the Customer-Provider metaphor behind us? It does not really help us in guiding our ideas, thoughts, or behaviour in any direction that is well aligned with the agile values of modern system development. If we absolutely need a metaphor, we must look for another one.
On a side note, Swedish agile front figure Marcus Ahnve pushes the idea that it is time for us as a line of industry to stop working by metaphors. Instead of saying "system development is like construction" or "is like product manufacturing" we should start talking about system development on its own merits. I agree that it is about time - actually my grandfather could have been a programmer, and I could have been the father of some young developer in the profession today. On a similar tone, Eric Evans has also mused on the subject under the tittle "Software is (Not) Like That". I second both of these thinkers that we need to be careful with metaphors, and we would be better of without metaphors that with bad ones.
Joint Venture as a Better Metaphor
However, if we necessarily must metaphors at all, and if we insist on picking them from the business world, then I would suggest another one. I would suggest one where the information exchange is more intense, where both sides have the initiative all the time and where booth sides care about the result they create in common. I would suggest using Joint Venture as a metaphor instead.
In a joint venture two parties decide that they should cooperate over some common interest and work towards some goal that is out of reach for either of them. A good example is the cooperation between Sony and Ericsson to create a mobile phone that was also a nice piece of consumer electronics.
Sony and Ericsson did not solve this by Sony putting in an order to Ericsson to build a mobile phone according to their consumer-electronics specification. Neither did Ericsson put in an order the other way around. Instead they started a new company, SonyEricsson, where each part owed 50% of the stock of shares. In this shared company, both parties invested by sending some of their best engineers, and both parties committed to supporting the joint venture so that it would bear fruit. Which, by the way it did. I think this situation better resembles the interactions and caring that we experience in a well functioning system development initiative.
Bear in mind; we still use "joint venture" just as a metaphor. If someone wants to actually do business this way I will be really interested to sit by and watch. But I will probably not line up to invest. However, used as a metaphor, I think joint-venture would be more helpful that customer-provider.
And being helpful is all I ask from a metaphor.
So, at Monday morning, we might as well stop referring to the business people as our Customers, and instead refer to them as our Partner. It might actually make a difference.
Yours
Dan
ps I did present a blitz presentation on this idea at the Agile Sverige 2010 conference at which the use of language and metaphors was a theme that emerged.
Categories: Discussion