Chapters 1 & 2

Silicon Valley Patterns Group

18-Nov-2003
Domain Driven Design (Evans), I.
Notes by Jeff Miller

This meeting revisits the final published form of a book that SVP reviewed in an earlier draft two years ago.

Russ: (introduces Eric Evans as author). The group spent 9 weeks 2 years ago with an early draft of this book. It was already good, and has matured since, so we should have high expectations. Anything to say?

Eric: I'd like to let it speak for itself. I found the group's input very valuable during the reading of the draft. It was very helpful to me. Now what I'd like to do is focus on what can you get out of the book and how can I give back to the group.

Russ: We have some faces from the Thursday group, now that the Thursday group has finished their track through the Gang of Four patterns book. I know that there is some interest in starting a reprise track on Thursdays. There are some other interesting potentials for future readings, including new books and books in progress. Maybe we can talk about that up front later.

Russ: So again, to open, remember we're focusing on getting information rather than providing help to the author. Do people have questions on the conversation at the beginning, the printed circuit board example?

Robert: It feels pretty representative in terms of the level of congruence between the software guys and the domain experts.

Eric: It seemed to me to convey... first, it was a domain that I knew nothing about, so it was an example where the software guy didn't know anything to speak of about the domain. It also had interesting behavior on the object side. I had another complete version of this with a different scenario, but some of the readers of the draft said that the objects were really a data structure, not too compelling as a first example.

(discussion of drawings schematically representing the domain, some are more domain specific in style, see page 41)

Eric: But eventually you want something that could represent the object model. From the early, domain-specific diagrams, I move toward UML. Domain experts can usually understand it without much trouble. But deciding whether something should be a class or a method, that is probably in the realm of the software developer rather than the domain expert. But the whole sequence of interactions should make sense to the domain expert.

Charlie: Do you always start with boxes, to UML-style things?

Eric: I jumped into the conversation at just about the point where I could start drawing boxes. But it also shows the way that we were communicating in a frustrating earlier period where we were trying to define common terms. So when I looked at all the use cases, they all mentioned nets. What's a net? It's something that connects several chips. So chips and nets became objects in the model. When the words and the pictures and the explanation all match up, for the simple cases that you can become confident about, and you can draw some simple UML models that the domain experts can understand, that's where you start. I stay away from the more complex UML, maybe as far as I go would be diamond for aggregation, or perhaps numbers on each sides of a line to describe how many instances might be associated. It's not enough at this point to build a system, but it's a step toward building a common language, partly visual, partly verbal.

Jeffrey B: So what you're saying...when you have a conversation, and you draw boxes, and have the domain expert confirm that how you describe them is how they would describe them...

(pause to finish ordering food, get coffee and salad)

Eric: So on the subject of Figure 1.1, I don't want to emphasize UML. When I sketched the boxes ... eventually the layout person found a font that looked really hand-drawn. I didn't want to emphasize the diagrams by showing them rendered in this crisp, sharp way. That's why I show them as if they are roughly sketched.

Tim: I've seen that people who have multiple names for the same things sometimes have trouble understanding UML. They make connections implicitly, and it takes a while to drag out that information. And they describe different connections under different preconditions.

Eric: This sounds like you were getting into more sophisticated questions about modeling, rather than about notation.

Tim: So I'd draw a representation, and they wouldn't understand that the diagram was trying to capture all the connections, all the information. They'd carry along their implicit knowledge of relationships.

Jeffrey B: People with a background in OO, they understand about why you might break things up, to separate responsibilities. It can confuse people who don't have this background of analysis and design about why you would break things up a certain way.

Jeff M: Q: to get a little phase-ist here, isn't this a question of the design phase rather than the analysis phase? You break things up when it becomes important to turn them into objects and collect groups of related responsibilities.

(Eric makes a gesture of warding off evil re "phase-based" software development)

Jeffrey B: I think that maybe the conversation here may go too quickly toward the design, rather than capturing information at a higher, summary level.

Russ: So when you're drawing at a whiteboard, are you introducing more words to the language, or are the domain experts?

Eric: I usually listen to them talk, and then draw and label boxes. It's rare that the domain expert will add something to the diagram. But even before the first diagram, there's a lot of talk. You never know where the initial discussions will take you, but it gives you a place to start. For the circuit board example, I learned early that a chip can connect to many nets, a net can connect to many chips.

Tracy: So maybe you have some concept about chips, and wires, and nets. But do you really understand what they mean by these terms that you think you have in common?

Eric: And in the other direction, there were cases where unfamiliar terms in the domain experts' vocabulary got turned into terms of more common use for the ubiquitous language. For example, they talked about "Ref-des" rather than
"component", which was the word that the common language later settled on.

Russ: How far down do you go in modeling?

Eric: As far as is useful. That's how you make a judgment on a model, if it makes hard things easier. There's always more than one way to model things. Occasionally there may be two models which are equally useful. Like a mathematical iteration, you don't necessarily have to start out your model in a good place. With any sort of reasonable starting place, you will converge, following an iterative path. And you may converge on a different valid solution, a different useful model, than you would from a different starting place. For example, if I had started out with the Signal as an object rather than the Net, I would have ended up in a reasonably useful place. If you get down to a point where you've gone a long way down the path, and the model is
not feeling like it's solving your problems, it's useful to pull back and ask if some of your initial assumptions were wrong.

Russ: So if you started with wires and endpoints?

Eric: I could have, but I imagine that I would pretty quickly find out that it was below the level of abstraction at which the printed-circuit designers are actually operating in creating their designs. Later on, talking about "deep modeling", with respect to the shipping model... you have cargos, ships, containers, places to pick them up and drop them off. But it ends up more useful, rather than thinking about ships, that thinking about a "voyage", about some cargo capacity available from point A to point B at a time, is the right way to think about planning and allocating cargo. And you find out that even though they may talk about a ship, the ship is not the thing they care about in making their plans. They could substitute a ship at the last moment.

Russ: So what would they say?

Eric: That would often come from their side, the domain expert would ask, "why are you fixating on particular ships? We care more about voyages... " If a ship doesn't usually have a meaningful identity to them, if it fills a role of "cargo capacity provider", then perhaps ships are secondary to the model which helps the domain person the most. Later in the book I spend time talking about how to /listen/ to the domain expert. There's a difference between a model and a language, but they are tightly bound, and it helps to listen very closely to how the domain experts would naturally like to express their concepts. If the domain experts are tripping over the language of the model, perhaps the model needs refinement. Or perhaps they're not quite familiar with how the model is expressing the concepts in the domain. I don't believe you should have an important abstraction in the domain part of your design that your domain experts don't understand. It comes up all the time because software developers are good at abstraction, often better than the domain experts.

Russ: I'm wondering if you have any rules for words that you might hear in describing the domain, or the design, that indicate a need to dig deeper. My "watchwords" for "needs more information": "thing", "stuff"

Charlie: "activity"

Tim: "manager" (an object which is a "cargo-manager", perhaps).

John: When you see their arms waving off into space, or in CRC card design, pointing to a blank card as an actor in the design.

Patrick: When is a noun used to describe the domain not an object in the model?
When it's too vague or abstract.

Russ: Or when it's over-concrete, like the wires in the printed circuit design.

Eric: I find that "nouns=objects, methods=verbs," is often a better way of explaining OO programming rather than a way of doing it. I would lay aside that whole thing.

Russ: Do you have rules-of-thumb for the early stage, for retaining meaty pieces of information that will make it into the eventual domain model? How do you keep yourself from going out too far into the woods on irrelevant detail?

Eric: Well, it's sort of a question of use cases. In the printed circuit board example, we started with a use case that ended up as a "nice-to-have". The objects and concepts that came out of it weren't all retained in the domain model created with the application. But it ended up being valuable in developing understanding of the model in depth. I would rather end up going down a path where we threw some stuff away, as long as we chose a use case that had some behavioral meat on it, not too far off the mainstream things to do, but related to the core objects of the system.

Tim: Would you ever pick something smaller to try and get the communication dynamic with the domain experts going? So, assuming you have some idea of the system, might you pick up a well-contained chunk that neither you nor they think is the most complex, but it would help to get into the give-and-take pattern of crunching knowledge?

Eric: It might in some cases be useful to start out off on the side, but I really advise people to start out close to the core domain. For this PCB example, the use case we chose really allowed us to explore the domain well, even if we didn't implement it as a story in the end.

Carlos: Do you delay putting things down on paper or on a whiteboard to avoid steering the conversation into premature concreteness, to avoid fixating on early things?

Eric: I haven't found this too much of a problem. Usually I can get people un-fixated. You don't want to do it too early, you need enough background to find a starting place. By the time I describe, entering the printed circuit conversation, we've had maybe a couple of days of discussing use cases.

Carlos: When do you start writing things down for the model?

Eric: Whenever it feels like it's time. But if it doesn't feel useful, you stop and go back to the use cases. There's a tremendous amount of trial and error in the process, brainstorming and experimenting. You'll have a lot more bad answers than good answers. You have to keep throwing up more models, more variations, more possibilities, and going round and round, back to the use cases if you need to ground yourself.

Charlie: For use cases, do you prefer them to be verbal descriptions? Or in some particular format?

Eric: I wouldn't be too prescriptive, a lot of things work. I'd rather just say that any form would work, as long as it's pretty concrete, really talking about what is the system doing for the user that is useful to the user, not just the user interacting with the UI. More like the Cockburn trimmed-down sort of use case. Not as high-level as, say, "book a cargo".

Jeff M: Sounds similar to Larry Constantine's "essential use cases", especially the focus on value to the user.

Russ: I like this phrase, "powerful new features emerge as corollaries to existing features" as a benefit of a good domain model. Will we read more about that?

Eric: I hope so. If you think about software as changing and evolving, I've seen several paths it can take. Either, it's typical, it changes quickly to start, then slows
down, then slows to a point at which it's considered "legacy" and continues to run, but changes little. Or sometimes the code starts slowly, but can accelerate in
development at a certain point.

John B: So is refactoring more about preventing the progress from slowing down, and deep modeling about enabling acceleration of progress?

Eric: Yes.

Shiva: How do you avoid analysis paralysis?

Eric: Don't separate analysis phase from design. Code up a prototype that you can demonstrate and play with to help discuss what was needed. If you get a wrong answer, then it helps the domain experts catch an issue, discuss it with you, clarify it.

Until you get through a full circle, use case, design, and prototype, you're not really firing on all cylinders. Analysis and design are really based on each other, fused together, which is another reason why it's important to have a running prototype early, based on the model.

(Ubiquitous Language, p. 24)

Russ: Do you have some thoughts about, to name a couple of examples, "toast and dumptrucks", programmers' metaphors that don't communicate with the domain experts.

Eric: Perhaps you could model the system that way. For example, you could model creating a shipping route as a parse tree, or an interpreter. But that's not part of the domain model. You could have a component which provides a routing service that uses these computer-science metaphors, and it may be the most useful way of handling the problem of constructing a route for a cargo. So when people talk about wacky metaphors, you may abstract a descriptive pattern or map of what the system is doing, but you have not captured the meaning of what it is doing. A metaphor or model defined in colorful terms can simplify your entry into a complex body of code, but it doesn't really help you see the purpose of the complex activity.

Robert: So... why do you think "regular" software development, often lacking a good domain model, works at all?

Eric: I don't know. Victorian machines certainly worked. They were complicated things, looms driven with programs punched on cards. The end result is whatever the machine is intended to produce, but if you all those moving parts, they have no meaning in isolation. That meaningless intricacy, the endless detail...it's what you might see in a movie to convey the bewildering feeling of being caught in the gears of a machine. Modern machinery tends to be simpler to the eye because the control has been extracted from the machinery.

Russ: So, to those of us who haven't worked in enough dysfunctional shops, maybe you could outline what you're contrasting it with?

Tim: I've got a few examples. With a few previous failures, management stops thinking of products as members of a product line... "just kick it out the door, we'll fix it later" and "oh, that one didn't work? Let's start over." No attempt at capturing or identifying the domain, or understanding the connections to the problem. We ended up having to add more special cases, more exceptions, more conditions. We never have the case where a new feature was, "aha, we can drop that in quickly, right there, there's a space for it!". The most successful managers at dysfunctional companies leave right before the inflection point.

John B: Mozilla is a good example. The Netscape code was running fast from 1994 to about 1997, and the code got more brittle and harder to work with over time. You could look at the CVS tree.

Eric: I'd contrast a domain-based approach with conditional logic...pages and pages of four-deep nested ifs, elses, whiles. No abstraction.

Eric: It's hard to recover once you've gone down that path too far. I recall with one client, there was a chunk of business logic. And then in the middle of all that was a subroutine that checked for dangerous Javascript. So I thought that at least they could move it out into a utility class. I looked to see if it was used. And I saw that it wasn't used at all. But the same method signature was implemented 4 times in different places, looking like copies of one another. The point is that once you've got a big system that's built like that, it's really hard to get out of it.

(some discussion of "who's left the dead code in there?", "who's made the copy?")

Charlie: In software I worked on, developed by a small team quickly, there ended up being the same fields, essentially, called by three different sets of names in the database, the domain layer, and the UI.

Eric: Even in dysfunctional groups, where there isn't a ubiquitous language, you end up having little groups each using their own terms, and they you have a couple of the best developers translating between the terms used by one group and those used by another -- the database, the UI, the other parts. It's easier just to translate than to try to bring the language back together.

Eric: Do I emphasize speech enough? I feel that the speech center of our brain is one of the best modeling tools. You can say, "a net connects chips". You can draw diagrams, but they are often hard to relate to what we can describe in speech.

Debbie: So even written language may not be a good substitute. In written language we tolerate more complexity, as we spend more time in writing and reading.

Eric: But it's not as fluid. In speech, we could go through a hundred variations of a model in an hour. You really need that. Tweaking the little pieces, changing the nature of a relationship, dropping an object, it makes things a little better. When I'm really working with a domain expert, I need to move quickly and fluidly.

Robert: So in a traditional project, you might have that cycle once a year.

Debbie: So if you don't have a real domain expert, or the domain expert is busy with their other responsibilities...they might want to send off their requirements.

Eric: You can do this to some extent with a set of developers, to talk with one another, to refine their understanding.

Carlos: Speaking exposes awkward terms in a way that writing doesn't.

Eric: Even reading out loud is enough like speaking that it picks up these things. I don't want to overstate these things, because a software model requires you to be much more structured, more restrictive than in a natural language.

Jeffrey B: I often feel like I wish I could roll back something I said in a discussion, to capture the words I just spoke which were the perfect way to express something.

Russ: We need to Tivo ourselves.

Eric: Like those people who wear cameras everywhere they go.

Russ: When you mention "pidgin", not sure why you're ringing it up here. Isn't the ubiquitous language a subset of the domain experts' language?

Eric: No! You're not trying to shield the domain experts from the process of development, since the model, the terms of the ubiquitous language, needs to be useful in designing a software system. The way domain experts would be happiest to describe a problem is not a way that you could use to program a solution for them. That's why I introduce pidgin as a metaphor for the ubiquitous language for modeling the domain and the application.

p. 34, Venn diagram of overlapping ovals, the center overlap is the realm of the ubiquitous language.

Russ: Is the pidgin in there? (pointing to the center)

Eric: Yes, there is some of that. And you do learn some things in the right side, the business terms and context that may not appear in the model.

Russ: What part of the technical side shows up in the overlap?

Eric: Objects. But the domain experts' world is probably not broken into objects in the same way a software model would break things into objects. For example, a "voyage" may be an object in the model, where they consider it something that
happens. And I didn't mean that you couldn't invent an abstraction that the domain experts hadn't already created. You might break a piece off one of their abstractions and re-cast it in a way useful for the system.

Russ: Though in terms of the vocabulary of the ubiquitous language, it sounds more like a subset of the terms used in the business arena rather than incorporating technical language.

Eric: You don't want to introduce purely technical concepts, but the way we break things down for software purposes shows up in the structure of the ubiquitous language. If you just brought a panel of shipping experts together, they could create a model of their industry, but it wouldn't be an /object model/. That's the technical aspect which shows up the most. There's a /thing/, and a /thing/, and a connection between them, and there are actions that can be taken on them.

Russ: So would a rule be something like that, a layout rule, a way of expressing what they're doing that the domain expert might not have thought of?

Eric: Yes, perhaps the overbooking rule. If you ask an expert, "is an overbooking policy a separable thing", they will answer "yes", but they won't have the OO point of view of thinking of it as the same order of concept as a ship or a voyage or a cargo.

Mark: People think about what happens, not necessarily breaking it down into the terms of objects and responsibilities.

Eric: You could apply probably half of the book in a non-OO context, though that is what I'm most familiar with. Getting knowledge from your domain experts, "grinding it down" until you have what you need for your model, these would be common factors.

Patrick: Does OO frame of mind bias you toward creating more objects than you might need?

Eric: Since everything is an object in OO development, you are forced to express everything as an object in one way or another.

Patrick: But does that impede communication with your domain experts? Having too many objects?

Eric: Then you may need to pare down your model.

Russ: Objects are a nice way of breaking down many problems.

John B: Not something that's primarily rules-based, for example.

Mark: Yes, people tend to think about business rules, usual flows of work and exceptional cases, it's hard to translate that into object-speak.

Eric: And mathematically oriented models are also not so naturally matched to objects.

John B: Or classic E-R relationships, in a database context.

Tracy: Eric, I think you and I have both have an experience like this. When you discover a new insight, and all of a sudden see that what you have isn't quite
matching what you really need for your model, what often happens is that the model collapses down into a finer, more perfect example. And a lot of objects just go away in the better-matched model. Russ and I had an experience with this happening. The new model was so simple that it was almost embarrassing to present, but if you looked at it, you wondered if you could have done it any other way.

Eric: If you find that you can compose your model components in flexible, powerful ways, often your application logic will get much, much simpler. Or you probably haven't got it. That's what you're going for. I relate model and language on a lot of different levels. Your application code is /saying things/ in terms of the model language. And when you really get it right, not that your domain experts will be able to read Java, but you could sit there with the domain experts, and translate it line-by-line into how they would express it in terms of the model.

Russ: A lot of people report that their domain experts can read their unit tests.

Eric: When you design for testability, you are also designing for something that is easy to manipulate, and easy to explain.

[Some discussion of the shipping example, the "overbooking allowance of 10%" re the code sample on p.19... when to change "*1.1" to put in an explaining variable for "1.1", or perhaps "overbooking allowance = 0.1, compare voyage capacity * (1+overbooking allowance) to the booked cargo size.]

[some discussion of 0.1 as a non-terminating binary fraction which might cause a rounding issue; discussion of accounting for pennies in a banking context]

Russ: I liked what you said about documentation. (p.37-40). About XP focusing on the running, functional aspects of code. Though there are aspects of code that can get out of date, like naming conventions.

Tracy: We had an experience where the domain experts had decided they needed to change a name to make it more clearly express what they meant. We had to translate for a while since the code had the old name, and it was uncomfortable.

Russ: And it was pretty tedious to change all of the names of the places it was referenced.

[...]

Eric: I feel that in Java, where you've already had to declare the type of something three times, if you can find a name for the variable reference that expresses something more, it's best to do so. Something like "nextDepartingShip" rather than aShip.

Jeff M: So it may be better to find a way to express a reference name as a role, perhaps, rather than anInstanceOfSomething.

(some return to naming discussion re overbooking allowance)

Debbie: The biggest missed opportunity I see for local variables is to name one "result". I often have to read back to the beginning. I prefer something like, say, matchingEmployee" rather than "result".

Charlie: I have conventions for naming return values.

Eric: Maybe we've drifted a little off the topic, but I feel it expresses something about this style of coding -- that naming instance variables within a method is tying it to the model, the ubiquitous language reaching down into the finest grain. That's why it's important, I think. In other modeling books, we'd be talking about high abstractions, and here we're talking about the details of practice that make things work.

Charlie: As I've worked, I've found it more and more important that you choose the best name.

(discussion of object-relational interface as a place where changes in the model may be slow to propagate, or the cost of change may be a deliberate choice; domain layer and UI is another possible interface where you may have to work to maintain a ubiquitous language)

(discussion of "watershed moments" in losing agility -- first, where you have a
database; then, when you have data you can't just throw away but have to migrate/ preserve)

(discussion of comments, commenting styles, limits of translating comments into
self-expressive code, usefulness of explanatory comments)

Charlie: There are some books we've read that seemed a little rough, but were too close to publication for our comments to make a lot of difference. It's really
gratifying to see how good this book is, in its final version.

Eric: I'm glad the work shows.

Tracy: I think the addition of the examples really adds a lot. And the explanation of the back-and-forth, so that we see you learning as you go. I think it's going to be a classic.

Jim: I've been reading and re-reading these first three chapters. This is great stuff, and if people really read it, they will make better programmers.

Attending:

Tim Huske
Carol Thistlethwaite
Patrick Manion
Walter Vannini
Wayne Vucenic
Mark Taylor
Robert Benson
Russ Rufer
Debbie Utley
Jeffrey Blake
Ken Scott-Hlebek
Carlos Sepulveda
Shiva JayaramanBatman/Tao
Tracy Bialik
Jeff Miller
Chris Lopez
Charlie Toland
John Brewer
Azad Bolour
Jim Sawyer

Chapters 3 & 4

25-Nov-2003
Domain Driven Design (Evans) II.
Notes by Jeff Miller

This meeting revisits the final published form of a book that SVP reviewed in an earlier draft two years ago.

(some comments and questions unattributed)

(notes start with discussion in progress)

(previous to this point, Eric compared and contrasted model-driven design with domain-driven design)

Eric: It depends on whether you demand that the classes in your model have meaning in the domain.

Russ: "bits of responsibililty"

Eric: It matters that the classes in the model... The reason some models give you that kick is that they are are close to the domain.

Robert: Is it easier to refactor the model, classes and objects, than services?

Russ: Let me do a quick check here. "Most talented developers don't have much interest in learning about the specific domain in which they are working" ? (p.4) How true is that?

Robert: Probably, let's say, at Sun Microsystems, your domain is technical stuff you're doing for someone else.

Eric: That's cheating. That's the exception that proves the rule. ("developers get interested in the domain when the domain is technical").

Russ: Let's exclude technical domains, then. What do people think?

Mark: Key word... I think "talented" is misplaced. I'd use the generalization without this word.

Eric: You may be partly defining "talented" as a person with enough insight to be interested in the domain.

Russ: And connecting "talented" with "successful".

Mark: People may not care about "widgets" in the product sense rather than the GUI sense. The only way you can care about it is to really learn it.

Eric: Often on larger projects there's an infrastructure team broken off... The most talented developers get to pick what team to be on. Where do they end up? The infrastructure team. They don't work on the account reconciliation feature. Instead they do fancy framework generalization about the UI or something.

[...]

Eric: I'd certainly like for talented developers to have interest in the domain. When they were working on the account reconciliation code, they did a better job than the person who replaced them. So part of what I'm hoping from domain driven design is that we could give the talented person a better way to think about the domain, a way that it can be interesting. I don't think the group here is representative.

Russ: Of talented developers?

(general laughter)

Eric: Present company excepted.

Chris: Technical knowledge is often more transferable than domain knowledge.

Eric: And the way domain-related development (notice I don't say domain-driven development) is done is often very boring. And when you add things to your resume... if you say you have mastered the specifics of the cargo handling operations at a port, it would just be clutter. You wouldn't put it on at all.

Chris: Yes, accounting, finance, there are a few generic, transferable domains that would count.

Eric: And does your peer group of other developers... they think you're really smart when you can do cool technical tricks. Do other developers think you're smart when you find a better way to reconcile accounts? No.

Tim: Interest...my personal interest sometimes in the complexity of a domain. Multi-state tax law for multinational companies. So if I tackle it as a complex problem, it's interesting, but if I got assigned to it, it's not clear at the outset that it's actually quite an involved problem. Now I'm working on music playlists, I'm a little more interested in the domain.

Eric: If people in some domain want a new application to do something more than create, read, edit, delete... It's like you're a 15th-century physicist.

Q: "Getting your head chopped off?"

Eric: Sometimes, perhaps. People may not support and understand your way of thinking. But think how exciting it could be to be among the first to see rhyme and reason in how things move. Before that, people would speculate on motion, and often they got it wrong.

John B: It's a very similar analogy, actually, to what often happens in software development. Speculation rather than testing.

Eric: We're still at that level with domain modeling with most domains. There are a few that are better done, say accounting. There are some powerful abstractions, a ledger, credit, debit, double-entry. It's well on its way. But most other domains are virgin territory.

Eric: What's missing... what's missing is the idea that domain modeling is a discipline. Saying "I'm good at diving into domains, finding a model, building software based on that." Being able to put that on your resume, having it mean something among your peers.

Chris: I've certainly seen contract positions advertised for people with skill doing domain modeling. They call them "Rational Rose Modelers" or "UML modelers for the domain."

Eric: I guess there's a silver lining in that cloud.

Russ: Are you sure it's silver?

John B: There's this meta-skill, then, domain modeling, that you can capture, otherwise you couldn't write a book on it.

Eric: Well, I guess _I_ couldn't write a book on it if it didn't exist.

Q: How do you tell if you have a good model for a domain, vs. a good architecture in the model. You were saying that there are a lot of things in the domain that aren't captured well by UML.

Eric: Perhaps it would be best to keep moving through the material to answer that question.

Eric: Sometimes, when the code is closely tied to the model. You might choose to move a method, to re-assign a responsibility. That can mean the model has changed; the model in your head, not necessarily a particular UML diagram.

Russ: You had a good example of having to create a larger-grained set of objects in the system to be able to work well with an object database, and that caused you to re-think the model and how you were going to deal with the domain.

Eric: In our original model, perhaps we might call it an analysis model, we had a large number of small objects in intricate relationships. It was a serious performance problem in connection with the database. So we needed to model the problem with fewer objects. There were 3 possible outcomes: (1) the same concept, but different implementations, rolling in subcomponents into the object which aggregates them, absorbing variant behavior; (2) a modified model, not as nice, but it works; (3) a new way of thinking about the problem that also works well as a way of breaking down the domain. We had some of each. But in each case, we said the way we think about the problem will be tied to the implementation. Maybe the model will not be as beautiful, but we do have to support it. There might be "teaching models" to tell people about the domain, but there's only one domain model that

Russ: Does anyone have a story about when there was an analysis model that was at variance with the code?

(1): I had this issue. There were often subtle problems with the model, things that were not implementable. We had to go back to the analysts again.

Tim: We didn't go back to the analysis phase.

Russ: And you'd already used that time?

Tim: Yes. We would have had to cut into QA time. And yes, it was a problem that the model was not really useful. Did the analysts get paid? Yes. It met their criteria. Measured against our phase, well, no. We had a notion of the model, but we were too busy coding workarounds.

Eric: Did you change your objects to reflect new concepts or stick with the original ones from the model?

Tim: Both. Sometimes we worked around it with a mapping layer, other times the model was loosely enough defined that we could implement it in a reasonably flexible way.

Eric: I'd like to distinguish between "UML", a representation of what someone thought was the model at some time, and "the model", which is how you are thinking about and communicating with the code.

Tim: An interesting part of this is that I'm sure the analysts had a model which was not the UML. But it was transcribed to UML and we just had to guess at what was missing.

Eric: UML just doesn't capture enough.

Russ: Isn't that what OCL is for?

(2): Usually you give people X and they are happy, occasionally people ask for X, you give them X, and they wanted Y. But when you hand it over, it does not provide people what they need to accomplish the task in the environment -- speed, memory use, usability -- not a conceptual mismatch, necessarily, but a poor implementation in terms of the working context.

Eric: Extreme Programming says, "just don't try to understand it all up front". So they have elaborate practices to stay customer-focused and deliver something useful. You can get something done, delivering feature-by-feature code. But part of the story is also agility, responsiveness to changing requirements. Partly it's things like test frameworks, refactoring, etc. to keep the _code_ quality high. But if you have also been using domain driven design, the internals of the software reflect what the customer cares about and wants to do, not just the external interface, the services the software provides. So you need a close relationship with a domain expert. This is not necessarily the customer who is choosing and assigning value to features, or the user who will be working about it. There is iteration in code, while in domain knowledge, there's what I call knowledge crunching, reflecting on what you have learned about the domain and evolving your model to a more representative and more useful state.

Eric: Probably that's true.

Russ: It goes further than that. By modeling the domain well, by coding that into the software, you also find that the next thing you haven't designed yet, you're more likely able to fit it into the existing software because of the way its structure reflects the structure of the domain.

Eric: XP has more things to guard your process against spinning your wheels and not making useful progress. You could apply a very straightforward, procedural approach to an iterative development and make good progress.

(2): You're talking about more about architecture, and I'm talking about features.

Eric: You'll notice I don't actually talk about architecture except toward the end of the book when I talk about architecture teams as a way to organize larger development efforts. But for a domain with complex requirements, where there are interrelationships among the various pieces, then model-driven, domain-driven design is important.

Keith: I've worked with code where there's a model, but it has kind of evaporated, it isn't visible in the code as you see it and work with it. Do you need, at least, a good object oriented language?

Eric: Well, code of that description is not a model-driven design. And how you get one is much of what this book is about. Especially part 2, and part 3 is about getting a good one. So modeling -- what your "Rational Rose Modeler" would be expected to do (looking at Chris), is the opposite of hands-on modeling.

Keith: In the case I'm thinking about, the team created a model and implemented it.

Eric: Does it show up in the code?

Keith: You can't really see it. It's sort of like a fragmented mirror or a Picasso painting.

Eric: Then it's not model-driven design.

John B: "Team = Code" ?

Keith : Conway's Law? The structure of the code reflects the structure of the organization?

Russ: I think you were a little harsh about not being able to do model-driven design in C. You can do object-like stuff if you don't need inheritance. We had structures of function pointers, we passed around a "this" pointer, and everything corresponded and felt much like you would feel programming in an object-oriented language.

Eric: I don't want to overstate it. But I think that choosing the right language is really powerful. It's harder to write, it's harder to read.

Russ: But I think you could read the code and see right back to the model we were working with.

Mark: It depends more on the programmer. I've seen awful code in C++ and really clean code in C.

Eric: I'm not trying to pick good and bad among languages. But when you want to do model-driven design...the model belongs to some style, and mostly these days the model is in terms of interacting objects. So choosing an object-oriented language gives you code which can be expressed in terms closer to the model than other languages can easily express. You need a programming language which supports your modeling paradigm, whether it's object-oriented, mathematical equations, or a chain of logical rules.

Eric: I'd certainly like to see a language that's better at supporting object-oriented design than Java. I have worked with Smalltalk, which I consider to be better, but it is not radically better, not in a different league.

Keith: Have you considered Scarlet? You write the algorithmic parts, the object and methods, in one part of the language, and separately describe how the objects are composed.

Eric: You could consider the current fashion for compiling a system out of UML and OCL as a programming language which is close to the model, but I don't think it's a good programming language. I'd have to see Scarlet to see what I think. I don't want to get into any religious wars about programming languages, I'd just like to point out that you want to choose the right tool for the job.

Eric: I try to stay away from the dichotomy between the "analysis model" and the "implementation model". Each one is useless in isolation, since the analysis model doesn't tell you whether the system actually obeys, say, a constraint you've noted in the model. This is not to say that all parts of the model are equally important, but that the core, meaningful, abstract concepts from the domain are expressed in the model that your implementation is based on.

Tim:

Eric: I advise people not to get attached to any particular model. There are always alternatives, or almost always. If you find that your original model does not work for building the system, then I take that as a sign for rethinking the model. So the shift I describe in the book -- from composed groups of little objects, to a stack of tokens and a parser which interpreted these as a little language. It was equally elegant, but perhaps less accessible to some developers. And it reflected a different model, a different way of thinking about and expressing the concepts from the domain.

Russ: Key points from the introduction?

Eric: One point is not so much about whether you have a good model (that's more part 3), it's about how to keep your model in sync with your development. If your "smart people" are doing nothing but modeling, but you don't implement the model in the terms it presents, then

(3): Don't you want your junior people in on modeling, so they can get mentoring and absorb skills?

Eric: I go further than that. I say everyone should be modeling, and everyone should be programming. If you aren't doing that, you aren't doing model-driven design. You have to do modeling as you program.

Russ: Let me say something heretical. What if you had a "thinker" and a "coder", and the coder goes back whenever there is a problem, and asks the model to be adjusted and fed back. Is that model-driven design? Clearly there are communication costs.

Eric: I guess that could be model-driven design. But if you adapt your implementation to be more do-able, you introduce a change in the model. I wouldn't advocate that. You want someone's eyes on the code who understands the model, and can recommend whether it's consistent with the model to tweak the code a bit for the sake of implementation, or whether there is some issue in the model which needs to be re-thought. Every time I've seen this approach succeed, the people who had created the model were also participating in coding it.

(1): Scaling of this approach?

Eric: Part 4 of the book discusses this. But there are also issues about how to partition the model into chunks which make good objects (Part 2).

Russ: Like the comparison between the elements of the model... and saying that they are not things that you try to identify like constellations in the night sky.

John: I see that when people try to reverse-engineer a system. They look at all the stars, and say, "well, there's a scorpion in there somewhere."

Russ: And you can extract the patterns with an automated tool.

Chris: (p.70) You seem to have defined the "application layer" very strictly. I might equate it to a "services layer". There is some business knowledge, often, in doing things like exposing tasks, in doing error handling. That amounts to business logic. Recovery strategies in the case of failures, perhaps.

Eric: I might not want to put those in the application layer. I talk about services in the domain layer, services which are fundamentally domain behavior, and others which are more application behavior.

Chris: I agree that there are two kinds of services. But I find that even in things, like services for aggregating network traffic, there is some business logic.

Eric: All right. Then there's a little workflow in there. But you want to push it back if you can. A lot of workflow is not really business related. Things like going from screen to screen in an application. Or having an application which constructs a new account in a particular way, but it doesn't necessarily matter that you do it in that way to the domain.

Chris: I'm thinking of metaservices, which call to services which exist in the domain. Have you worked with Struts or JFaces?

Eric: I've been on such a project, but I wasn't involved in that part of the application.

Chris: I wanted to ask about applications of Struts or JFaces which might be either in harmony with this approach, or at variance with it.

Eric: J2EE entity beans, I guess the recommended way to use them, really separates your data from your behavior. Does Struts carry baggage like that? I don't know.

Chris: What I wanted to see is if the recommended or common patterns for using Struts were problematic. As an example from a different framework, say, the .NET suggested practice of exposing RowSets all the way from your database up to your UI objects?

(discussion of Microsoft approach... view through to the database as a way of scaling through stateless servers)

Azad: And what you might call, not the "Smart UI", but the "Smart Database" pattern, controlling things with stored procedures. For certain applications, you can get quite a ways with this approach.

Eric: You could make it work in a model-driven way, but your modeling paradigm would have to be in terms of entities, relationships, triggers, etc. It strikes me as a rather masochistic way to do modeling.

Russ: That's kind of a lead-in to the Smart UI pattern.

Bob: First... having the domain objects responsible for persisting things. But unfortunately I've seen readers and writers all over the system, knowing bits and pieces about the serialized form, breaking encapsulation. I wanted to check in about where you were going. Perhaps "what" should be persisted should belong to the domain object, and "how" should belong to the implementation?

John B: I think that is Chapter 6.

Eric: Yes. Let me just refocus on the goal. When you're looking at the domain object, you can think about its domain meaning, not about the implementation details of how it talks to the database. If you have a framework that lets you disentangle the domain meaning and behavior with the implementation.

Bob: I've used Aspects a lot lately, to isolate things like persistence.

Eric: I've dipped my toe into this, and I thought it was neat. But I'd just like to focus on whether the persistence framework helps you read the code in terms of its domain nature (or at least gets out of the way), or whether it does not.

Russ: I was curious to know about how aspects and refactoring interact in practice. How well do they work together. Bob, have you been using Aspects in a serious way?

Bob: More playing at this point, but I can give you a pointer to a blog from a fellow at Microsoft, an entry something like "A Year With Aspects", where he talks about the difficulties with multiple aspects, dependencies and duplications, after working with AOP for a while.

Azad: How does the model interact with documentation?

Eric: If you start with the core notion that the model is a set of concepts that you want to be able to communicate, to understand, to use in talking about the domain and the software implementation, then any way of documenting that is compatible with that is reasonable.

Azad: Or if you were dealing with something that was more heavily rules-based. Or when your domain experts are expressing the domain in different terms than you find useful for implementation.

Eric: I'd recommend using the same style for capturing domain knowledge from the experts and for designing and implementing the system.

Azad: No question about that.

Eric: Or have a rules engine and integrate that into the system. But it may be worse than modeling their rules as Java objects. You still need a ubiquitous language which you can communicate in terms of the domain, in terms of the design, and the implementation.

Robert: Perhaps in a lot of domains, there are a lot of little surface rules in the processes you go through. But in order to build things that will go a longer way, I feel there's a danger in not trying to discover the objects.

Eric: The concepts of the domain.

Eric: It's hard to carry two models with a mapping between them. The mapping is itself a modeling problem.

(4): Or in some languages, being able to overlay knowledge about how to connect things, to expose different ways of

Eric: So you're introducing candidate abstractions into the language, in the terms of the domain?

Russ: So if your domain experts are not communicating in terms which map very closely to the system you are building, you've lost a lot of their expertise.

Azad: So I think that we disagree on one little point.

Eric: "Never do it" on my side, vs. "Avoid it." But when I expressed it that way in the book, the reviewers recommended that I remove all the caveats and reservations. I really do believe that whenever there is one model for talking to domain experts and another model for programming, then we are really missing the boat. We won't get the agility in the design.

Azad: Partly what I was trying to express was that in the currently popular programming language, whatever that might be, there may be some concepts that are key to the domain that aren't easily mapped or modeled.

Mark: Something I wanted to ask that isn't in the book so explicitly ... trying to get the developers to buy in, having the whole team involved ... but what about management? How do you change the culture of management to allow teams of developers to do these things?

Eric: Managers are very buzzword-driven. If they are interested in the buzzword (today maybe it's XP, tomorrow maybe Domain Driven Design), you'll be more likely to be able to do it.

Robert: Managers also pay attention to success as well as buzzwords. You try and make strong connections between this approach and successful projects.

Eric: On the web site, there's actually something called "Managers' Guided Tour To Domain Driven Design"

Mark: Is it in PowerPoint format?

Russ: A good lead-in to Smart UI. You really need forward-looking management to take on model-driven development with nice testing and all that. Smart UI gives you a quick burst of productivity up to a certain point, then you run into a point where you can no longer scale it. Model-driven and domain-driven... you start with something kind of linear, lose a little from complexity, gain some from the set of strong, interacting abstractions. It takes a little while for the more advanced techniques to really bring home their benefits.

Eric: I do say that MDD is more important for cases where the domain is complex.

Bob: It's robustness. YAGNI works if you don't have to go back and touch it. MDD is about robustness. A small change in the problem domain is a small change in the objects and classes.

John B: XP's "system metaphor" speaks to this.

Eric: YAGNI applies a lot in model driven design, as well. For analysis modeling, you don't have something that tells you when you can stop.

Eric: Do the _simplest_ thing that can possibly work... it might be really hard. Thinking profoundly in order to express the problem most elegantly and most simply. It's really a profound breakthrough.

John B: Kent Beck expressed this as a question: "What is the simplest thing that can possibly work? Why aren't you doing that?" rather than simply "Do the simplest thing that could possibly work." There's a level of reflection there that's missing without the questions.

Eric: Especially when you know you'll have a version 2, you need a robust way of developing software like model-driven design. There may be simple ways to solve simple problems where you may not have to use such techniques. If your problem fits into a pre-existing tool, get it off the shelf.

John B: I often start with a smart UI. I start in domains where the UI is the biggest risk. Then I factor the domain objects out of the UI as time goes on. I'm working on a graphical interface to a text-based conferencing system, and I can see there are a set of objects wanting to come out. But the next several stories don't call for that to happen.

Russ: And then, when you've learned what you can from the first prototype with the smart UI, you have a choice between factoring out the domain objects, and retiring the prototype and re-building it.

Eric: You may have different experience from I about how easy it is to refactor out of a smart UI. My experience has been that when I step into a client's place where there isn't some sort of layered architecture present, then I often will say that I do not know how to help them. I've had a lot of trouble with escaping from a bad design.

(discussion of experiences factoring out of a smart UI: John B, Russ, Jeff M)

Eric: Very early in the history of a system, especially if you are probing for key risks, you can get away with a "Smart UI". But if you plan your system to have any sort of a life in development, you should plan to have a domain layer.

Bob: How about non-layered architectures. Pipes and Filters? Blackboard?

Russ: Composite?

John B: Composite sounds like the structure of the domain layer.

Russ: Are we the only people who have made a success out of a non-layered architecture? (Russ describes a system that separates I/O and other implementation concerns in specific service classes, but not separate layers).

Eric: The four-layer architecture diagram I present is probably too specific. The essence is isolating the domain. Have a domain layer representing the software constructs relevant to business logic. It sounds like what you describe is not a layered architecture, but it is a separation of the domain. It's too bad it comes across as so much of a specific prescription.

Bob: For domain-driven business applications, they pretty much do look a lot like this layered architecture you present.

Attending

(partial list only)

Don Chin
Vince Nibler
Tim Huske
Mark Taylor
Robert Benson
Russ Rufer
Jeffrey Blake
Ken Scott-Hlebek
Carlos McEvilly
Batman/Tao
Tracy Bialik
Jeff Miller
Chris Lopez
John Brewer
Azad Bolour
Keith Ray
Bob Evans
Walter Vannini

(and several more, total 23)

Part III Intro & Chapter 8

December 2003
Notes 9, 4th Session
Guest Author: Eric Evans

Part III introduction

Robert: The 3 issues on p.188, the qualifications for doing domain-driven development

(mention of OO databases)

Eric: [use of OO databases is rare] Because there aren't so many people using rich domain models.

Tracy: Isn't there also a comfort level with relational databases?

Robert: And ad-hoc reporting, etc.

John B: This is discussed in Crossing The Chasm, etc. People only change technologies at gunpoint, figuratively speaking. But when there's a shift, often the change is permanent (e.g. CodeWarrior beating LightSpeed C).

Tim: Very few people believe the #2 issue, willingness to evolve and change the model.

Eric: I really thought about this list very hard. People might have a domain model, and the willingness to evolve and change it, but they didn't have the knowhow for #3, the design skills. Or sometimes they would isolate the designers from the coders.

[some notes lost to editor program failure]

(question about deep models)?

Russ: We worked on an autodialer where we made effective use of a deep model of the problem domain. The customer was happy to find out that our solution had lots of areas of flexibility where they were able to ask for new features that they had not thought of before.

Tracy: We had another project where we had solved a problem with respect to arranging some resources in memory blocks. The customer gave us a new variation of the problem where they thought it would be a difficult elaboration of the previous version.

Russ: But it fell out naturally from the solution of the specific problem we had solved previously.

Jeff M: Was this a question of a "deep model" or more a question of solving a general case and being able to then handle specific cases more easily?

Russ: It was specific to the problem domain, and their "difficult elaboration" was one that made a lot of sense in terms of the original problem.

Jeff M: Close in concept space, then.

Russ: Yes.

Eric: Once you've experienced it, it seems real. It's the fact that this occurs so rarely that is part of the reason I wrote this book.

(some additional discussion, mention of O-R mapping, "well-worn glove" metaphor of an effectively evolved domain model)

Chapter 8, Breakthrough

Eric: O-R mapping means you have to change a translation layer when you change the model. It means you have to carry more along as you refine the model, as you make changes to reflect deeper insights. Or when you refactor away from a code smell. You still come to points where you make a breakthrough, it's just slower.

(show of thumbs, how many people have 1,"seen a deep model?" 2, had a "breakthrough" from better understanding a model")

Eric: Ah, interestingly more people have had the "breakthrough" experience than the deep model.

Russ: Who broke through and didn't get to a deep model?

Tim: In your own little parts which you have freedom to change, you can have significant breakthroughs, which don't affect the model of the system at large. Deep but narrow, perhaps. A small enough subsystem that I don't consider it the systems model.

Chris: Or if you start with a bad enough model, getting to a mediocre one is quite a breakthrough.

Eric: And I think I've experienced multiple breakthroughs in the same project.

Russ: (Looking at a chapter head chart, re the breakthrough pattern) I don't really get what this chart is trying to tell us. So this is time marching along...

Eric: As you refactor again and again, and then you have little jumps of value, and at some point, voom! You get a big jump in value.

Tracy: So maybe the vertical line should be a little diagonal?

Eric: It's at the granulation of significant steps. Imagine if the horizontal axis is a year. So how wide would a 3-week change-over be?

Russ: So this is potential value vs. value delivered?

Eric: You're trying to read a bit too much into this picture. What I'm trying to express is that progress in refactoring, in what you get out of it, is like punctuated equilibrium. You have a steady practice which takes you to a point at which things change suddenly. On really successful products. If the refactoring is just keeping you out of total chaos, you just keep going and going and don't get big jumps. But when you're improving the system, you get little jumps, little improvements, and then occasional big jumps. The axes are kind of vague since it's trying to express things you experience. Things that may be difficult to measure.

Mark: So the length of the jump? The height is the amount of code you change?

Eric: No, more the horizontal axis. The horizontal axis is either time, or amount of refactoring. If you're keeping up, they're the same thing.

Mark: I've often seen that in a breakthrough, you can throw away a lot of classes and methods. The jump is in the amount of code that changes, in my experience.

Eric: What I'm trying to express is the value of the code. The flexibility, the ability to do more and better things.

(comment): You could express this as shortening the arcs, reducing the cost, the distance to some new things you could get to. New features.

John: If you take the derivative of Eric's graph, you can tell what the expression on the manager's face is.

Russ: Let me talk out of both sides of my mouth now. I think I disagree with the notion that all programming days are the same. Productivity-wise. When you're really where you want to be with the model, you can accelerate in how much code you can write. Whereas when you are trying to figure things out, you're a lot slower. When this doesn't come early in working with the code, I feel you're more in the realm of re-engineering then re-factoring. There are no small steps that get you out of the hole. Now we're in the area where hopefully there's some piece on the outside that you don't have to change, but the stuff on the inside has to be radically changed around.

Eric: I think all these things can happen. I don't like to get too hung up on the details. I call it refactoring because it has the same definition -- changing the design without changing the behavior of the system. It doesn't say how big that change is. It's good practice to make these steps small and continuous. But sometimes you have to take a leap across a big chasm. In a story I have later, fixing the core stuff was quick, but fixing the stuff that depended on it took quite a bit longer. Nowadays I might be able to figure out how to break that design shift into smaller pieces. But three weeks was small in terms of the total project.

Russ: For other people who have had the breakthrough experience?

Chris: For us, I've seen it where "everyone gets Friday off except you, you, and me." Then the change happens over the weekend, and everyone picks up the new architecture on Monday.

Eric: There was a point at which a team I was working on got rid of the EJBs in their system. They'd been whittling them down over time, and at the last point, they took their last few EJBs and ripped them out over a weekend. Sometimes you have to say hands off the code.

Chris: I find that that tends to be a property of the "aha" story.

John B: Or if we just move one thing, suddenly everything else becomes much easier.

Chris: I tend to think of those as "d'oh"s rather than ahas.

Tim: Sometimes you get in a stuck position where you end up getting told, "Okay, you're not making progress, just stop and think." Then sometimes when you get an insight, you don't have time to work out all the advantages by the time you have to ship.

(Chris, Eric: discussion about software architects resisting a change that simplified the model and made the architects' job "too small")

John B: When you ask the question, "Why is this code that way", and the answer is someone's name. That seems to be a recurring pattern of dysfunction.

John B: Ward Cunningham's talk, his metaphor about waiting for insight. You don't have to wait, but if you don't it won't come. And when you get an insight, it can be like climbing a mountain. Going down the other side is working out the implications of your advance in understanding on the rest of the system. You can abort a climb, you can climb to the top and plant a flag, or you can carry out the whole thing, climbing up and carrying it down.

Eric: Businesses keep an eye to new opportunities constantly. They don't expect that they can predict things, that the environment will remain stable. They look for new places where they can gain advantage, new markets, new ways of simplifying things. You don't see this search for opportunities in software projects, and it's rare you get the serendipity you can get from just keeping your eyes open.

Eric: One of the things I worry about, on the highly iterative methods of software development, especially frequent releases. It's hard to stay agile if you have a large installed base.

Tracy: The Poppendiecks' book -- Lean Software Development. They talk a lot about experimentation. The right amount of experimentation is when you succeed half the time. You learn something from your successes, but you learn a lot more from your failures.

Carol: Detail-oriented people, are they different from ambitious, entrepreneurial types?

Eric: The people who do manage to set up their organizations in the right balance, who allow upside potential, they will leave the others behind.

Robert: OK, you talk about sort of a portfolio approach in business, trying out a lot of business opportunities. But there's a difference between software development and business opportunities.

John B: 3M does this with atoms, even. They've institutionalized a version of what's called "playtime" in Tom DeMarco's work. When one company DeMarco described looked at their successes, all their best-selling projects all came out of playtime.

Eric: And software development would be more fun AND more productive. In squeezing out the fun, they've squeezed out their unexpected opportunities. There are only unexpected bad things.

Robert: Evolution from a "big up-front specification and design culture".

Russ: The large corporate culture of software really followed a grass-roots, individual level orientation at first.

John B: Everybody knew this who was any good. That you couldn't anticipate everything up front. That you should go through the whole cycle of the project in miniature, early on. That you should deliver something part-way through as a sanity check.

(comment): do people just have the wrong impression about software? That it's like architecture or manufacturing, large fixed costs with a heavy process?

John B: Software is nothing but metaphor. What do we have here (indicates Apple iBook), a chunk of LCDs and a backlight?

Russ: Is software like a magazine?

Mark: Like publishing? You can cut a few things out, the quality will suffer, but you can still read the article.

Russ: Agile software methodologies beg to differ.

Eric: A literary journal?

John B: Or a math journal? Where everything is a proof?

...

John B: And the bad metaphors for software are based on misunderstandings of the other fields. Even manufacturing isn't like manufacturing, as we learned from reading Lean Software Development.

Eric: Let's take a break from the metaphors for a while. Just do software development for a while, without saying it's like anything, and see where it takes us.

Carol: Why do people keep picking terms from other fields? Programmers, analysts, engineers came in at some point.

Eric: For me, writing this book was like programming.

Jeff M: Did you have a breakthrough?

Eric: Yes, I did. Multiple breakthroughs. There was a point where I realized I had to reverse the order of all the pages.

(wide ranging discussion of metaphors)

Chris: Shouldn't you write down the list of the top 10 things you hate about software development? These things need breakthroughs.

Eric: Companies want to produce software with unskilled labor.

Russ: I was wondering, for people who have had these breakthroughs. Can you remember anything beforehand that led to the breakthrough? I can remember sitting and thinking, "It just can't be this hard." So there was one time I recall looking at two sets of client specifications for two different projects, thinking "these are too similar to be different."

Bob: I had a time when I was working on a project and that everything seemed much too hard. So what I did was to step away from the framework I was using and started by just sketching out classes. It was a case of constantly running into something, stepping back, and then bringing back my insight back into the system.

Chris: I've seen that "share pie" example from the other side. ("share pie": referring to dividing up the moneys from a syndicated loan as a somewhat complex domain issue) The domain experts think of everything as special cases, and when you put them together, you find out that they are just slight variations of a strong central theme, and the domain experts are startled.

John B: Duplication in the code that you can't quite get at, the way it's organized. You can tell they are doing the same thing, but there's no way to put them together.

Eric: You think there's a deeper model in there?

John B: Usually I think that it's very annoying. And when I do little refactorings, reducing duplication, bit by bit, finally I often realize that there's a way to fix the problem by introducing a new object, perhaps.

Chris: It's turned out in my experience that sometimes the domain experts don't themselves have the best understanding of the model. Sometimes there are simplifications and unifications that can be made.

Eric: I'd draw a distinction between an expert who knows his field but does not have an abstract thinking talent, versus someone who is only borderline competent in the domain... I would work with someone who knows things in detail but doesn't have a clear abstract view, but I would try to avoid working with someone who doesn't have a firm grasp on the field. You lose a lot by not having good give-and-take with someone. Sometimes I turn to books about the field.

Russ: Has anyone (p.206-7) seen a better explanation than the brief discussion here, of the nouns-as-objects vs. not discussion?

("hmmms" from around the table)

Eric: Chapter 9 has something of a discussion for dealing with the case where the local domain expert is not so available, or not so useful to you. Sometimes you have to become a middling expert yourself. But it's a lot harder.

Tao: We're about halfway through this book. From Chapter 1-9. It sort of dawned on me. I used to be in manufacturing for a while. To be a domain expert, you have to do it yourself. You have to go down to the shop floor, to see the products, to find out for yourself how things are made. Was it worth it, for the shipping example, say, for a developer to try booking a cargo, or to sit beside the people who book the cargo?

Eric: The people who book cargo don't have a broad understanding. You could learn what you needed from them in a few hours. I went to a port and saw some port operations. It was valuable. I want to draw the difference between a domain expert and a user. The user's point of view is what you get from sitting and booking cargos.

Chris: You'll learn how it has been done, which is not necessarily how it should be done.

Eric: I most like to work with a domain expert who has some ideas for new ways to do things.

Chris: These seem to be on the endangered species list.

Darlene: How do you deal with a domain expert who is making mistakes in what he tells you?

Eric: Most of the things in this book assume that what the domain expert says is basically true. That he's not making mistakes.

John B: Although you would catch inconsistencies pretty quickly.

Eric: Or you would look at your model, your design, and ask about where there are seeming contradictions or difficult points in the design. So you might go read a book to see if you could get another opinion, another angle on things. You might hear contradictory things from two domain experts that you could resolve in this way. Or you might have to go back and ask for clarification in the light of each others' assertions. So you'd hear, "Oh yeah, I made a little mistake there, it's supposed to be like that." Or you'd hear that you were missing some context. Or occasionally you would get a sense that one is more reliable thn the other.

John B: It's like a Columbo episode. Eventually he'd tap his forehead and say, "There's just one thing bothering me..."

Eric: Software development is like a Columbo episode?

Chris: Refactoring is like a Columbo episode?

Tim: How often do people end up having to deal with "users" rather than "experts"? So sometimes we have to reverse-engineer the domain out of a usage pattern from a previous generation of application.

Russ: We've more often had the experience of building the first one of a thing, so we as developers are considered as expert as anyone else on the team. It's a pleasant way to work.

Tim: So sometimes we've had to lead with a user interface. We walk through them with the UI, and find out what we got wrong in our understanding of what they wanted.

Eric: So, to do this, to do domain-driven design, you really need a domain expert. I don't say who it is. But not all domains are so approachable that a software developer can be their own domain expert.

(reference to a glossary entry on "domain expert")

Eric: The printed-circuit board design case, the example in the beginning of the book. The domain experts were users. But many of the users were narrower in their understanding of the domain. The people I was working with had a somewhat larger vision, that the right tool could help guide other users toward good practices in how they do their work in PCB design. It reminds me a little of our earlier discussion, a previous session, about writing OO code in a non-OO language.

Jeff M: What would you want first? Maybe long identifiers?

(laughter)

Jeff M: Then polymorphic dispatch?

Russ: Encapsulation first.

...

Tracy: You can go through 100 ideas in an hour and a half at the whiteboard, at the level of concepts. If you tried to write up 100 ideas, how long would that take to run through the trial-and-errorl

Eric: Not only would I be at the whiteboard, but I would be speaking out loud, that I would get the immediate feedback to move quickly.

Tracy: And the idea that it's OK to float 100 ideas, that that's a good thing, that you can go through a lot of ideas to converge on a good one.

comment: One of the things that takes a lot of the fun out of programming is that if the iterations are too slow, you don't get the reward of the feedback, the sense of progress.

Eric: You can end up in a local minimum through iterative development. Then you might deliberately destabilize the situation, do a few things to see if there's a better place around.

(Russ describes annealing in metallurgy for strengthening metal, and simulated annealing in numerical optimization)

...

Russ: (smile) Now we know that we aren't getting enough reuse in software because this bucket doesn't show up later when we're mixing paint.

Eric: Did anyone get into the Specification pattern?

Russ: We're supposed to read into Chapter 10.

Tracy: We're ambitious.

Eric: Specification is supposed to serve as a case study...

Russ: Do you think you might run into a case where the situation would be too complicated to be satisfied by the simple Specification you describe?

Russ: Next week's homework: Come in with an inappropriate metaphor for software development.

(some discussion of the Wiki on domaindrivendesign.org and SVP's wiki).

Russ: Who's used this? John, raise your hand. ("Constraint")

(looking at the Warehouse Packer prototype)

Eric: Sometimes you get into a design logjam, where everyone is waiting for everyone. Just any implementation of one of the subsystems allows everyone to move forward. Even a dumb implementation of the piece that fulfilled the contract gets things going.

comment: We had a teaching software implementation where the "intelligent" part of the interface started out as a random number generator. And it was surprising how long people thought there was something more intelligent and complex was behind it.

Russ: We've found out for things like the prototype that it can be hard to move from the simple prototype, beyond to the more complete system. You need more global organization and structures to support where you need to end up.

Eric: Obviously a lot of optimization problems require global reasoning. That just makes sense. Still, the analysis that goes into doing a thing like having

Chris: (to Russ) When you needed to back out the constraint code to optimize the packing, did you lose the ability to use the constraints as pure validation? "Does this solution satisfy...?"

Eric: I think he means, how do you know you're answer is right?

Chris: Yes, the basis for your tests.

Tim: Some problems are so hard that the ability to prove it optimal is the same level of difficulty as being able to create the optimal solution.

John: What you'd do is set some sort of floor.

Eric: Would you have been able to go directly to the final, optimized solution? Or would the prototype be useful for clarifying the structure of the system?

...

Russ: We had a prototype Builder, a complex but low-risk piece of work in full, where the prototype handed back one of three representative structures of the types of things we wanted to work on. Having the prototype helped the rest of the system take shape.

Tim: It helps for more than just the first pass to break the logjam.

Russ: The prototype builder actually lasted much longer than we expected.

Tim: We had a brute-force temporary optimization module that was just supposed to keep us developing until we signed the contract with the optimization engine company. Which never actually happened.

Yahoo! Groups Links

To visit your group on the web, go to: http://groups.yahoo.com/group/siliconvalleypatterns/

To unsubscribe from this group, send an email to: siliconvalleypatterns-unsubscribe@yahoogroups.com

Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/

Chapters 10 & 13

Domain Driven Design (VI) + Industry Roundtable
Guest Author: Eric Evans
Notes by John Brewer and Jeff Miller

(some discussion of John Corbett's picture discussed on the list representing evolution of a software system and easy/hard features within reach)

(lengthy roundtable discussion, omitted, of interesting new trends in software, eventually rolling around to outsourcing)

Russ: It seems like a lot of jobs that are still high-skill are apt to be commoditized in the next year or two. A lot of the web design tasks, for example. There are easy-to-use tools to put together a site, even with a shopping cart.

Eric: It seems like a lot of companies try to ship the wrong things offshore. The domain business logic, which is actually one of the worst parts.

??: So they're thinking it's up front design, they'll just ship off the spec.

Eric: But it doesn't work.

John: For big companies, they have a branch over there, not quite like outsourcing.

??: A new thing, and widely perceived to be successful so far. Not so many horror stories at this point.

Eric: Programming as unskilled labor? It's certainly some of it is more semi-skilled than it used to be. Perhaps building a web page is rather less technical than it was. So that's one thing. Partitioning jobs that are less technical from ones that are more so. But it seems to me that the domain layer is the last thing that will actually be automated. Then there's the process. You can imagine an offshore shop full of smart people using XP, and they would still be less in price. It's a little scary to me, but in practice, I've never seen a big company get a really effective process going locally, so...

Charlie: Savings of 20-35% seems to be what they're actually realizing. Wages are going up in India, it's harder to manage a remote project.

John B: And there are things which end up in the wrong part of the balance sheet, or opportunity costs that aren't measurable in a direct way. I think this year we will hear many more offshore horror stories. It might take two or three years before things really come crashing down, before things finally come to account.

Mike: My company is partnering with a couple of established offshore companies, and they're doing fairly well with it. They have a heavyweight process, and it moves a little slow at the requirements phase, but the code comes back in reasonable shape.

Eric: Yes, if you can write a spec and be happy getting back something that meets the spec. But it doesn't necessarily make the next step easier. There's no incentive in the structure to make to code supple, easy to work with.

Russ: Not so much motivation, as just the way it works under this process.

Eric: But even without a breakthrough, the crafting of the pieces of software.

Tracy: And the lack of access to interaction with customers, quick feedback. The suppleness goes into the right place. People in a traditional waterfall are hard to get to make changes. And unless there's someone right there local, you don't get the little "aha!" insights, "Oh, that's what you mean by a cargo!". Some local person who understands the domain will make a little remark which will clarify issues for you, and you'll have an insight that makes things much better.

John B: From "This Discipline of Market Leaders", you compete with offshore people by choosing customer intimacy. Ex: FedEx: "innovation". UPS: "operational excellence", Airborne Express, "customer intimacy", they become part of your shipping division. They'll host your company's warehouse in their shipping hub. We can't do operational excellence, so we need to focus on innovation and customer intimacy. We should take advantage of that, in the way that agile processes do.

Dave Smith: When I was managing a project, sometimes the customers are distributed. In my case, it was financial companies around the Pacific Rim.

Charlie: And you can use offshoring effectively for routine development that's well understood.

Tim: When do you outsource? If it's well understood, get the requirements exactly right and send it out to get typed up into code. But getting the requirements exactly right *is* the central problem in software. Sending the work outside the company is more apt to make things worse.

???: The better companies will provide a local representative to gather requirements from the customer directly, and clarify anything that needs more direct contact.

Dave Smith: Infosys, for example, has requirements teams local to the customer. But wage inflation...there's high turnover in India, people get hired away.

Eric: I think we should be glad that there are good opportunities there now. A billion people pulling themselves out of poverty.

John B: A country of a billion plus people ought to have 3x as many genius programmers. I think we should be very grateful, as programmers drawing high salaries, that the regimes in these countries have held things back up till now.

Darlene:

John B: Stories about mistreatment of knowledge workers.

Eric: I've heard of more problems here, like the H1B workers.

John B: I have another topic...the social networking software, LinkedIn for example. It's a way of extending one's contacts. You can only initiate requests directly through the set of contacts, and each contact has to forward your request along.

(some discussion of automating programming...what level to write the specs at... perhaps Java is a sufficiently formal language... but you'd want a layered architecture with a good domain layer...so)

Eric: That's a good lead-in to the part of the book, Chapter 10 (Supple Design). I favor a declarative style of design (p. 273). But there's a problem with that mindset, you can get the wrong focus, which will lose the details that will lose the agility of the process. I'm all for raising the level of abstraction, but there's a problem with that particular direction of abstraction, trying to make UML an executable specification.

Russ: When I first talked with Don Roberts, I couldn't see where some of his points were coming from. You can't get all of the details right up front, all of the bits of the landscape that give you the most appropriate design.

Eric: ...building a language for my application. I used to think about putting components together. But now I think about building up a language. The fine-grained, well-crafted base level supports the next level of abstraction. A layered language more than a layered architecture.

John: Some design methodologies inadvertently sabotage opportunities for synergy, interplay between levels.

Russ: It felt like we used to design top-down, now it feels more like inside-out. Starting somewhere in the middle of the system, and incrementally fleshing it out as we go.

Jeff: That reminds me of one of the OOPSLA keynotes, that often the most salient categories are those in the middle of an abstraction hierarchy, rather than the most general or most specific categories.

Eric: I think this chapter is really important, but I'm not sure how it comes across.

John B: It's sort of subtle, but it is self-reinforcing as it goes along.

Eric: And I pull from earlier stuff. The distinction between entities and values becomes more important as you put side-effect-free functions onto a value, etc.

Mike: I got one of those enterprise e-zines today, "go here and look at the latest enterprise governance releases." What I read today was the antithesis of it: "standard, standard, standard" rather than a set of flexible guidelines to produce supple software. They aimed for good design, and they ended up with a bunch of rules crafted by ivory tower geeks.

Russ: Standards are motivated by fear.

Dave: But agile processes can have that motivation as well, once you understand them. Fearing getting stuck in wasteful, painful activity.

John B: I prefer "winning" vs. "not losing". Agile processes play to win, while standards-based processes play to not lose. When a doctor says, "we did everything we could." And it's often about big companies vs. small companies. Big companies have a lot to lose, and so they play not to lose. Startups have to play to win, because playing not to lose will just delay the point at which they lose.

Tim: How is one, as a manager, to see through the stories and choose the right process?

Eric: I think that Supple Design is where the connection between design and agile processes shows up the most. How to design things so that we can change them when we need to.

Russ: Managers want to hear things in different ways. Some are more receptive to the "Lean Software Development"

Charlie: Kent Beck's approach is to show something useful delivered every iteration.

John B: I'd say that you should study persuasion, more than agile methods or software development.

Eric: But frequent delivery can cause some distortions of the supple design process from the pressure to deliver more each iteration. Ward Cunningham recently was talking about "making the software softer." Let's say you want to add a feature that doesn't look like it's going to fit very well. You can add the feature in two different ways -- to stick it on the side of the old system. Or you can rework the old system until there is a place to put the new feature, and drop it in. His point is that most people will do it the first way. They don't believe that reworking the old system has value. But if you do it the second way, the system ends up very easy to understand.

Tracy: "Hanging a bag on the side". You have to understand the system, and then separately understand each bags. You end up with a bag on a bag on a bag, and finally you have a pile of spaghetti code. And the mantra you hear is "but it works!" And until you've gotten there, to where it encumbers you, you don't really see it.

Eric: Or when you've gotten to a really supple, unencumbered place. If you deliver every three weeks.

Jeff: Version 1.0 of a product often has features but little design. And then it comes time to start working on 2.0...

Eric: I think the payoff for supple design comes within weeks. It gives you a better version 1.0, and makes a huge advantage in version 2.0

Russ: I've seen a problem with people reaching for a supple OO design and dig a deeper hole with OO than they could have with a procedural language.

Eric: Even advanced designers can be too perfect, too abstract in their design.

Tim: (describes a situation where success was redefined as the very limited success that was actually achieved). We have a deployed product with a lot of revenue in the field, but it costs so much in the support end that it isn't a big win.

John B: A functional organization understands the cost of each tech support call. So you make design decisions based on that if it's a significant force.

Eric: At the place I've been working lately, they work hard to be compatible with all the ancient versions of the web browsers. Who knows what happens, if you make a few people unhappy. It changes the perception that it's a site that is very easy and always works for everyone.

(some discussion of ease of use, supportable software, use of facilities that only some people will be able to use or choose to enable: [Javascript, ActiveX])

Eric: And sufficiently unpleasant things to work on will drive people away. Supple Design makes things not only easier to work with, but more pleasant.

Darlene: And easier to understand.

John B: Let me mention a phrase Ward Cunningham uses, "Design Debt". He was working with financial services companies who understand the appropriate use of debt. But you're talking about design surplus here, starting to earn interest on the quality of your design.

Eric: I prefer supple design as a way to describe it, to describe where you're at when you're free of design debt.

Russ: So at Eric's suggestion, perhaps we can pick up Chapter 13, which is pretty short. "Refactoring Toward Deeper Insight".

Eric: It tries to tie together the whole of Part 3, since there are several chapters there. It says similar things, but shifts the angle.

John B: This reminds me of Ward Cunningham's phrase about "waiting for insight". You don't *have* to wait, but then you don't get the insight.

Eric: But sometimes you have to go looking for it.

Charlie: Insight usually comes from hard work.

Eric: So you don't actually go looking for a breakthrough, but you keep refining your knowledge and then you can be in a place where you can get insights.

Russ: So, earlier you say, "you can't plan on a deep model." But it seems you talk about a set of practices which give you opportunities to break through to deeper insight into the model. Making your own luck.

Eric: You can still get a workable supple design.

Bob: Jazz improvisation. You still practice, you listen to other players, learn new motifs, polish your scales, and then you're more ready when the opportunity for an improvisational leak happens.

John B: For me, it often happens when something in the code bugs me. And I reflect on it and I can make a breakthrough to a better design.

David V: When a system is in deployment, and you discover something where you want to make a change, you may need to get management's agreement to accept a risk. So right now we're looking at a "patch" release. If you just fix the bug, versus using the opportunity to clear up a design issue by making larger changes.

John B: Politically there are things you don't want to do in a small-point release. Version 3.00 vs. Version 3.01, you wouldn't expect much of a change in features or in how the system does its job. But if you're going from version 1 and you have a big insight, then you may have to work that out in the version 2 branch of the source, where you know that there will be bigger changes and a full quality assurance cycle, and all the participants will expect the larger changes.

Wayne: These breakthroughs are crucial for making a deep model. The little stuff is important, but not the major part of the progress.

Eric: The little stuff is necessary to make the breakthroughs possible, and is worth doing in its own right. I compared it to punctuated equilibrium in evolution. (Which is itself a sudden insight which changed the evolutionary model).

David V: It's all self-similar.

Charlie: The punctuated part was part of Darwin's theory that was lost. Stephen Jay Gould discusses how the gradualism was a cultural accretion, and in some ways punctuated equilibrium.

Eric: Right before a breakthrough, things don't feel good, and sometimes right after as well. "There is a gaping hole in the model.... the old model looks poor." (reading from page 326) So there is much work to do, you look at the system and see how inadequate it is based on your new understanding. And if people can see things from a different point of view -- that they've learned something, that this is an advance. When your model starts to look bad, you know you've learned something important.

Jeff: So this is a disillusionment with your original Hello Kitty picture of the world.

Eric: Which shows you've learned something and can now make things better. If people were more prepared to deal with the consequences of the breakthroughs in knowledge, of feeling bad at that point, then they can work much more effectively.

Charlie: It's kind of like breaking up with a girlfriend or boyfriend

Eric: And then you get another one just like the last? (teasing)

Charlie: "The design is wrong!" -- mostly you feel pain and loss.

Eric: You're attached to it.

Charlie: And you were proud of it.

Eric: Emotionally invested.

Mark?: People get identified with the design, especially if they participated in creating it.

Eric: I've been in that situation, where I wasn't the one to see the breakthrough. It isn't as satisfying as when you can see the change that's needed. It's an essential part of our craft to detach, to let things go when they aren't right any more.

Charlie: And six weeks ago you were really proud of it.

Eric: And perhaps it was the breakthrough from the previous version.

Mark: I'm often earliest to let go of my own code if there is something better. Sometimes it's the other people on the team who don't want to let go.

Eric: There's the notion of occupational temperaments. A perception that software developers like to work alone. XP has changed some of that by emphasizing a team focus, a social activity. But here's a different level yet, of being detached from your own creation.

John C: And good diplomacy skills.

Eric: Yes, when you see a flaw in someone else's design. But it's not one I think people are looking for when they're hiring, yet.

Carol: Why do people think there has to be one superior design. In architecture, if you can give a client several designs, that's great, that you can see different ways to do it.

Eric: The reason you can get away with a single model for your analysis and design is that the model space is large, that you can explore it until you find one that works for you in both ways. But I don't know why we limit things.

Charlie: Architectural customers are better, more insightful about buildings and needs than typical software customers.

Eric: We should remember that the original patterns movement was started by Christopher Alexander, who was an architect dissatisfied with the building architecture that he saw around him.

Tracy: Building design is more limited scope.

Russ: You can have discussions with the client about architectural issues, alternatives in technology.

Carol: In architecture, earlier in the business, all the design documents were manually drafted. After the advent of CAD, we ended up with the same cost and time to deliver a design, but it was more detailed, more fully developed. And the difference was in the cost of making changes. Moving a wall was very painful if you had to manually redraw a lot of plans. In software, we may not yet have the appropriate tools.

Eric: Better tools would be good, but again I resist that architecture metaphor. I think these metaphors are throwing us off. [...]
Writing this book was much like software development. The experience of inside out, top-down as well as bottom-up. Expressing some points you want to make, and at the same time how to present it, how to organize it. Crystallizing and refining concepts as you go along. You work, polish things, then you finally understand it and you have to let your work go and rewrite it to what it needs to be. I recall a quote from Hemingway, if there was a really polished, really beautiful page or paragraph, the best thing he could do for the rest of the book was to throw it away. We get stuck endlessly in choosing the right metaphor -- architecture, manufacturing. And as you point out (to John B), we don't even do them well.

Dave Smith: Architectural designs, at the time they are sold to the customer, are far more visions than designs. A construction company would be very angry with trying to actually implement the design.

John B: There are two outcomes in software development, "I'm a great architect" or "You are lousy programmmers".

Chapter 10 continued

Notes 13 Jan 2004
Domain Driven Design (VII)
Guest Author: Eric Evans

(Ken Scott-Hlebek starts with a discussion of the "Master of Software Arts" program at UIUC. He is part of the group of students involved in a trial run.)

Russ: Perhaps we could move on to Chapter 10? (starting p.246, Intention-Revealing Interface)

John B: Smalltalk Best Practice Patterns, "Intention-Revealing Selector." I tend to think of the Java keyword for interfaces.

Russ: When you are talking about "intention revealing interface", is it the design-by-contract sense of syntax and semantics?

Eric: Yes, Design By Contract was one of the main factors. I don't see it as just the method signature.

Tracy: You've gone to the trouble of creating a Ubiquitous Language...I like the way the intention-revealing interface pattern is connected. And your example is good to show how it can be hard. "Paint.paint()", for example.

Russ: artPaint.GetBlue() vs. artPaint.Blue()? I'm a little surprised you went that way.

Eric: It's kind of entrenched as a convention in the Java world, and it's not too bad.

Russ: Having "create" on creational methods... it's useful, sometimes, to have a naming convention that is a common idiom to help prevent surprises.

Eric: I'd like the user not to care whether they were fetching a variable value or calculating a derived value.

John B: I've run into that problem.

Russ: We typically use "get" for retrieving primitive values, like 'int'. We use "get" a lot for accessors.

Tracy: Whether it's calculated or not.

Russ: We don't tend to use "compute" or "calculate".

Mark: It's like J2EE or JavaBeans conventions.

Phil: Yes. EJBs use the JavaBeans conventions to make introspection easier. You can look at the JavaBean and see what state is accessible.

John B: They call it a Design Pattern but obviously it isn't.

Phil: But it's a convention that makes it easier to discover properties.

Mark: But even Sun, you get String.valueOf(...) etc.

Eric: But isn't a naming convention an awkward way to put in metadata for a tool?

Jeff: But it gets used over and over and over again. Custom attributes in Java are the

Eric: Sometimes "has" is better than "is" for a boolean.

Mark: I've seen "isHas..." and "getIs..." on accessor methods.

John C: Were these native English speakers?

Mark: Yes.

Mike: You want the convention to extend beyond is/has for your accessors. If you enrich that language for describing, for removing ambiguity of the interfaces, than the current state of things.

(some discussion of accessors and mutators, including conventions in Java, Smalltalk, and Forté).

Scott: If you're going to follow a convention that you use get..., you shouldn't have to care whether you fetch a field value or compute it.

Eric: It would be OK, if it were side-effect free. A lot of "calc" methods actually modify values.

John C: Truly side-effect free? Or are you willing to have things cached?

Eric: I define it here, as no observable side effects. So caching is OK if it's appropriate. I don't suppose anyone would like to make the case against an intention-revealing interface?

John: "Too much typing?"

Jeff: "It's such a pain to change the name of everything all the time when you change the behavior!"

Eric: That's the worst case, when someone has chosen a good intention-revealing name, and someone else changes the behavior. The method name is lying to you.

(question: should the name of iterators or index variables be long/meaningful?)

Russ: I label an iterator with the type of the instance coming out, so iColor is an iterator over colors.

Phil: eachColor is the convention I'd use.

(internal vs. external iteration mentioned)

Phil: There are things you can do with external iterators you can't do with internal iterators.

Eric: But 99.9% of the time you don't. You just run down all the items in the collection.

Russ: But it makes you feel like a real programmer to write an iteration loop.

(a discussion of iterators and collections, sometimes the application is just lists, and it doesn't matter whether you use iterators or whether you count up via the index...)

Tim: Iterating over a 2D thing... I find that people do a better job naming when they use iterators than when they index with integers. There was one case where they used i and j to loop over the results, and in one place they had misplaced an i for a j or vice versa, and something that was supposed to be sent to one user was sent to all of them.

Eric: (discussion of a difficult case refactoring a nested loop with embedded conditional logic)

Mike: A server test program that opened a number of sockets, one for each thread, and sent for each socket, a number of messages, and waited to see if each one had the proper completion token returned from the server. So there was a loop of bursts ...

Eric: Wouldn't it have been easier to have an object to manage the threads? Wouldn't it have been quicker:

Mike: Probably. I just ran out of time, and it worked.

David V: Do you declare an iterator in the same scope as the loop (so it is out of scope after the loop)? (expanded question: "Do you think it's important to declare a variable as close as possible to its point of use?")

Show of thumbs: some up, some down, several sideways.

Mike: You may need it in a larger scope, so you can't declare it right next to use.

Eric: Sometimes you can have a declarations section up at the top of a method.

Phil: I put my fields up at the top of a class, certainly. Though I've seen people sprinkle their fields around.

John: I use IntelliJ to write my iterators. What does it do?

Phil: So it writes the loop for you.

John C: There was a "GIST" specification language. They compared 'declared once" specifications with "declared plus shown as summaries", and the ones with generated summaries were more useful.

(some extended discussion of how to organize functions, logical and physical representation of the code, etc.)

p. 250, "Side-Effect Free Functions"

John B: So do side-effect free functions take a value object and give you a value object back?

Eric: Sort of. Often the return value is a value object. But the connection is rather that *every method on a value object must be a side-effect free function*.

Phil: Couldn't a value object function take an entity, change it and give it back?

Eric: I wouldn't write it that way. But maybe if the value object was designed for that purpose. But I'd rather pass it to the entity, "change yourself according to this transformation object".

Tim: More like a service than a value object, that its job is to change things?

Eric: That alone wouldn't make it a service, but a service might do that sort of thing.

Russ: Perhaps you could clarify this? You're talking about changing function arguments?

Tracy: That started with Phil's example.

Eric: I think it's risky business to change function arguments. A lot about supple design is to be very obvious. Side-effect free methods are easy to test.

John B: And to combine.

Eric: And to combine.

Russ: This kind of lets out the notion of Collecting Parameter.

Eric: When the design is exactly that, it would be sensible.

Phil: Collecting Parameter is more like a mechanism than business logic.

Eric: That might be a way to discuss Collecting Parameter, to try and keep all the business logic out of it and put the meaning in the hands of other objects. I don't want to be too all-encompassing here. But you can make things really easy to understand if you can put the complexities onto side-effect free functions on value objects, and keeping your mutators on your entities really simple. And having assertions (in the design by contract sense, though they could be implemented as unit tests) on the simple state of the entities. So client code will be really easy to understand, as a combination of intention-revealing side-effect free functions. That's why these three patterns work very well together.

Russ: You've spent a little time thinking about this, then.

Eric: Yes, a little [spoken heavily].

John B: Closure of Operations following Standalone Classes?

Eric: Yes. This last part of the chapter is about trying to reduce the number of dependent concepts. There are two different things here. They both contribute to reducing interdependence. When I talk about dependencies, I don't mean compile-time necessarily, it's a matter of conceptual dependency. If you have a manager class, it doesn't make sense out of the context of an employee class. But perhaps depending on your model, you could understand Employee all by itself. A genuine standalone class is one that you don't have to refer to other classes. Closure of an operation yields a method that can be understand all by itself. Some of the things I talk about here aren't strictly closed, some of the Standalone classes aren't completely standalone. But if you had a class that only referred to system types, strings and ints, these classes aren't strictly standalone, but for practical purposes it doesn't add any intellectual load.

Mike: If you can move it to the next directory over, and it will still compile and be usable by client code there, that's a good test.

Eric: At the level of the code. Let's say, often a street address is a standalone class whose state consists of ints and strings.

John C: Then when you start doing business overseas, you start caring about how an address is composed for use in other postal systems.

Eric: You could have other approaches, you could have a bunch of smart formatters for various countries' conventions. But most likely, as you say, you'd model an address as a composed object.

John B: Or one big string.

Eric: There are two very important points here. Dependency is not a matter of tracking compilation dependencies. In fact, the worst dependencies are often ones which are implicit. For example, a class in one place which operates on an ID, or on some object state which is represented as ints and strings. That sort of <em>implicit</em> dependency counts. Dependencies are very costly in terms of adding complexity. So these rules talk about (1) Make the dependencies explicit and (2) Make them as few as possible.

Russ: The conceptual version of coupling and cohesion.

Eric: Yes.

Russ: I know about closure of operations from mathematics. But perhaps you could give a clearer sense of what it means here?

Eric: Operations where you put in one type and get another instance of the same type are examples of closed operations. Selecting from a collection gives you another collection.

Russ: So the way you wouldn't want to do it would be to mix a pigment and a pigment and get a paint, then pigment would not be closed over mixing, and you'd have to be concerned whether you were dealing with a pigment or a paint.

John B: So the part of Java's String protocol, toUpper(), substring(), etc. which all return String (not StringBuffer, char[], etc.), would be a set of operations closed under String.

Eric: Closed operations are much easier to compose, and that makes them easier to work with. The fact that they are instance methods on the same type which they return groups them cohesively together.

Conceptual Contours (pattern)

Eric: I feel it's very important, but I have difficulty explaining it. It's too much the product of intuition, I guess. Some of the ways I suggest to find them are sort of circular.

Mark: So perhaps some of the advice is the right level of detail for your model. Things like atoms are too fine-grained for a model of a car, they're outside the conceptual contour. You'd want steering wheel, tires, etc. Not a monolithic "Car", and not atoms.

Eric: And not front half/back half.

Jeff: Unless you're doing finite element modeling of your car.

Russ: Conceptual distance is something you might want to discuss here. And the sense of what concepts are close to one another. There are interesting things (Kohonen maps) that are self-organizing clusters of concepts, ways to get a more global picture when you're just starting with the sense of a bunch of individual things and their conceptual distance to one another.

Eric: That sounds like it's the right neighborhood. As long as we don't get too carried away and turn it into another Rational Rose.

Russ: But a tool that understood conceptual distance could be useful in some sense. Maybe even notifying people who are concerned with issues that are conceptually close.

Russ: Dirk Riehle has done some work in executable UML. It might be good to have him in to discuss the other side of things.

David: There's this design that if you attune yourself correctly, it's not necessarily the code, or the diagrams, but you can talk about it.

Eric; "The model that can be named is not the true model"?

Mike: Modeling as a distraction rather than a catalytic activity. You want a catalyst but you have a modeling language? So developing a ubiquitous language

David: I had a professor say: "after a while, your math is in your wrist". But as soon as the diagram becomes "the thing", where it's important, where you're generating things off it, you can get in some very bad places.

...

Eric: When I review this chapter (Chapter 10, "Supple Design"), it starts out talking about how good design has this tactile quality (picture of a bicycle), talks about some of the old Smalltalk idioms, moves on to Design by Contract, this vague thing about Conceptual Contours. So by the end of the chapter, the odd assortment of stuff is what you need to accomplish the goal, a supple design that is pleasant to work with. These pieces are not traditionally grouped, but I see a deep connection. Do other people?

Robert: Some of each. Are they connected? Yes.

Wayne: I didn't think of them as an odd assortment until you called it out. Then I thought, "yes, this is a mixed bag."

John B: Like listening to a director's track on a DVD, they note all the inconsistencies that you'd never notice.

Russ: Wouldn't some other things belong here? Value Objects vs Entities could belong here.

Eric: They show up in discussion. I didn't want to repeat a pattern definition. I didn't mean it to becomprehensive, to cover all the techniques you would use to make the design supple. But enough to convey the idea.

Mike: You are at different levels in the practices of good design. The language connecting the pieces may need to be enriched, but the practices are all important.

Eric: The first three patterns, to me, seem to hang together very well, very closely. Conceptual Contours... that's where the picture of the bicycle comes in. You can see there are some things that are easy to take apart, other things that are tightly bound assemblies. At the level of the designers of a bike, the stack of gears, the gear assembly, was the part, rather than each gear individually.

Mike: Bikes have these parts which are interchangeable to such a degree because there are standards involved. Tolerances and specifications.

Mike: The success in communications software is based on standards. TCP/IP, SMTP, etc.

Eric: I think that's an example of a supple design as well. They're well composed, and well separated as well. The breakdown was one that allowed for growth and adaptation.

Russ: Isn't that a topic from chapter 14?

Eric: And those things are related, aren't they. But I think of the Internet protocols as a set of supple designs done right. It's not the first thing they came up with. The initial prototype isn't what we have now.

Jeff: It took them at least four versions to get it right. (reference to IPv4).

Eric: If you freeze too early, establish the standard without investigation and practice, you end up with a stiff, awkward kind of design.

John B: "rough consensus and running code", at least two successful independent implementations of a proposed standard.

Jeff: And it helps if the two implementations talk to one another.

Eric: And the OMG is trying to create this standard before anyone has actually implementing anything like it. (refering to UML support for MDA)

David V: Eric, I'm looking for a way to "sell" supple design. So can I say, "If you use supple design, it's less likely you'll have to throw away an application and rewrite it from scratch?"

Eric: Yes, it's key to being able to hold onto your program and being able to modify it for a long time.

David V: I was hoping your chapter conclusion would make a stronger point of it.

Robert: That's the whole-book benefit, rather than just the Chapter 10 benefit, right?

Russ: Chapter 10 applies even when you're outside the domain part of your application code.

Mike: I might express it as "Debt-free design" in the words we used last week.

Eric: I would say that with a supple design, your project may just start taking off at the point a conventional project would be heading for legacy status.

Eric: Did anyone actually read Chapter 8 and the example at the end of Chapter 10? Chapter 8 is an extended example, and the end of Chapter 10 has a more close-up view of the same transformation. Almost, that is. It's more like taking the stiff early version and turning it into something supple. That's a nice combination to get into.

Chapter 14

Notes 20 January 2004
Domain Driven Design (VIII)
Guest Author: Eric Evans

Russ: Let's start with Strategic Design (Part IV) -- what do we need it for? How about considering the ways to get in trouble on the extremes. How do you look at a system and understand whether we're doing what Eric's doing, or one of the bad alternatives he discussed. (p. 328, halfway down). The more systems I see, the more surprised I am by the range of systems I see. Some of it's really good, some of it's kind of scary. This was a good way of describing a large class of the problems, so I thought it was a good way of introducing tonight's topic.

(Russ referred to text in second half of second paragraph p. 328, "monolithic" vs. "ad-hoc", avoiding extremes).

Dave: Systems glued together out of lots of pieces are also often full of contradictions, an unresolved argument about the system.

John B: Like Unix command-line tools, having varying views of the world?

Russ: Is it ad-hoc components, or ad-hoc interfaces?

Mike: the alternative being an enterprise-wide bus, a common interface for things to plug into, that tries to be everything to everybody. It's hard to get systematic designs through, because you have many sponsors and many forces.

John B: And you have to actually establish the protocols, how to get things to communicate.

Mike: My company has tried for the system-wide bus, and it seems to address the needs of only a small number of the people who want software.

Russ: The author is talking about more than just the technical breakdown of the system, but about the model, the vision of the system.

Eric: Dave said an interesting thing about small pieces composed together being sort of a dialogue or debate about world views of the system. That is very much about what Chapter 14 is about.

Phil: A middle approach to bridging the extremes is to compose the pieces, then to look at what you have, and refactor the composition into the thing it needs to become.

John B: I see on p. 389, there's a discussion about refactoring to Shared Kernel from Separate Ways,

Eric: Refactoring is the *means* by which you change strategies, rather than the strategy you use.

Phil: Refactoring as using the code to change your ideas, you don't see that as well as refactoring as a mechanism?

John B: Eric has listed all the ways to evolve and change the implementation of the model that he can think of, rather than the actual change of the model as ideas.

Eric: All the ones I wanted to recommend.

Tracy: If I understand, Phil, you're talking about refactoring as a process of understanding.

Phil: Yes, learning, and eventually you get to a global view, a similar result. The code is telling you what the model has to be.

Eric: I'd like to shift rails a bit, I start from a more fundamental premise. I start from an observation that on any large project, with more than 10 people, you will have more than one conception of the domain that the system is based on. More than one conception means more than one domain model, and that will affect choices about design. You can just plunge ahead not realizing that, and not realize when one person's concept was controlling the design and when another person's different concept was. So that's where BOUNDED CONTEXT comes in. You would like to have a sense of which model applies in what part of a larger process. There is sometimes a very clear situation, where you have smaller subsystems worked on by different people, and each can be its own model, its own context. But with a large subsystem, you're more likely to end up with multiple inconsistent views. This chapter is about understanding that there are different points of view, and different ways to describe things, and that becoming aware of these different perspectives is important in dealing with larger systems. The "Blind Men and the Elephant" story would be less interesting if each blind man knew that he had a separate point of view on the same thing. And we want things to be less interesting in this case, with software development. So you want to start out by mapping out the territory as it exists, before making changes. So tactics like CONTINUOUS INTEGRATION support the decision to have a unified model within a single BOUNDED CONTEXT. This sort of CONTINUOUS INTEGRATION is the code, *plus* people consciously synchronizing their concepts with one another through exercise of the UBIQUITOUS LANGUAGE. But there are other successful ways of connecting things.

John B: System model vs. domain model? Most of these patterns, to me, seem like ways to integrate a system rather than a domain.

Eric: Usually people think of system integration as connecting a set of distinct programs. But you could create two systems based on the same domain model. Or you could create a program which was internally split between two domain models. You'd prefer to split your components along the boundary of the variant models. But I've seen components and subsystems (even classes!) with multiple models reflected in their implementation.

John B: Is Extreme Programming's continuous integration a different concept from yours?

Eric: I think it is the same thing. The XPers are exactly correct about continuous integration with a context, within a single model conception. If you need it to interoperate, you need continuous integration of the *code*, and also of the *model*.

John B: They're fumbling that direction with the System Metaphor.

Eric: And co-location.

Phil: And pairing.

Russ: I've seen a single developer with 3 different bounded contexts with name clashes in the same source file.

Eric: Three *unbounded* contexts.

Russ: Intermingled contexts.

Eric: If they had bounded those three, they would have been in good shape since they would have been separate; or more likely they would have unified them to remove the contradictions. But knowing what the practices help you achieve, knowing *why*, allows you to adjust to situations, to know where you are at. And knowing when to stop. You can't CI a whole company, you can't co-locate everyone. And at a certain scale, the overhead of continous integration in a large system becomes too much. So by making a context map, you tell yourself, your teams, where you are making CI work, where you can run XP full-out within a single bounded context, and where you want to establish a boundary.

Russ: There are two very different notions going on here. 1: You and I have different models of a system, with shared understanding of a subsystem, and no assumptions about the non-overlapping parts -- two separate but possibly overlapping models. Vs. 2, if you and I were working on things, and there was an overlap of concepts which were not part of the shared kernel, but were in conflict. It's an interesting distinction to talk about.

Eric: Yes, that's a very instructive example. So in the first case, let's say there *is* a shared set of concepts. What I call SHARED KERNEL. That's both concepts and code.

Russ: So if you were working on the dessert and I were working on the floor wax, we wouldn't have to interact on those distinct parts of the system.

Chris: Until someone comes up with a product which is both dessert and floor wax.

Eric: So in this case, you might subclass the thing we've already got in hand, say for me working on the dessert, I'd say a floor wax is a special kind of dessert, and you with the floor wax, you'd say a dessert is a special kind of floor wax.

John B: Elephant is a subclass of wall, floor, snake, tree, and rope. (p. 379)

Eric: The blind men were lucky, that they were each looking at a single part of a larger system. But with desserts and floor waxes, the systems are more complex. At first we might not have any problem. One way is we can say we go with your model, one way is to go with my model, or one way is to come up with a new model that meets everyone's needs. Let's say we introduce a new abstraction.

John: They're both Sprayables.

Eric: So we've discovered both the dessert topping and the floor wax are sprayables.

Dave: It opens up new product opportunities because you could make something that's a floor wax, dessert topping, and bug spray.

Eric: I think the marketing department would have trouble with that last one.

Jeff: Maybe lip gloss as something in the same unified space as floor wax and dessert toppings.

Russ: How are we making these trade-offs to decide how much of the system should be unified?

Eric: It's not always unification. you might have a system that's grown to the point where integration has become burdensome, so you decide to split things off and go separate ways.

Dave: As things scale, I've never seen it be easy to abstract a core.

Chris: Could you abstract out a framework?

Eric: I'm not talking about a huge company with a "chief architect" someplace dictating to everyone in the company "these are the objects you'll use."

Phil: During the development of J2EE at Sun, different groups had a concept of "qname" that was slightly diffferent. We got everyone in the same room and agreed on a common interface that everyone could use, and we created a JAR with just qname in it that everyone could use.

Eric: That's a Shared Kernel.

Phil: Right, and it's a little thing, not a big one.

Dave: What was the overhead involved in getting all the interested party together in a room to make that decision?

Phil: You have no idea.....

[...]

Eric: The first thing is to have contexts that are bounded. And have a map so you can say, "This thing belongs to this context."

John C (?): What about the tragedy of the commons problem, since the shared parts...?

Eric: But all we're agreeing on is the _boundaries_.

John B: "Good fences make good neighbors."

Jeff: Get that down.

Eric: You want to know, "Am I in my own context, where I can go my separate way, or am I in a shared context where I need to integrate with someone else?"

John C: I'm thinking of a company where there are two products, one of which is a big revenue producer, and another that gets less resources. The one that makes a lot of revenue basically gets to disregard the needs of the other one.

Eric: That sounds like a case for Conformist, because of the power differential.

Eric: I'm not saying the company has some global enterprise-wide context map. I'm just saying for your own project, identify the different contexts there.

Eric: Even in a one-person project, you'll have multiple contexts, like "current understanding", vs. "last years understanding". (The code you wrote last year, but haven't had a chance to refactor to your latest insight.)

???: What about upstream/downstream suppliers where one is stuck with a supplier.

???: "Ball and chain" pattern.

(Mike: discussion of a protocol with a large installed base with messaging to distributed clients -- difficulties with changing the protocol to enrich the service.)

Eric: So your clients have a presumption, a model for what the messages in the protocol mean. And how they handle the messages reflects that model. If they get back a response message in a different format than the one they asked for, then they have to handle that. They have to handle the fact that the variations in the new model are exposed by the new protocol. So what made this hard is that within your context, you changed your domain model, but the other systems at the boundary are related to you via an OPEN HOST SERVICE. The interface expresses a frozen model.

Mike: Because so many groups are connected to it.

Eric: And part of the problem was a change in the model as expressed in the protocol. If you had just made a slight re-arrangement of the same information, it would be no means as difficult. There's a technical dimension to the problem, but there is importantly, a conceptual one. And the conceptual difference is what it makes it hard.

John B: Sounds more like ANTICORRUPTION LAYER rather than OPEN HOST SERVICE.

Mike: We have an invariant API.

John B: If you had an ANTICORRUPTION LAYER, you would be able to fold the new protocol into the old one.

Eric: But let's say he is getting his message data from somewhere else, a legacy system. So what he does is set up an ANTICORRUPTION LAYER to translate the old stuff into the new, enhanced model. But it's the client-facing side, he's got clients who are asking for stuff. There might be a few with ANTICORRUPTION LAYERs because they need to express the concepts of the message internal in unusual ways, but most of them use CONFORMIST, expressing his message stream in the same terms.

Mike: We want to enrich the service without affecting the clients who are not interested in / capable of dealing with the information in the new protocol.

Eric: They've always had Sweet 'N Low before, but you want to give them sugar.

Russ: You can give them more feeds, by establishing a an upstream message feed that you can translate into Sweet N' Low for the old clients, and to Splenda for the new clients. (Arranges two diverging lines of pens and flatware with a pink paper packet in one line of the V, and a yellow one in the other line).

Eric: OK, so I understand you have a TRANSLATION LAYER which takes the new, enriched internal format and emits a message feed in the old format for your established customer base.

Eric: There's a stress point on the OPEN HOST SERVICE pattern. If you have two sets of clients who want different services, it's somewhat difficult. OPEN HOST SERVICE says that everyone gets the same thing, everyone can accept the same protocol. So if your situation is that each customer wants little, extra bits on the service, perhaps you want a bunch of one-off translators, one-off versions for the different groups, to keep the primary protocol simple and clean. Now we end up with more of a classic CUSTOMER-SUPPLIER relationship for the custom parts of the protocol. That frees most of the clients from dealing with the special interfaces, and the individual special needs can be managed without changing anything about the common assumptions.

John B: You have an off-the-rack interface, and then a set of tailor-made ones.

Eric: But you want it recognized that that's what you're doing, that there is a decision process being made.

Chris: The more I'm listening to this is more like a pattern I applied in a system I worked on. In this case, the requester specifies the format. If they didn't specify, you got the old format. If you needed something new, something else offered by the current system, the client would change their format specifier.

Charlie: Do some of the specializations, some of the customizations really belong on the client side?

Eric: If you're using OPEN HOST SERVICE, you need to keep it simple and consistent for you as the maintainer. For variations, you may choose to make a special adaptation, negotiating it one at a time based on company priorities or revenue, etc. If you have a custom protocol that only *one* customer can use, you can change the interface as often as your two groups agree to. Once there are multiple clients of the protocol, it's far, far harder. It's the OPEN HOST SERVICE situation, where the interface is frozen in place.

Darlene: At what point might you go toward a general solution, a new protocol with the space to accommodate a new set of variations? Rather than maintain half a dozen variations on the common protocol?

Russ: Maybe we should define ANTICORRUPTION LAYER, we've used it several times.

(p. 364, ANTICORRUPTION LAYER: picture of the Great Wall of China)

John B: I'm seeing this adapter-service thing, that's a lot like...

Eric: I use the terms TRANSLATION LAYER and ANTICORRUPTION LAYER. An ANTICORRUPTION LAYER is certainly a TRANSLATION LAYER. It's the difference in intent, in posture, that makes the difference. An ANTICORRUPTION LAYER defends your system from having to adapt to a divergent external model. I don't think you could mechanically tell the difference. But it's the difference between a wall and a bridge, almost. A bridge is like a regular TRANSLATION LAYER, where one side is reaching out to the other, or both to each other. An ANTICORRUPTION LAYER is protected, a wall, a moat, and a gate. You don't have an ANTICORRUPTION LAYER separating you from a protocol that you designed.

Russ: Maybe a protocol you designed in the past, that you don't want to work with any more.

Eric: With ANTICORRUPTION LAYER I make stronger statements about why you're doing it. It's that you are protecting your model from the concepts on the other side. The concepts on the other side of the wall will mess you up if you have to try and deal with them.

John B: Not just corrupting data, corrupting your model.

Eric: Your way of thinking. There isn't a functional distinction between the two.

Mike: The OO-SQL translation layer as one example of trying to bridge the gap.

Eric: An ANTICORRUPTION LAYER is one-sided, built by one side to protect itself. Vs. the Chunnel, a cooperative effort by the British and French to bring each other together.

Mike: You could think of an O-R persistence layer as an ANTICORRUPTION LAYER if you have to adapt to an organizational standard which interferes with how you can deal with things.

John C: A Cartesian to radial coordinate translation.

Eric: More like where you have Cartesian coordinates on one side and on the other, a tribal society which describes how to find things by how many days' walk between locations, considering weather conditions, and refers to locations by their historical association. You would want to protect the clean, simple Cartesian model from the complex expression of locations on the other side.

John B: So let's say you have an RDBMS with transaction scripts talking to a database. That's where you might put a translation layer to protect the database?

(some additional discussion of ANTICORRUPTION LAYER vs. TRANSLATION LAYER in different contexts)

Mark: Web pop-up blockers are an ANTICORRUPTION LAYER.

John B: They protect you from the advertisers' protocol.

Eric: By doing complicated gymnastics inside the browser. If you provide a compatible interface to protect your clients from changes, that's being nice.

John B: It's like being CONFORMIST to a layer that keeps squiggling out from under you, you conform to what the ANTICORRUPTION LAYER presents and adapt to the changes inside that layer.

Eric: If the old model is nice and clean, if it isn't a conceptually difficult translation... unless there were no longer a straightforward relationship between the new model and the old model, it's not something where the service is protecting yourself. I don't think that offering a

Russ: Insulation but not anti-corruption.

Eric: But if the old model were a BAD one, incompatible, a messy, complex translation from the current system's model, then perhaps it is an ANTICORRUPTION LAYER on an outward feed.

Tracy: Maybe I can make a point that will help. So let's say the new internal model is the brown sugar. So if the guys that wrote this offer a new interface that's sugar, and doesn't offer the interface of Sweet N' Low, then the client might build an ANTICORRUPTION LAYER to defend themselves from the incompatible new interface.

Russ: So if the motivation is trying to protect your old clients by offering a translation layer to a protocol they want, it's different from...

Eric: Yes, the attitude is different.

John B: It's where there is a meeting of minds (TRANSLATION LAYER) vs. no meeting of minds, a disagreement (ANTICORRUPTION LAYER).

Charlie: The word "corruption" is subjective itself, it depends on point of view.

(some additional discussion of ANTICORRUPTION LAYER to attempt to clarify its usage)

p. 371, SEPARATE WAYS

Phil: I love the picture for SEPARATE WAYS. You see all those team rowboat pictures in those motivational posters. Seeing that picture of that one guy in the rowing shell really captures the feel.

John B: It's like a standalone context, like your earlier discussion of STANDALONE CLASS.

Eric: Yes. But separate contexts may introduce duplication. If you have separate contexts, you can have duplication. But not sharing.

Russ: It's positive danger to share things between contexts.

Eric: Unless you have a SHARED KERNEL.

Mike: Does SHARED KERNEL drive you to use CONTINOUS INTEGRATION? Or another specific process to support the SHARED KERNEL.

Eric: Yes, if you have SHARED KERNEL, you have a process that needs to support it. CONTINUOUS INTEGRATION is an example of one process that could support it, but it's not the only way to do it. Frequent integration, appropriate to its rate of change; having each side supply some test cases, and each side runs all the shared, compiled test cases. I wouldn't recommend in more detail than that, but you do need a process.

John B: So you might have meetings or pow-wows between the teams sharing the SHARED KERNEL, to negotiate the interface changes, where an XP team would just go bam, there the interface changes.

Eric: *Within* a context, you can use full-on continuous integration.

Charlie: Does a SHARED KERNEL tend to be more stable than the rest of the system?

Eric: Sometimes, but not necessarily. Sometimes it has the most change at certain phases of the project, although it has more inertia from the multiple forces depending on it. You've traded away some agility.

John B: You need these patterns in order for XP to work, for changing the system to make it better?

Eric: The assumption that change makes things better is implicit to XP.

Chris: Does the way that XP works, that you surge forward, changing the model, in a couple of different contexts. Does that make integration of two divergent teams' work difficult at a later point? The lack of detailed design documentation?

Phil: Just read the tests?

John B: The tests will get you a long way. All the translations will have to be green-bar preserving.

Russ: That's where you encounter the greatest risk of the tests becoming irrelevant. When the model changes, the tests may be making irrelevant assertions. Incorrect assertions.

John B: The unit tests, yes. Not the acceptance tests. They reflect knowledge of the domain.

Eric: But those acceptance tests contain some assumptions about the model, they exist within a context. If the model changes slightly, what the acceptance tests express may be subtly different.

Jeff: Perhaps if the acceptance tests were expressed in different units, they might be able to be consistent within a larger framework. Say more detailed units of time or money or something. That you could translate the acceptance tests into a common larger perspective, but the new perspective might have lots of holes in the acceptance test coverage.

John B: But how do you do this in the context of a non-XP team? You have a document, maybe, but it doesn't reflect how things really are.

Phil: How things used to be, how things were supposed to be, how we thought the system was going to turn out.

Chris: Even if those two documents were inaccurate, they are marker points to bring up subtle points that people might forget. But they provide a place to start talking, a guide to find out if the two kinds of apples we're trying to merge are really two kinds of apples, rather than apples and something else.

Chris: To the extent you can automate the documentation (TogetherJ was mentioned)

John B: Supple design assertions, what's the invariant for an apple? Those can really help you reveal what the common points of view are between two models.

Russ: Sometimes it's how the actual artifacts of the code relate drives you in a direction. What Phil was saying earlier about refactoring driving you toward better understanding of how things were being used. When the problem is too big to hold in your head.

John B: The problem is not the technical issues so much that there is my model and your model, and clearly my model is better than yours, and there are two sets of (mine, yours) in the room. The difficulties are more sociological than technological.

Russ: But maybe we aren't even at this point. We have to decide whether the models have any correspondences where we can extract some common point of view. Using diagrams, pictures, descriptions, etc, to see if common points of correspondence occur.

Charlie: I was once a "swamp guide" for new hires at my company. There were lots of questions about certain idiosyncrasies in the code which related to evolution in the company's business model without the objects being changed to reflect the new point of view.

John B: The context of "now and then".

Eric: Similar to Mike's issue with the different messaging protocols. He had to deal with the protocol based on "then", and the internal model based on "now", and facing the challenge of how to get more "now" into "then".

p. 388: a diagram grouping the "context" patterns in relationships, from SEPARATE WAYS at one end (need no control of other systems, no communication) to Single BOUNDED CONTEXT (need high control, high communication).

Russ: Even on our small team, we have three different contexts in which there's a "Book" object

Eric: (discussing the diagram) But CONFORMIST has no control and requires high communication. ANTICORRUPTION LAYER is somewhat similar. Then there's a break to a set of patterns which require greater integration, greater control over other systems and more communication about internals... CUSTOMER-SUPPLIER TEAMS, OPEN HOST SERVICE, SHARED KERNELs are partway along toward the single BOUNDED CONTEXT.

Russ: You're really nice to XML on p. 376.

Eric: I wrote that back in the glory days of XML. We're all a little disillusioned these days. I really wanted to talk about XML as a stimulus to create common languages for interchange.

Eric: The E-commerce enthusiasts for XML as the magic bullet are not realizing that the really key thing is the common model. The problem isn't getting to a common syntax. With a common model, you can reach a common syntax fairly easily.

John B: The fact that there's a common syntax, a common way of parsing it did help take some of the syntax and formatting issues out of the way.

Russ: XML helps you with the easy problems, but not the hard ones, the conceptual meaning of the marked up object. Here's a price with a dollar value. But does it include tax? Not? How are you allowed to interpret this value you've been given, <price><value unit="USD">50</value></price> ?

(brief discussion of XML web services design, what might make it supple, will good designs win?)

Eric: p. 378, Notice the UML diagram is more compact than the poem!

Mark: What about the second and fifth guy?

Eric: Oops! Guess that was enough to make the point.

Chapters 16 & 17

Domain Driven Design Final Session
Chapter 16 + Chapter 17
February 3, 2004
Guest author: Eric Evans

Russ: p. 442, "large-scale structure can save a project?"

Eric: Yes, but the read the rest of the sentence -- "... an ill-fitting structure can severely hamper a project." Throughout this chapter I express skepticism about the large-scale structure, and point out how it's less important than some of the other concepts in this book. Say the model context issues in Chapter 14. And you desperately need to distill out the part of the system that gives you the most value. And refactoring toward deeper insight, those things in Part III, that's what really rewards your use of a model. And you really need to bind the application and model together using the techniques of Part II. ANd you need to cultivate a ubiquitous language. So now we come to Chapter 16, and large-scale structure. And it is less important than any of those things.

Eric: It's better to have no large-scale structure than one that doesn't fit.

Azad: I came into a project rather late. There were clear layers in place, and there wasn't much to be done about the large-scale structure at that point. If you have 100 people working on it, there's not much scope for change.

Eric: 100 people can certainly decide to stop honoring the layers, the software structure.

Dave: But getting consensus for change is very, very, hard.

Eric: If you can't get the key people in a room, you shouldn't be doing trying to pull off strategic design.

Scott: Is the workflow of an application, the overall business process... is that a large-scale structure? I worked on some applications where that became a constraint on the design.

Eric: Yes, I guess that would be. I haven't seen that, but I could believe it as a large-scale structure. Did it help?

Scott: No, it hurt.

David: Often you need a pretty serious crisis to break out of an existing large-scale structure.

Sam: Isn't it easier to start from some large-scale structure and refactor the solution to better meet the needs of a business?

Eric: I do find it easier to move from one structure to another, that's true.

Russ: Sam has an important point here. Is it always true that having some large-scale structure is better starting point for getting to a good structure than having none?

Eric: But clinging to a bad structure...that's not a good idea. When I say it's easier to transform one structure to another, I don't mean just any structure. If it was a structure that was valuable in the past.

Russ: And minimal.

Eric: And minimal. If it was based on a valuable structure, then it provides some useful organization to base a new structure on. If something is unstructured, or badly structured, forced into an unnatural shape, it provides you no guidance, or worse, is misleading.

John B: If you're having large-scale structure problems, those are good problems to have. It means your system has grown to the point where it has a large-scale.

Scott: I had an insight when I was reading this. We were trying to layer our applications -- architecture and infrastructure. On our web server application, the layering was fine. The web application server. Now I can see different ways to structure the domain layer.

Eric: Because you have a distinct domain layer, you can consider developing a strategic design for that part of the system. And probably the web service is a newer system than the others, it's less of a mess.

Russ: In response to Robert's question...how small a system would these issues, large-scale structure -- at what point of smallness would it be meaningless?

Eric: At some level this would degenerate down to a point where there was no distinction between the domain objects and the large-scale structure. Let's think about the XP concept of a System Metaphor. That's a large-scale structure aimed at relatively small systems.

John B: What forces drive you toward a large-scale structure?

Eric: "Developers cannot see the forest for the trees". If you can look at a small system, maybe look at the 10 objects of the system and understand the whole thing, you may not need any other structure. But with 500 objects, it helps having a structure to describe the significant relationships of groups of objects. But if you group things wrongly, it's worse than no structure.

Eric: Nobody seemed to complain about my discussion of the System Metaphor as part of large-scale structure. I was expecting more criticism.

Russ: We had some heated discussions on this in an earlier draft.

Jeff: I'm not convinced that "Firewall" is a developer's metaphor. It doesn't describe the structure and components of what goes into it. Maybe it's a marketecture metaphor, or a network administrator's metaphor.

John B: There was a certain point in the development of firewalls at which the metaphor hit a wall. It used to be that you'd "punch a hole" for port 80, for port 25. Active firewalls no longer match this metaphor of holes, in the same sense. They have to understand the context of a network conversation to allow protocols like File Transfer Protocol to operate.

Russ: I'm trying to learn something about System Metaphor. For the C3 project, it helped them decide how to structure and name their domain objects and methods. Your firewall example does not exactly match up with this usage.

John B: It's not a system metaphor in the XP sense. Maybe a system metaphor for a system of machines on a network.

Eric: When I asked Ward Cunningham about it, this was the example that really gave me that 'aha'. And since its development, the structure of the security products available has been shaped by the metaphor of the firewall.

John B: I've heard Ward use "Raincoat" as an alternate metaphor. I like "Doorman", because it implies some understanding of context.

Scott: The fact that you can have this discussion proves the effectiveness of the metaphor. But I don't understand the applicability to actual application development.

John B: When someone asks me what "System Metaphor", I tell them to forget what they've learned, point them to Eric's book and say, "read Domain-Driven Design".

Russ: The POSA architectural structures are examples of large-scale structure?

Eric: Yes, Pipes and Filters especially.

John B: One of the things that the firewall metaphor implied was "inside" and "outside". "Outside" was dangerous, exposed. "Inside" was protected.

John C: So to get inside, there were Trojan horses.

John B: And since not so much has been done about (nodding to John C) capability-based security, once something bad gets in, it has a free hand.

Eric: (Layers) I'm talking about just the domain layer, as opposed to architectural layering. Organizing the domain layer in terms of broad responsibilities. All of the usual layering rules apply.

Scott: I don't know if you intended this... you went from Responsibility Layering to Knowledge Levels to Pluggable Component Framework. I'm not sure how you would get to a Pluggable Component Framework without perhaps putting the interfaces at the boundaries of Responsibility Layers.

Eric: I don't define a navigation between these distinct large-scale structures.

Scott: But as I was reading your examples... say the Policy layer in Responsibility Layering... I could see how you'd decouple that through an interface and make that a pluggable component.

Eric: In Chapter 17, say Figure 17.3 or 17.4 p. 487, there's a picture of combining these structures with bounded contexts, some of which might span layers. Maybe that is relevant to some of what you're talking about.

Russ: Configuration of a pluggable component -- actually realizing an instance of an application -- is a difficult part of the story. There are a lot of ways to put these things together.

Eric: I wanted to give a picture of how bounded contexts might align with development teams, with layers and components. If you were to take it another step from this point (layers and contexts), you might make it pluggable.

Scott: Figure 16.8, one of the things I find compelling as a developer, is that the layers of objects would be apt to change independently. I appreciate your name of "Responsibility Layers", making that distinction in the summary.

Eric: I do, actually, mean to maintain all the rules of layering, of references only in one direction. I don't want to create any confusion. The concept of layers is simple and valuable in many contexts. Perhaps you mean where I am talking about Knowledge Levels -- they aren't layers. There are references in both directions in Knowledge Levels.

John B: Where did "Cargo Repository" come from and why?

Eric: It replaced a bidirectional association between Customer and Cargo, to support the layering. For a customer to say "I have these cargoes" is to violate the layering.

John B: And why is this a good idea?

Russ: Customers don't really need to know about cargoes, in terms of their responsibilities.

Eric: In this case, what I'm trying to say is that when you impose a large-scale structure, that it will affect how you make detailed design decisions. It helps people understand the system, and causes them to make decisions that will conform to the structure (for better or for worse).

Scott: Either having the line with the layers shown, or carrying that drawing to the next... put a line in 16.5 . When you explain it, it's clear, but I had to think harder when I was studying the diagrams by myself.

John B: Why is the layering between Operations and Capability important?

Eric: It's something you learn from experience as the project goes along. If you are working on a larger system, maybe 500 objects rather than 5, you may find a natural stratification of concepts that give you layers which are useful for design. It can help you make sense of a system you can't otherwise absorb at a glance.

Sam: I kept the 7 +/- 2 rule in my head in the time, as I was working on a growing system. When I started hitting the limit, I applied some organization, by putting in organizing principles to structure the system.

Eric: Yes, you get pressed into these things. You don't seek large-scale structure in a 6-object model. But when you have a 600-object model, you desperately need ways of understanding it as a whole. All the strategic design techniques help with this problem, but the large-scale structure directly addresses it.

Robert: Or when you have the need to distribute the system? Then large-scale application structure can be very important.

Eric: That's a different motivation than what I describe. What I'm talking about here is conceptual organization, being able to understand the domain objects of the application.

Russ: How did the "Knowledge Levels" discussion come across to people?

Sam: That turned out to be the exact pattern I was looking for. I could relate to it, the dead-on solution for what I needed.

Eric: Who's read Analysis Patterns?

(Russ, Tracy have)

Russ: A long time ago. I recall it was more likely to be composition rather than inheritance.

Eric: That is a mistake in the diagrams, those are intended to be object references rather than type derivations. [An aside, urging people to report errata here when they find it.]

Sam: The interaction diagrams really helped make sense of it.

Eric: More trouble than I intended to put people to, to understand. This pattern, and especially this example, was rather late to go into the book. So not as many people looked at the diagram before it went to press.

[...]

Eric: Russ, as you said, minimalism...less is more is a good principle to apply in large-scale structure. If you try to be too comprehensive, it'll just be a dead, suffocating blanket over the whole project.

Russ: p. 481, for a second printing, if you could reword the first paragraph in the refactoring section. It seems to claim that if you have a good understanding of your domain, you could impose the structure in advance.

Eric: If you really understand it in depth, and in the context of your system.

Russ: I was worried that a banking analyst writing a banking system would mistakenly conclude that their deep domain knowledge would let them go ahead and do detailed design up front.

Eric: But if you're on your fourth banking system, you could probably get away with it. I don't think it's so important a point. The point is really, don't do it up front! Don't do it up front! Don't do it up front!

Sam: So, someone who has written 3 systems... I beg to differ. If they were 3 procedural systems. Or 3...

Eric: OK. I'll just repeat "Don't do it up front!"

Eric: Chapter 17 is short, but it's quite important.

Robert: You make the case for not doing these things up front. It all makes sense. But what do you say to managers who need some predictability.

Eric: When I first encountered XP, XP's honesty was refreshing. "We can promise you a date, but not a feature set, though we will promise you that it will do something useful." People need to face facts, to recognize unknowns. You make a plan, you make a prediction perhaps, but your business plan needs to be able to deal with disappointments. What's more, you haven't set up a process in your company that prevents you from taking advantage of opportunities. You can't eliminate all downside risk, but software development projects frequently use methodologies which completely eliminate the upside potential. How's that for a business plan?

John C: Although "lucky accidents" -- features which are easy to develop, by surprise -- sometimes raise issues with the cost to document, train, test, and maintain.

(discussion of diagram on p. 487, related to "anticorruption layer" in Figure 17.4).

John B: So if you have a legacy system straddling two layers, like in 17.4, is that a motivation for the anticorruption layer?

Eric: It adds to the badness [that the legacy system doesn't conform to the large-scale system you want to apply]. It probably wouldn't be the only motivation for the anticorruption layer.

John B: Is 17.6 an example of a HIGHLIGHTED CORE?

Eric: It's actually more a SEGREGATED CORE. I've moved the core code into its own packages. A HIGHLIGHTED CORE might have core classes scattered among a number of packages.

Russ: So there are BOUNDED CONTEXTS and there are RESPONSIBLITY LAYERS. The BOUNDED CONTEXTS are areas between which you need to limit their references and contacts, because you are speaking different languages in these different contexts, and need to negotiate the common terms.

Eric: Yes, these are two orthogonal concepts.

Russ: I'm a little surprised you talk about tight integration within a BOUNDED CONTEXT. What would you call a tightly cohesive group of classes within a context?

Eric: If the clustering is really close, I'd call it a MODULE.

Russ: I really appreciate having the vocabulary for BOUNDED CONTEXT. We did this before we had the terms, and we really had to fumble around, we would say, "let's draw a line around this set of classes here, and this set here."

Eric: I think BOUNDED CONTEXT is my really original contribution in this book. It's something that I think successful teams have worked effectively with for some time, but didn't have a word for. Probably the existing word was "scope", but this is such an overloaded term, and it most often is used at a much smaller scale/finer grain.

Patrick: That'll be your middle name, Eric "BOUNDED CONTEXT" Evans.

(Six Essentials for Strategic Design Decision Making)

Russ: What will you do... what can you do if you have a team that doesn't necessarily feel ready to be generalists, to operate in all the roles you describe?

Eric: All of these are essentials, each of them based on observed lacks in projects in my experience. If you are missing any of these essentials, you can go ahead and develop and produce software, but you will not have effective strategic design.

Scott: (describes a problem with a bi-located development group)

John Brewer: This sounds like a team problem, not really a design problem.

Eric: I agree with John basically. Strategic design does give you some options to help deal with it. The pure design solution would be to separate the two teams' parts of the domain, to create separate BOUNDED CONTEXTS which are related via customer-supplier or other relationships.

Russ: Or sharply defining what's inside and outside the SHARED KERNEL.

Eric: Do you have a UBIQUITOUS LANGUAGE that everyone uses to communicate changes during your daily phone calls? To introduce new terms if necessary? But that requires conversations.

Scott: We capture some of that via code, some via conversation. We have a Wiki, but it's less useful than we'd hoped.

Eric: You may wish to look at p. 37-38 on what makes a design document useful.

Dave: We've effectively used a digital snapshot dropped as a picture onto the project Wiki to keep design documents up to date with little effort.

Russ: Are there cubes? Offices? Shared workspace?

Scott: Both. Mostly people work in the shared space.

John B: Maybe you should spring for two Polycom speaker phones and keep an open phone connection throughout the day, to keep communication going..

Eric: So, for the case of technical framework development, I added one more principle, which I wanted to emphasize: "Don't write frameworks for dummies!"

Russ: Anything you'd like to leave us with?

Eric: from the conclusion, the section called "Looking Forward" [short reading from page 505]. I want to point out that working in the domain layer has significant, interesting technical challenges. There are a hundred little tricks of the trade, and if you master them, you can move into new areas, and you have these tools in hand, you can rapidly learn the new area and create supple, useful models. Domain-driven development is a specific skill that has value, and a discipline that people can learn to be better at. And people should be selecting and hiring for it.

Russ: Then we can bring in some infrastructure geeks...

Eric: Yes, and some electricians to make sure there is good lighting in the workspace.

Eric: The core, the heart of the system is that domain layer. People who know how to create that stuff should be the most valuable people on the project.

John B: I think it's the third big O-O book, after Design Patterns and Refactoring.

Charlie: Maybe the second.

John B: Chronologically speaking.

Eric: I'm curious, I almost wanted to ask for a secret ballot... sometimes when I go to a discussion group, I've had a chance to read the material, sometimes I haven't. To satisfy my curiousity, perhaps if people could write down a percentage of how much of the book they've read.

(Tracy carries Jeff's hat around collecting scraps of paper for Eric to count and satisfy his curiosity on this point)

John B: I wanted to say some good things about p. 479, the AIDS Quilt example. It's a large-scale structure where you're really strict about a few things and very lax about others. It enables the whole thing to work well.

???: But quilt pieces can be anything. I don't see any large-scale design.

Eric: Each piece has to be a precise size, it has to be edged a certain way, and it has to be foldable. It is a minimal set of rules that makes it possible to take all those independently created quilt pieces and fit them together in any arrangement.

Attending:

John Corbett
John Brewer
Dave Smith
Azad Bolour
Darlene Wallach
Ken Scott-Hlebek
Robert Benson
Tim Huske
Wayne Vucenic
Russ Rufer
Charlie Toland
Tracy Bialik
Carol Thistlethwaite
Jeff Miller
Patrick Manion
Walter Vannini
Sam
Scott
Kevin