Larry Garfield – Software Management Lessons from the 1960s – phpday 2019

as we mentioned my name's Larry Garfield you may know me online as Krell if you wanna make fun of me on Twitter that's where you do so I highly encourage it I'm the director of the developer experience at platform Sh we're a continuous deployment cloud hosting company I'm also a member of the PHP fig core committee everyone familiar with fig it's kind of like the United Nations of PHP with all the good and bad implications that has if you want to know more we are having a open meeting today right after lunch in track three so if you're interested in fig it's work what we have done what we are doing stop on by we actually have elections open right now for the next round of core committee members and for those who get the joke I am in fact psra compatible who actually gets that joke and I never know a couple of people okay this is a computer this is a very old computer specifically an IBM system/360 model 20 system 316 was originally announced in 1964 and it was intended as the world's first complete range of compatible computers what does that mean before this every model of computer was unique every individual model of computer had its own architecture its own system its own programming and software was not portable between them in the slightest this was the first time that you had a line of multiple computers of different builds that you could run the same software across multiple models this is really cool this is a really good innovation in order to make that work you needed to have for the first time a distinction between architecture and implementation you needed to distinguish between how the system is designed to interface and how the system actually works down at the microchip level system/360 was developed over the course of two years because software projects and technology projects always run late but it shipped for over a decade there's one of the most successful lines on computers ever made and this you split between architecture and implementation this flexibility was incredibly powerful and incredibly useful for example it compatible with the standard reel-to-reel you know tape drives they've got the same direction reel-to-reel tape storage systems as well as this crazy newfangled technology called spinning discs that could store thousands of bits of information in something only about this tall it's really cool that give the data density these things had system/360 had a huge impact on the technology landscape for example 8-bit bytes yr bytes 8 bits in size because system/360 decided they were before that it was common for different computers to have different size bytes you don't think of that anymore but bytes used to be variable size and once fun story the most common standard before this was 6 bits to the byte which didn't actually give you enough characters to have both uppercase and lowercase letters so designers working on those systems that decide all right do we have all uppercase or do we have all lowercase and they did research and found out that all lowercase is easier for people to read than all uppercase and they took this to their manager and the manager said what we're ABM we can't have our name in lowercase and so the state of does all uppercase for the next 30 years because you go but yeah that 8-bit byte comes from system/360 the idea of addressing memory by byte offset only in many earlier systems you could in fact address memory down to the bit level which is not actually a good thing because you could do some really weird things then and weird in a bad way it also established Obsidian OTT heard of because no one else used it and they use ASCII instead so well can't win them all system/360 still lives on however in the IBM z-series mainframe computer which is still backward compatible for a lot of the original programs written in the late 60s our any of your computer is compatible with code written 40 some odd years ago and by the way these things do run PHP that the lead team on this project was these two men G Nam doll and Fred Brooks Fred Brooks is actually the one who coined the term computer architecture in the first place after the project was over they the team wandered off to various other endeavors other projects many of them ended up in academia like you do and 10 years later Brooks wrote a book called to the mythical man-month and updated it 20 years later in 1995 that's what this picture is from mythical man-month is a collection of essays by Brooks essentially lessons learned from managing large software projects both the system/360 and others drawing on both his own experience and academic research at the time of how does one manage large software projects and make them not be a disaster it's kind of useful the new edition included a number of additional chapters including you're looking back on his earlier predictions their earlier statements around how you should run software projects to say so these things I was saying are they still true and his conclusion was kind of yeah this they're still true because humans haven't changed technology's changed humans haven't changed so it spend another 20 ish years since then what I'd like to do today is ask are these recommendations are these conclusions are these guidelines he has still valid let's find out now I'm going to go through this in a some different order and structure than Brooks does I'm actually going to cover only about half the book one thing that I will note he doesn't get right is extreme overuse of male pronouns for no good reason so ignore that part please but I think we'll find the rest of what he says holds up pretty well who's heard of Brooks law before few people who's heard of this law before Wow not enough people well now you have adding people to late software projects makes them later why there's a number of reasons one communication is hard let's be honest why is communication hard because the more people you have the harder communication becomes because the more people you have the more communication channels you have when you only have two people on your team you only have one conversation to have when there's three people on the team there's three conversations four people there's six conversations five people there's ten conversations and so on and so on and so on the number of pathways you have to keep track of goes up faster than the number of people on your team and there is in fact a mathematical formula for this that's what it is don't don't bother memorizing it that's not important what's important is the fundamental idea the more people you add the more conversations you have to have and the longer it takes to just keep everyone on the same page just coordinating people is harder the more people you have it's also a problem the tasks are usually not parallelizable it would be great if you could just split a task up into a million pieces do it all quickly and be done most tests don't work that way that this is a management talk so I have to have a Dilbert strip in it I hired all of you because the project will take 300 days to complete there are 300 of you so I want you all finished by five o'clock you're fired doesn't actually work that way does it although I'm sure there are managers who wish it would it doesn't actually work that way why and it works this through for unskilled tasks – why because sometimes you have limited resources you may not simply have enough computers for 300 people you may not have enough chairs for 300 people you may not have enough server capacity for 300 people you may have different skill levels the person who is really good at SQL and the person who is really good at CSS are not the same person and you can't just throw 100 SQL developers at CSS and expect it to get done quickly it doesn't work that way believe me after I you can also have tasks that block others you know this task just you can't actually write something that uses this API until the API has been designed and built maybe you can mock stuff out but that's not actually going to give you a real experience of using that API you're not going to know what the timing is like you're not going to know what the latency is like a couple of years ago I was sapling friend I'm from Chicago and I was helping some friends move from Chicago to New York and so you know like you do they had a bunch of their friends over ordered a bunch of pizzas and we packed up their apartment to put it all into a moving van and they had somewhere around a hundred boxes or so which means they need a hundred friends and they'll be done in about five minutes right who's helped friends move before does it ever work out that way why we have one elevator in the building we only have two dollies you can only fit so many people into the moving van at the same time the people packing the boxes need to know what things can go on top of what the people packing the moving van needs to be the same people are going to be there to unpack it at the other end so they know which boxes are which even if they're labeled you still need to keep track of which type of boxes which just fundamentally you can't just throw people at the problem even when it is a basic manual labor task one thing we never did figure out though is why anyone would want to move from Chicago to New York but I'm biased quite simply people and months are interchangeable commodities only when a task can be partitioned among workers with no communication among them because once you have to communicate you have to coordinate and in fact adding more people to a project increases the cost at times more Dilbert how long will this project take if I add two more people well at one month for training add one month for extra complexity and one month to deal with their drama because people why when someone joins the project they know nothing it's gonna take time someone else's time to train them and teach them on the project tell them how it works where the bodies are buried explain how the code works or the parts that aren't documented because of course they're parts that aren't documented even if you do a good job of documenting it still takes time to learn the system and then of course you know that that time is time that someone else is not spending working and then of course there's their drama because more people more communication more opinions that keep track of and so on so can you add people sometimes I would argue you know if you add people or project early then you can absorb that additional cost and they can ramp up and by the time they're ramped up they can actually be useful it really depends on how early you add them and the number of swimlanes on a project swimlanes is a concept out of agile and roughly corresponds to the number of parallel parts of the code that you can work on without bumping into each other and different projects will have a different natural number of swim lanes sometimes the swim Lane is one sometimes it's two or three very large projects could have swim lanes of seven or eight it's the number of times you can have different people working and their commits are not going to bang into each other basically I would argue the number of developers you want on a project is the number of swim lanes plus one that gives you extra capacity for someone to go on vacation someone doing code reviews someone to get sick that's the maximum number of developers you want on approach it's a very large project with a lot of swimlanes maybe you could add this swimlanes plus two but adding more people in that is just gonna make things slower things just take time now note I'm saying here a number of developers not number of people involved so it's not counting your project manager your product owner and so on the number of people touching the codebase at a time is the number of swimlanes plus one this is even found in hardware – this is on dolls law same genomes all we talked about before and again ignore the actual math the point being this is a an equation for the number of CPUs you can throw at a problem before you're not before you get diminishing returns and the more paralyzed about the product or the problem space is the more CPUs you can throw at it and actually get benefit but eventually even with something that is 95 percent parallelizable project that the task is easy to split up into lots of different pieces throwing more than about a thousand CPUs at it you're not really getting any benefit because there's still coordination overhead of keeping track of all of those different pieces and synchronizing them the vast majority of software projects the process of actually writing them are nowhere near the 95 percent parallelizable they're more like 5% parallelizable so what do you think true still true 40 some odd years later yep I'm gonna say yes just totally fits my experience I mentioned before I'm only giving up talking about half the book or so the first time I gave this talk I was whining on Twitter like you do that there's just too much material in this book to give in one talk so someone helpfully suggested that I get a second speaker on stage to give a talk on the other half of the book at the same time yeah of course we all work most of us work in open-source in some capacity or use open source projects in some capacity do we know any projects that like to solve problems by just throwing warm bodies at a problem I used to be a Drupal developer I can talk about them oh dear what's the combinatoric number for that what should we learn from this next up who likes doing estimates that's what I thought why are estimates always so wrong there's a lot of reasons but one of the key ones that Brooks talks about is this when you as a developer give an estimate you're giving an estimate for this for a proof-of-concept that handles the happy path that's what your brain thinks you're looking at how long does the the basic version take if you actually want a product out of it which means something actually useful that handles all the edge cases the error cases as well test it it's good documentation you're looking at three times as much work three times as much work as producing the the proof-of-concept to turn it into something actually real and useful if you want it to be something others can build on you want to be extensible you want a programming system that people can leverage that takes three times as much work as just building the base proof of concept which means if you actually want a real stable product that people can use that's you know tested and documented and handled all of its air cases you're looking at nine times as much effort as the proof-of-concept in your garage type project nine times as much work you don't estimate for that because you don't think about those you're a developer you think about this part Brooks also notes that a substantial Bank of test cases exploring the input range and probing its boundaries must be prepared run and recorded in order to have a viable stable program programming product what does that mean it means automate the testing was already thing in 1975 what's your Excuse note this does not mean that it takes three times as long if you're writing tests it takes three times long to find all the bugs that tests find to fix all the bugs tests fine for you the bugs are gonna be there no matter what it's a matter of finding and fixing them via tests I also want to know how fast can you code just think about that how fast can you produce code you're wrong because I didn't give you enough information extrapolation of times 100-yard dash show that a man should be able to run a mile in under three minutes the current world record is three minutes 43 seconds because you cannot maintain that same speed for an entire mile you can't sprint a marathon in fact what's research found is that the amount of effort to produce more code is based on a power of the number of instructions already in the codebase translation the more code in the codebase they'll hang longer it takes to add more code to it it is harder to add more code to a larger codebase why because there's more things that can go wrong there's more possible interactions this is true even if there's no communication between people there's more interaction points internally in the code your brain can only hold so much code in it at the same time if you want to figure out okay what's the impact of this change the scope that it may impact the larger that is the harder it is for you to reason about which means you're going to get it wrong and introduced bugs you have to go fix those two bigger code bases are hard to add more code to in fact one study found that's I'm not quite sure which very few someone many mean in this case but a person can add ten thousand instructions to a code basement is a very few interactions for their definition very few if there's a decent number of interactions they can only add about five thousand instructions per year if there's an awful lot of interaction or almost an order of magnitude fewer the more complex the code base the slower it is to add code to it now what does large mean here in this study large meant twenty five programmers and thirty thousand instructions ish this would be a good indicator you want decoupled code and I set instructions here and someone's probably gonna call me out and say but but but that's you're talking about assembly here instructions what about in real code later studies found that in PL one which was kind of the C language of the 70s the same curve happened you are the same equation for a number of statements instead of a number of assembly line instructions what does that mean it means a more expressive language does give you more productivity you can get more done in a more expressive more powerful language than in a less capable language this is true but no matter how good your language the larger the code base the slower it is to add code what do you think true this is feel right seeing people nodding yep I'm gonna agree totally right one catch though we said large is twenty five programmers and thirty thousand instructions hmm how many lines of code are in symphony what should we learn from this who likes writing documentation ok sponsors looking to hire the people there hands up or the ones you want to hire planning and documentation Brooks argues are the same thing these go hand in hand you want to be writing your documentation upfront why because only when one writes to the gaps appear and the inconsistencies protrude the act of writing turns out to require hundreds of many decisions and it is the existence of these that distinguish clear exact policies from fuzzy ones huh your rubber ducky with your word processor this happens to me all the time I used to work as a consultant and all the time I go to a client on-site with my team we'd spend three days talking to them collecting information figuring out what it is they want what does they need not the same thing you come up with a plan we go back to the office I'd sit down and start writing it out and planning it in a Google Doc describing in prose what we're gonna do and describing in you know architectural diagrams what we're gonna do and every single time I came up with we didn't talk about this edge case what should we do here what happens when there's no data here what happens on leap years here because they kept asking us to do calendars and dear god I hate calendars writing down and trying to explain back to a document what do these you're going to do helps you figure out that you don't know what you're going to do the crucial task is to get the product to find many failures concerned exactly those aspects which are never quite specified if you don't know what you're going to build you can't build it well if you haven't tried to explain in detail what you're going to build you don't know what you're going to build but that manual or written specification is a necessary tool but not a sufficient one the code is your formal definition and that's a precise description of what you're doing it does not tell you why no matter how well written your code is it does not tell you why it is that's what documentation is for that the planning is for rooks argues for top-down design what does that mean architects this is the system first end-to-end break that architecture down into discrete pieces and refine those layer by layer and keep recursing until you have a plan for how your system is going to be built then you can implement each individual module that you've come up with debug them individually so that you can actually find bugs in them and as you are developing you can inform backup and modify the architecture as needed based on your experience you know building each piece basically divide-and-conquer which is the standard approach to anything in science or math which is programming when you have a problem break it down to smaller pieces that are easy to solve and put it back together again he also says plain to throw in a way you will anyway version 1 is always crap why you don't actually know what you need to do until you've done it wrong at least once – sure the old joke that it's throwing here from Microsoft or make fun of them all right there's an old joke you know Microsoft doesn't get anything right until version 3 yeah that's not just Microsoft they they the joke but that's everybody because yeah your first version you don't know what you're doing yet at this point I'm sure there's someone in the room who is going to go Larry you're talking about waterfall we all know that's terrible and in the second edition Brooks said yeah the kind of his waterfall isn't it that's not great all right let's let's tweak it design top down but implement iteratively what does that mean start with once you have your plan start by building an end-to-end skeleton this is a not necessarily complete but functional and we'll compile system it doesn't necessarily do anything useful or much useful but it does give you a baseline and then always make sure it works always be compiling or in PHP case running whatever and once you have this working baseline grow modules into place implement new functionality replacing stubbed out pieces and as you go and make sure that at every stage of the way you have a system that may not be useful yet but it still runs and still pass the tests if you need to just rip out a piece and throw it away cool do so if you need to tweak a piece cool do so now note building the skeleton could take a long time this could be half your project just building that baseline that's okay this should look familiar to people it's called refactoring refactoring is the art of throwing it away throwing your first version away a little bit at a time but if you still have a complete running system or a functional running system at least the entire time but this approach necessitates top-down design for this top-down growing of the software building individual pieces in isolation and then hoping that I'll work when you pull them plug them together is not gonna work out well building the framework and then plugging pieces into it as you go that works a lot better that said common sense if not common practice dictates that one should begin system debugging only after the pieces seem to work architect top-down debug bottom-up unit tests first in other words now this this should also look familiar and the same person as before is probably in start shouting but Larry isn't this agile who's seen this picture before in some form who thinks this is actually a good picture this is terrible this never works who here is managed to successfully build a bicycle into a car I never have in this setup you are in fact throwing it away twice you're throwing away your entire system twice don't do that don't do a complete rewrite we know those are a bad idea instead you do this you don't start with a useless individual piece you don't start with something that is not going to work for what it is you're trying to build you start with a baseline that functions it's not done but it functions and then you can add pieces to it and add pieces to it and change the paint and change the hubcaps and possibly change the top but you're a frame your baseline of the system hasn't changed you start with a skeleton that supports your end goal and then build from there with decoupled components so that you can swap out the wheels when you need to nope this means you are not going to build the most user important feature first you're going to build the foundation first if you're building a house and the customer says the most important thing for them is to have a fireplace on the second floor are you going to build a fireplace in the second floor first no you're gonna start by building the basements and pouring the foundation because if you don't you won't have a second floor on which to put a fireplace same in software the most important thing that the function how the user wants is not the first thing you build the first thing you build is the baseline that's going to let you eventually build a good thing the user wants what do you think true see people nodding yep I'm gonna give Brooks to four three two and a half out of threes at this point because he had to revise it a bit Brooks gave a presentation in 1986 at a conference called no silver bullet which turned into an essay in the updated version of mythical man-month in which he talks about how there is no silver bullet that is going to fix all of our productivity problems anymore and Brooks draws a distinction between essential complexity and accidental complexity essential complexity is this problem space is just hard they're just the problem space itself the logic business logic itself is complicated as opposed to accidental complexity which is my computer is too slow and so I can't work very effectively who's worked on an e-commerce system of some kind yeah ecommerce is hard why is e-commerce hard not because the e-commerce software is bad some of it is but that's not the point ecommerce itself is complicated because tax law is complicated because shipping is complicated nothing you do will make those not complicated you can only paper over that and make it look easy but it's not accidents the complexity is my software is crap which also happens but that's a different matter the hard part of building software is the specification design and testing of the conceptual constructs not the labor of representing it and testing the fidelity of the representation the design is the hard part remember when Brooks was working on system/360 the way you tested your code to see if it worked is you took your stack of punch cards walked down the hall to the elevator went down five floors to the basement where the room with the computer was but he only had one handed it to the computer operator who would put it in line and eventually put it into the feeder and go through the computer and if it barf and you come back three hours later and they tend and says it aired on card 417 have fun and you carry the stack of punch cards back up to your office and stare at card 417 while you figured out what was wrong with it we are a long way from that debugging by punch card is not what any of us doing anymore I certainly hope and in here it still deluged by punch card good you don't want to there is no single development which by itself promises even an order of magnitude improvement and productivity reliability or simplicity there's room for improvement but we are not going to get and productivity improvements anymore that's on the level of replacing stack of punch cards with computer sitting on my desk with an IDE you that I can just step through individually there are still improvements we can make programming techniques programming languages and so on but order of magnitude something's gonna make us ten times more productive those don't exist anymore we've already handled all of low-hanging fruit so how do we attack essential complexity how do we we've solved the accidental complexity enough how do we make the actual fundamental problem we're trying to solve easier when Brooks offers a couple of options here rapid prototyping where we grow a system organically based on user feedback so we don't guess wrong buy versus build and mentoring better architects we already talked about the first one so I'm not going to cover that again survive versus build Brooks points out that the most radical possible solution for constructing software is to not construct it at all well think about it in the 60s and 70s a computer cost twenty million dollars spending fifty thousand dollars sameen euros both you didn't have euros lira on custom software for it is a rounding error in your budget you don't even notice your accountants can't find it these days you can get a computer this big for 10 bucks that is a fully capable development system just plug it into a monitor I have one it's kind of you the software is not the expensive part if you want to save money on your system if you have to write less software how do you do that by reusing software and their components why would you get if you need a spreadsheet why in the world would you write a new spreadsheet you got it cell you've got schools sheets you've got I think there's something else still on the market it's a solved problem no one in their right mind is going to write a commercial spreadsheet from scratch or custom build one for yourselves if you need a CMS why would you write a CMS completely from scratch at this point when there's 30 on the market just in PHP that are all battle tested and used by literally millions of people if you need a web server you've got Apache about nginx you've got iis you've got the standard library and go why would you write your own web server from scratch you wouldn't it's easier to just take these pieces off the shelf and use them this sounds freely to anyone it should Brooks just described open-source he didn't use the term it didn't exist yet but that's exactly what we just described the way to be more productive is to write less code it means the way to be more productive is to reuse more code open-source is the number one productivity improvements of the last 20 years this gigantic library of code we can share and reuse and repurpose as needed nothing in the last in the entirety of my career has had a larger impact on improving developer productivity than the ability to share code easily and effectively if you are not using at least 90% open source in your applications you are burning money you are simply throwing money away the other thing that Brooks recommends is growing great designers because software construction is fundamentally a creative process no to shift here architect to designer from the 70s to the 80s study after study shows the very best designers produce structures that are faster smaller simpler cleaner and produced with less effort the difference between the Great's and the average approach an order of magnitude oh my god 10x developer discrimination privilege bubble bubble you know no one is going to you know if the great developer is not going to be able to produce 10 times as much code per hour as a junior you and if they do you really want to see that code but a great designer will be design a system that requires 10 times less code to be written in the first place or code that is 10 times less likely to have secret hidden bugs in them ok software architecture is much more like graphic design than it is an engineering discipline there's a gestalt to it that you simply don't have in engineering architecture is a separate skill set from programming you can have someone who's a great programmer and only mediocre architect and someone is a great architect and kind of a so-so programmer and Brooks argues architects designers need recognition on equal level with management it's not engineers and architects and managers you have a technical track and a management track and they have to be parallel with each other and whatever that means in your company in terms of salary in terms of prestige in terms of who has corner offices the technical leads should be at the same level as management because they are just as important as a good senior level manager how do you do that through mentoring identify candidates early on who seem to have good design and architectural skills who may not be your senior engineers this is a different skill set someone can be an OK engineer and a really good architect if you train them properly really good at solving the big skin large-scale problem give those people a career mentor active mentoring give them someone to work with who can help train them give them apprenticeships give them formal education in software architecture at classes at given solar work to work on senator conferences like this one where they can collaborate with other designers on their other architects in the field all of these are things the design world does the graphic design world the product design world they do all of this kind of stuff why don't we we should because this is how you build better designers who can then produce a better system true yep yep he's right on this one too and finally Brooks is strongest point conceptual integrity the conceptual integrity of the product as perceived by the user is the most important factor in ease-of-use as perceived by the user you want the user to have one mental model that they have to keep track of only one set of nouns they have to keep in their head at a time if they have to think about the program one way over here and it definitely over here they are going to get confused which means you need to have one mental model throughout the entire system ease-of-use is not just a question of how little documentation do I need to read ease of use is a factor of how much you can get done versus how much conceptual complexity I have to think about in order to get it done a complex 3d rendering program like blender or light wave is an incredibly complex piece of software it takes you need to take a class to learn how to use this stuff it's not something you just pick up yourself in an afternoon but the stuff you can do with it I mean have you seen that those amazing animated films that Marvel has been putting out you know The Avengers movies those are three quarters computer graphics and they look fantastic those have a high ease of use for what you can produce versus the complexity at the other end Windows notepad has about three functions in it and entirely this is what all it does but you can learn it in about four seconds am I saying that notepad and blender have the same ease of use maybe relative to what you can do with them insert joke about vim here a lot of Englishman's the room I guess simplicity and straightforwardness however proceed from conceptual integrity that ease of use these you want to lower that complexity in order to make it easier to use in fact the conceptual integrity Brooks argues is the most important consideration in system design it is better to have a consistent clear design than to have more features see also iPhone the original iPhone sucked from a feature standpoint didn't have copy and paste didn't have an app store didn't have high-speed Internet it had 3G if you were lucky maybe the few things that did however it did extraordinarily well and they sold millions of them for a reason but conceptual integrity the product not only makes it easier to use it also makes it easier to build and less subject to bugs that same one conceptual model concept applies to the developers as with just as well as the users if you want your developers to be able to keep track of the system they're building they should have one mental model one conceptual picture of how the system works one set of nouns one set of ap is that they end up using rather than seven that are kind of slightly different you may have heard this term from the DDD people ubiquitous language same concepts just a bit older so how do we have a good conceptual integrity in the system Brooks argues you have that through smart division of labor what does that mean the design must proceed from one mind or a very small number of residents agreeing mines you want to have one clear vision that is directed for how the system works how do you do that Brooks offers two possibilities when it's a surgical team where it's modeled on actual surgical teams in hospitals where you will have a surgeon working on the patient and she's standing there cutting and doing whatever else and there's someone over here who say you know give me a scalpel scalpel will give me a sponge sponge ham sandwich ham sandwich you have someone over here who's making sure the lights are just right you have someone over here who's wiping sweat off the surgeons brow you have someone over there who's keeping the clients away I mean the family away so that the surgeon can do everything with zero distraction I've never actually seen this happen in software so let's not talk about that again it would be a nice idea but I've never actually seen it work the other is two splits the architect from the implementer have someone architect and someone else build that architecture this is not mono think but it's avoiding conflict think you have one or a very small number of people who design the system and then you have people who build the system and those are different people the people just building the system don't get to redesign it on the fly as they go because that is how you get a mess the architect is the users agent it is their job to bring professional and technical knowledge to bear and the unalloyed interest of the user as opposed to the salesman fabricator and so forth users agent the architect is there to make sure the system works and is understandable to the end user I'm sure again that same person is going to say but but Larry that's Cathedral design that's aristocracy that's top-down Cathedral in the bazaar we want the bazaar cathedrals are still standing you've been thrown oh you've got a number of really nice entire cathedrals I've seen someone they're fantastic those has an architecture that was planed out from the beginning and took 500 years to build and they followed the plan yes the art varies over time but the general architecture of the building was relatively constant for centuries and that's why they're still standing and why they look gorgeous I personally love cathedrals are just beautiful for creations look at Apple Apple has dozens and dozens and dozens of designers very good designers but at the end of the day Johnny Ives is calling the shots when is Apple's brand design image it's what Johnny I've says it is Google has thousands of people doing design and developments and and programming and at the end of the day Matias Duarte is the one who designs material design ties to arte is the one who says this is what Google's brand will look like and so you get a consistent brand out of it if you're working with any modern UNIX system Linux BSD Mac you're using the POSIX standard why because that's the standard if you don't follow that standard things don't work right you don't like that standard that's nice your codes not going to work right unless you follow the same standard as everyone else when someone doesn't follow the standard you see why it's a pain in the button we work on the web you're following HTTP if you don't follow HTTP things break you don't like no HTTP caching caching works that's nice if you don't follow that caching standard you're not going to get any good caching or worse you're gonna get buggy cashing in us that's a bad place to be these architectures you inherit and you work with because that is what the architecture is and if it's not followed things don't work the architect is responsible for designing watch dials and hands this example Brooks uses the implementer figures out bells and gears both of these are fun both of these are interesting challenges but a good architecture supports both anyone have a rotary watch still with watch hands few people exact same mechanism in that watch runs Big Ben runs astronomical clocks the various cities have building giant clocks is still a fun challenge believe me but the basic architecture of New York hands moving at different rates was set down a long time ago and you still follow it but the architect must always be prepared to show an implementation of a feature described but not dictate the implementation you have to able to show you developers I am designing something for you that I know it's possible to build I'm not just you know giving telling you them creates anti-gravity or something but now that we've said the case this is what we're gonna build to how that's the implementers job here's the parameters go build it I can't tell you exactly how you should do it that's your job to figure out that's your responsibility not mine that's your fun place not mine this also reduces the communication channels instead of having everybody talk to everybody the architect talks to each developer and developer talks to the architect and now you only have n plus one that cannot change the communication that you have to keep track of because the architect is the keeper of both the architecture this also lets you divide and conquer larger systems if you have an especially large system you have subsystem for their own architects their own keepers of the design and they coordinate with the senior-most designer senior-most architect and then the developers work in their own areas see also Linux the Linux kernel linus does not review every line of code language reviews aggregate patches sent to him by subsystem maintainer x' who review every line of code in their subsystems is this a hierarchy yes it is and it works democratic architecture is known as mud I have worked on this system before it was painful I've tried to fix the system before it is even more painful this is you know it does require an ongoing collaborative environment so you can't have an architect to just designs and walks away the architect must be there for the whole project available to help quick things because you are going to need to take things do we know any projects that have no clear organization and no clear person in charge no clear architecture that end up being kind of a muddled mess of inconsistency what can we learn from this how big does a project need to be before all of the stuff we're talking about matters after teaching software engineering lab more than 20 times Brooks came to insist that student teams as small as four people choose a manager and separate architect for manager architect to implementers either the manager or the architect can be the lead person in charge either of them can be the boss doesn't matter as long as everyone agrees which one is which works either way but a team as small as four needs to have someone coordinating the vision because as soon as you have three or four people you have disagreement you have cliques you have different people just making different assumptions than each other and those are going to bite you and that's when you need to have someone who is the keeper of the design so what should we learn from all of this software construction is a creative process it is much closer to design than it is to engineering when in doubt divide and conquer is your friend this is how you solve most problems in any scientific field how do you do that you want to build shareable programming system products leverage open source buildin components that you can swap out you can involve independently of each other build divide and conquer into your architecture be that way when not if system requirements change you can adapt more easily decouple your libraries from your framework so that you can do that you want these independent libraries that you can release that you can swap out that you can replace that you can test into but you still want to have top-down design so that the whole system has a coherent consistent vision behind it consistent architecture behind it and to do that you need to empower architects to make decisions you have a person whose job it is to communicate with everybody to talk to everyone to take in input from the developers and decide this is how we're doing it this is the standard we're gonna have this is the approach we're gonna have this is the architecture we're going to have and have that stick back them up if you just let everyone do their own thing and architecture emerges the architecture that emerges will be a pile of mud fundamentally the idea that people knew a thing or two in the 70s is strange to a lot of young programmers from Donald Knuth father of the analysis of algorithms and the author of the book the art of computer programming what can we learn from this that's seemingly [Applause] you