Today's Lecture

• Brief introduction to Extreme Programming (and how it relates to your lab section)
• No Silver Bullet by Fred Brooks
• Brooks' reflections on it that he wrote nine years after the original

Extreme Programming

• Extreme Programming (XP) is a software life cycle that belongs to a class of life cycles known as Agile Development
  • A software life cycle defines the stages of a development project and typically includes the following activities
    • Requirements
    • Design
    • Implementation
    • Test
    • Deployment
    • Maintenance
• Agile Development is a response to more traditional and heavyweight software life cycles (such as the waterfall method)
  • It advocates values such as “deliver early and often”, “welcome changing requirements”, and “measure progress against working software”
• Extreme Programming is an “implementation” of agile development that advocates certain processes and techniques to help developers create software in an agile manner.
  • In this class, we are going to look for opportunities for you to apply in lab some of the techniques that extreme programming advocates

Extreme Programming Practices

• Extreme programming advocates fourteen standard practices; we are going to focus on two of them this semester
  • Pair Programming
  • Test-Driven Development
• Pair Programming
  • All production code is written by pairs of programmers working together at the same workstation
    • One member drives the keyboard and writes code and test cases; the second reviews the code, looking for errors and possible improvements, while also thinking strategically about what the team needs to do next
    • The roles will switch between the two frequently
  • Pair membership changes once per day; so that each programmer works in two pairs each day
    • this facilitates distribution of knowledge about the state of the code throughout the entire team
  • Studies indicate that pair programming does not impact efficiency of the team, yet it significantly reduces the defect rate!
    • [Laurie Williams, 2000; Alistair Cockburn, 2001; J. Nosek, 1998]
• Test-Driven Development
  • All production code is written in order to make failing test cases pass
    • First, we write a test case that fails since the required functionality has not yet been implemented
    • Then, we write the code that makes that test case pass
    • The time between writing tests and writing code is very short; on the order of minutes
  • As a result, a very complete set of test cases is written for the system; not developed after the fact

Test-Driven Development in Detail

• The life cycle of test-driven development is
  • Quickly add a test
  • Run all tests and see the new one fail
  • Make a simple change
  • Run all tests and see them all pass
  • Refactor to remove duplication ← this is where the magic occurs
• Example
  • Consider writing a program to score the game of bowling
  • You might start with the following test

    public class TestGame extends TestCase { 
        public void testOneThrow() {
            Game g = new Game();
            g.addThrow(5);
            assertEquals(5, g.getScore()); 
        } 
    }

  • When you compile this program, it fails because the “Game” class does not yet exist
  • You would now write the Game class (remembering to always keep individual step simple):

    public class Game { 
        public void addThrow(int pins) { 
        }
    
        public int getScore() {
            return 0;
        }
    }

  • The test code now compiles but the test will still fail. In test-driven development, to keep things simple, we get the test case to compile before we get it to pass
  • Once we confirm that the test still fails, we would then write the simplest code to make the test case pass; that would be

    public class Game { 
        public void addThrow(int pins) { 
        }
        
        public int getScore() { 
            return 5; 
        } 
    }

  • The test case now passes!
  • So, where is the duplication? How do we “refactor” this to smarter code?
    • The duplication in this instance is the “5” that exists in the test code and the “5” that exists in the program code. No matter how many times you throw the ball, our Game class will always say that the score of the game is 5.
  • So, to move forward, we need to write another test case that fails.

    public class TestGame extends TestCase { 
        public void testOneThrow() {
            Game g = new Game();
            g.addThrow(5);
            assertEquals(5, g.getScore()); 
        }
        
        public void testTwoThrows() {
            Game g = new Game();
            g.addThrow(5);
            g.addThrow(4);
            assertEquals(9, g.getScore()); 
        } 
    }

  • The new test fails, and we would be forced to add a variable to our code to get it to pass.

    public class Game {
        
        private int score = 0;
        
        public void addThrow(int pins) {
            score = score + pins;
        }
        
        public int getScore() { 
            return score;
        } 
    }

  • To move forward, you would then write another test case that is sure to fail, such as:

    public class TestGame extends TestCase { 
        public void testOneThrow() {
            Game g = new Game();
            g.addThrow(5);
            assertEquals(5, g.getScore()); 
        }
        
        public void testTwoThrows() {
            Game g = new Game();
            g.addThrow(5);
            g.addThrow(4);
            assertEquals(9, g.getScore()); 
        }
        
        public void testSpare() {
            Game g = new Game();
            g.addThrow(5);
            g.addThrow(5);
            g.addThrow(3);
            g.addThrow(1);
            assertEquals(17, g.getScore()); // note: current implementation will produce 14, not 17
        }
    }

• Benefits of Test-Driven Development
  • Develop tests as you go, use them to gauge the “health” of your program; when you are finished with implementation, your test suite is pretty much done
  • Code with confidence; if you do something wrong, tests will start to fail, alerting you to the problem right away
  • Writing test cases before code causes you to really think about the public interface of your code. Does your class provide methods that make sense to the developer who has to use it.
  • Writing tests early in development can cause you to catch bugs sooner allowing you to reduce the number of defects that appear in the final system

Applying XP in Labs

• Other than the first lab, labs this semester are going to be designed such that you and your lab partner must work together to create non-trivial software products
  • You will need to work together to design, implement, and test these projects before turning them in
  • I would like you to utilize the principles of pair programming and test-driven design while working on these projects
  • As such, not only will you be asked to turn in the software that you created, but you will also be asked to turn in the test cases that you wrote during the design and development process
    • This will give you the experience to write test cases not only for source code but also for things like shell scripts, build files, configuration management systems, etc.
  • You must also be prepared to work as a pair: two people, one keyboard (!), and be willing to switch roles (i.e. hand over the keyboard) fluidly
• So, FIND A LAB PARTNER BEFORE LAB NEXT WEEK!
  • Otherwise, the TA will find one for you
  • Note: we may have to create a team of three people, if a lab section has an odd number of students enrolled

No Silver Bullet

There is no single development, in either technology or management technique, which by itself promises even one order-of- magnitude improvement within a decade in productivity, in reliability, in simplicity.

— Fred Brooks, 1986

i.e. There is no magical cure for the “software crisis”

Why? Essence and Accidents

• Brooks divides the problems facing software engineering into two categories
  • essence: difficulties inherent in the nature of software
  • accidents: difficulties related to the production of software
• Brooks argues that most techniques attack the accidents of software engineering

An Order of Magnitude

• In order to improve the development process by a factor of 10
  • first, the accidents of software engineering would have to account for 90% of the overall effort
  • second, tools would have to reduce accidental problems to zero
• Brooks doesn't believe that the former is true and the latter is nigh impossible because each new tool or technique solves some problems while introducing others

The Essence

• Brooks divides the essence into four subcategories
  • complexity
  • conformity
  • changeability
  • invisibility
• Lets consider each in turn

Complexity

• Software entities are amazingly complex
  • No two parts (above statements) are alike
    • Contrast with materials in other domains
  • Large software systems have a huge number of states
    • Brooks claims they have an order of magnitude more states than computers (i.e. hardware) do
  • As the size of a system increases, its parts increase exponentially

Complexity, continued

• You can't abstract away the complexity of the application domain. Consider:
  • air traffic control
  • international banking system
  • flight software for space craft
  • etc.
• These domains are complex and this complexity will appear in the software system as designers attempt to model the domain
• Complexity also comes from the numerous and tight relationships between heterogeneous software artifacts such as specs, docs, code, test cases, etc.

Complexity, continued

Conformity

• A significant portion of the complexity facing software engineers is arbitrary
  • Consider designing a software system for an existing business process and a new VP arrives at the company
  • The VP decides to make a mark on the company and changes the business process
  • Our system must now conform to the (from our perspective) arbitrary changes imposed by the VP

Conformity, continued

• Other instances of conformity
  • Having to integrate with a non-standard module interface
    • Arbitrary because the interface was created at some developer's whim rather than being demanded by the application domain
  • Adapting to a pre-existing environment
  • May be difficult to change the environment to make developing the software easier; however if it does change, you can be sure that the software system will be expected to change!
• Main Point: Its is almost impossible to plan for arbitrary change; instead, you just have to wait for it to occur and deal with it when it happens

Changeability

• Software is constantly asked to change
• Other things are too, however, manufactured things are rarely changed after they have been created
  • instead, changes appear in later models
  • automobiles are recalled only infrequently
  • buildings are expensive to remodel
• With software, the pressure to change is greater
  • in a project, it is functionality that is often asked to change and software EQUALS functionality (plus its malleable)
  • clients of a software project often don't understand enough about software to understand when a change request requires significant rework of an existing system

Invisibility

• Software is by its nature invisible; and it is difficult to design graphical displays of software that convey meaning to developers
• Contrast to blueprints: here geometry can be used to identify problems and help optimize the use of space
• But with software, its difficult to reduce it to diagrams
  • Hard to get both a “big picture” view as well as a set of detailed views
  • Hard to convey just one issue on a single diagram; instead multiple concerns crowd and/or clutter the diagram hindering understanding
• This lack of visualization deprives the engineer from using the brain's powerful visual skills

What about <X>?

• Brooks argues that past breakthroughs solve accidental difficulties

Promising Attacks on the Essence

• Buy vs. Build
  • Don't develop software when you can avoid it
• Rapid Prototyping
  • Use to clarify requirements
• Incremental Development
  • don't build software, grow it
• Great designers
  • Be on the look out for them, when you find them, don't let go!

No Silver Bullet Refired

• Brooks reflects on the “No Silver Bullet” paper, ten years later
  • Lots of people have argued that their methodology, technique, or tool is the silver bullet for software engineering
    • If so, they didn't meet the deadline of 10 years or the target of a 10 times improvement in producing software
  • Other people misunderstood what Brooks calls “obscure writing”
    • For instance, when he said “accidental”, he did not mean “occurring by chance”; instead, he meant that the use of technique A for benefit B unfortunately introduced problem C into the process of software development

The size of accidental effort

• Some people misunderstood his point with the 90% figure
  • Brooks doesn't actually think that accidental effort is 90% of the job
    • its much smaller than that
  • As a result, reducing it to zero (which is probably impossible) will not give you an order of magnitude improvement

Obtaining the Increase

• Some people interpreted Brooks as saying that the essence could never be attacked
  • That's not his point however; he said that no single technique could produce an order of magnitude increase by itself

• He argued that several techniques in tandem could achieve that goal but that requires industry-wide enforcement and discipline

  Brooks states:

  We will surely make substantial progress over the next 40 years; an order of magnitude over 40 years is hardly magical…

Next Week

• No class on Monday
• Lecture 4: Unix Shell and Chapter 1 of Mythical Man-Month
• Lab 1 Assigned: Shell scripts and software deployment (for week 3)
• Homework 1 Due