Lecture 09: Advanced Make and The Surgical Team

Today's Lecture

Advanced Make Techniques
The Surgical Team (Brooks, Chapter 3)

Advanced Make Techniques

Make Macros
Make View Path
Make Automatic Variables
Make Pattern Matching

Make Macros

Make has variables known as “macros”
- which is an unfortunate name because they act like variables!
They are similar to shell variables with a few differences
- Each macro holds a string value
- They are defined using an equal sign
  - INSTALLDIR = /home/faculty/kena/tmp/
- They are referenced by preceding its name with a dollar sign
  - ```
  INSTALLDIR = /home/faculty/kena/tmp/
  
  $(INSTALLDIR)/program : program
      cp program $(INSTALLDIR)
```
- The parentheses above are required, otherwise make assumes that a macro name is just one letter long
  - That is, $INSTALLDIR is interpreted by make as $(I)NSTALLDIR

Increased Abstraction

Macros help to increase the level of abstraction of a Makefile
- It takes this:
  - ```
  program: main.o input.o output.o
      g++ main.o input.o output.o -o program
```
- and turns it into this:
  - ```
  EXECUTABLE = program
  OBJECTS    = main.o input.o output.o
  
  $(EXECUTABLE): $(OBJECTS)
      g++ $(OBJECTS) -o $(EXECUTABLE)
```
- The former can only work for one situation. The latter better explains what the rule is for and can be adapted to many different situations
Why is this increase in abstraction important?
- Can save keystrokes (minor)
- Can improve readability (important)
- Can improve understandability (important)
- Allows rule to be used in many different situations (very important)
  - We can change the specific files being referenced without changing the generic rule

Make Automatic Variables

Make has a special feature called automatic variables
- It is the second abstraction mechanism provided by make; the first being macros
Automatic variables can only be used within the actions of a make rule
- The specific value of an automatic variable depends on the target and dependencies of the rule in which it occurs

Automatic Variable	Definition
$@	The target of the rule
$<	The first dependency
$^	All of the dependencies
$?	All of the dependencies newer than the target of the rule
$*	The stem of a pattern matching rule. If you are building `input.o` from `input.c`, the stem is “input”. This automatic variable only works in pattern matching rules.

Examples

Below is a makefile that shows each rule with two actions

The first is a standard action
The second is the same action using automatic variables

program: input.o output.o
    g++ input.o output.o -o program
    g++ $^ -o $@

input.o: input.c defs.h
    g++ -c input.c -o input.o
    g++ -c $< -o $@

output.o: output.c defs.h
    g++ -c output.c -o output.o
    g++ -c $< -o $@

Use of $?
- The automatic variable $? can be used in situations where you need to update archives
  - For example, where you want to replace files in a tar file that have changed since archive was created
```
lab5.tar: README makefile lab5.cpp
    tar rf lab5.tar $?
```

The View Path

Make applies special meaning to a variable called VPATH, aka “the view path”.
While VPATH is not an automatic variable, it does interact with them (as we shall see shortly)
VPATH consists of a list of directories, just like the PATH variable of bash
- If make cannot find a dependency in the current directory, it looks for the dependency in the view path.
- Note: just because a file is found in the view path does not mean that the shell can find it when executing commands (see below)
In the example below, assume we are invoking make from a build directory. As you can see, the VPATH points to the source directory
```
VPATH = $(HOME)/csci3308/src/lab05
lab05.o: lab05.c
    g++ -c lab05.c -o lab05.o
```
When you invoke make in the build directory on this file, you will see the following output:
```
g++ -c lab05.c -o lab05.o
g++: lab05.c: No such file or directory
```
Why does the action fail?
To solve the problem, we use automatic variables
- When a file is found in the view path, automatic variables are set to contain the full path of the file
- Therefore, our action line needs to make use of automatic variables to reference files in an action
- The full path of the file will be passed to the shell, which will then be able to find the file

Hence

VPATH = $(HOME)/csci3308/src/lab05

lab05.o: lab05.c
     g++ -c $< -o $@

$ make

g++ -c /home/faculty/kena/csci3308/src/lab05/lab05.c -o lab05.o

Here, the action references lab05.c correctly. Note: lab05.o is created in the current directory, even though the source code is located elsewhere (which is similar to how we compiled gnuchess in lab 1)

Make Pattern Matching

Setup: below is a fairly standard makefile.

CXX = g++
CFLAGS = -c -g

program: main.o input.o output.o
    $(CXX) $^ -o $@
    
main.o: main.cpp defs.h
    $(CXX) $(CFLAGS) $<
    
input.o: input.cpp defs.h
    $(CXX) $(CFLAGS) $<
    
output.o: output.cpp defs.h
    $(CXX) $(CFLAGS) $<

Note: how the structure of the last three rules is exactly the same?
Make has a mechanism for capturing these similarities, called pattern matching
- This is make's third abstraction mechanism
We can capture similarities between rules based on file suffixes
Thus our rules in the previous examples that took care of compiling files can be expressed as:
```
%.o: %.cpp
    $(CXX) $(CFLAGS) $<
```
This is not exactly the same as our previous rules, why? Does it matter?
Make supports pattern matching through the presence of the character “%” in rules
```
%.o: %.cpp
    $(CXX) $(CFLAGS) $<
```
If you type “make input.o” the rule becomes
```
input.o: input.cpp
    $(CXX) $(CFLAGS) $<
```
Note: automatic variables are required. Why?

Benefits of Pattern Matching

Scalability
- The same rule can apply to thousands (or more) of files
Compactness
- Small compact specifications are easier to understand and debug
These benefits are similar to the benefits of wildcards and regular expressions
- which should come as no surprise

Implicit Rules

Make's abstraction mechanisms…
- macros
- automatic variables
- pattern matching rules
…make it possible to have a common set of rules automatically defined by make
These rules are called “implicit rules”
Example
- If you create a makefile that contains just the following rule:
```
program: program.o
    $(CC) $(CFLAGS) $^ -o $@
```
  Make will act as if you had also included the following rule
```
%.o: %.cpp:
    $(CC) $(CFLAGS) -c $&
```
This allows the develop to write extremely compact make files that only define the rule needed to link their .o files into an executable
A list of make's pre-defined implicit rules can be found in make's manual.

The Surgical Team

Or: How should a development team be arranged?
The problem
- Good programmers are much better than poor programmers
  - typically 10 times better in productivity
  - typically 5 times better in terms of program elegance
- but we often do not have access to these “super programmers”

The Dilemma of Team Size

Consider the following example
- 200-person project with 25 managers where the managers are also experienced software developers
Previous slide argues for firing the 175 workers and use the 25 managers as the development team!
- However, this is still bigger than “the ideal” small team size of 10 people (general consensus)
But, the original team of 200 is too small to tackle very large systems
- Consider: OS/360 had over 1000 people working on it; consumed 5000 person-years of design, construction, and documentation!
Two needs to be reconciled
- For efficiency and conceptual integrity, a small team is preferred
- To tackle large systems, considerable resources are needed
One approach: Harlan Mills' Surgical Team
- One person performs the work, all others perform support tasks
This is only one approach, there are many ways in which a development team can be organized!

The Proposed Team

The surgeon: the chief programmer
The co-pilot: like the surgeon but less experienced
The administrator: relieves surgeon of administrative tasks
The editor: edits documentation
Two secretaries: to support admin and editor
The program clerk: manages structure of program files (this position is completely obsolete)
The tool smith: builds tools to support surgeon's work
The tester: creates test cases and manages testing process
The language lawyer: knows how to get every last bit of functionality out of the chosen programming language, most likely shared across multiple teams

How is this different?

Normally, work is divided equally
- Now, only surgeon and copilot divide the work
Normally, each person has equal say
- Now, the surgeon is the absolute authority
Note: communication paths are reduced
- Normally 10 people => 45 paths
- Surgical Team => at most 13 (See Fig. 3-1.)

How does this scale?

Reconsider the 200 person team
- Communication paths => 19,900!
Instead create twenty 10-person surgical teams
Now, only 20 surgeons must work together
- 20 people => 190 paths (two orders of magnitude less!)
Key problem becomes ensuring conceptual integrity of the design now that there are “too many cooks”

The Modern Surgical Team

The surgical team, as conceived by Mills and described by Brooks, is not widely used today
- On “Internet time”, the chief programmer approach is impractical
Now, it is more important that there be one to three designers, or software architects, that guide the design of the system
- with many people implementing the system
This is true of the Microsoft approach
- The program manager is responsible for the feature set
It is also true of many open source programs

Summary

Make's abstraction mechanisms…
- macros
- automatic variables
- pattern matching rules
… provide a lot of power to developers in creating a carefully controlled build process for their executables
The Surgical Team: It is important to think about how a software team is organized.
- What roles exist in the team's organization and what communication paths exist between them.
- A major goal is to reduce the number of communication paths between team members (to avoid work being bogged down by too much communication). Note: this is antithetical to the newer Agile Design methodologies such as Extreme Programming.
- Another goal is to allocate tasks and authority among the team members.

Coming Up Next

Lecture 10: The Ant Build Management System
Homework 4 Due

Software Methods and Tools

Lecture 09: Advanced Make and The Surgical Team

Today's Lecture

Advanced Make Techniques

Make Macros

Increased Abstraction

Make Automatic Variables

Examples

The View Path

Make Pattern Matching

Benefits of Pattern Matching

Implicit Rules

The Surgical Team

The Dilemma of Team Size

The Proposed Team

How is this different?

How does this scale?

The Modern Surgical Team

Summary

Coming Up Next