Today's Lecture

• Software Tools (Brooks, Chapter 12)
• Build Management
• Make (a build management tool)

Software Tools

• Fred Brooks talks about the importance of software tools in Chapter 12 of his book. Many of the concepts discussed in chapter 12 are “dated”
  • in that modern programming environments provide a standard set of tools that are used across many software projects and development organizations
• Gems
  • “If a separate machine is needed, it is a rather peculiar thing—it need not be fast, but it needs at least a million bytes (1 MB) of main storage, a hundred million bytes of on-line disk (100MB), and terminals.”
  • “System debugging has always been a graveyard-shift occupation, like astronomy. Twenty years ago, on the 701, I was initiated into the productive informality of the predawn hours, when all the machine-room bosses are fast asleep at home, and the operators are disinclined to be sticklers for rules. Three machine generations have passed; technologies have changed totally; operating systems have arisen; and yet this preferred method of working hasn't changed.”
• However, there are still some lessons to be learned

Lessons Learned

• A balance must be struck between general- purpose tools and customized tools for a project
  • general-purpose tools are maintained by a separate organization and are widely known
    • so you don't have to waste time maintaining such tools or training your people to use them
  • custom tools, tailored for a specific project, can result in higher productivity, since it can automate a repetitive task that otherwise would need to be done manually
    • Take my “roster processing” script that I demonstrated in Lecture 6. I only needed to write it once, and now it can process new rosters each semester in less than a second!
  • Hand-in-hand with these observations is the need for a tool builder on software development teams; a person who can create the custom tools that are used by other team members to get their jobs done quickly and efficiently
• Maintenance of Program Libraries
  • (This section foreshadowed configuration management, which we will discuss later in the semester)
  • Each programmer has a separate workspace
  • Finished components are placed in a system integration library for testing
  • Tested components are incorporated into the “official” release
  • This process should be automated by tools
    • And today it is with: rcs, cvs, svn, …
• High-Level Programming Languages and “Interactive Programming” are listed as important tools
  • The important lesson here is that tools we take for granted today were at one time “new” and “untested” techniques
  • Back in the 60's and 70's a significant amount of programming was done in assembly; software engineers encounter a lot of accidental difficulties in these languages
    • like remembering to save registers properly on a context switch
  • “batch” programming
    • create punch cards, submit to machine room, get results of run back the next day! (Imagine debugging a program with a 24-hour turn around time!!!)
  • back in 1975, they already had preliminary data on how much more productive interactive programming is on debugging (the data ranged from 2x to 8x better, page 136)

Build Management

• During the implementation phase, the process for constructing a system should be engineered
  • What are the steps to build a system?
    • e.g. what subsystems need to be built before the system can be built?, what libraries are needed?, what resources are required?, etc.
  • Who is authorized to build a system?
    • Small projects: individual programmers
    • Large projects: build managers and/or configuration managers
  • When are system builds performed?
    • e.g. perhaps a system is so large that it can only be built at night when there are enough resources available…
• Most modern programming environments have build management capabilities built into them
  • For instance, a Java development environment typically has the notion of a “project” and it can compile all project files in the correct order (and it only compiles files dependent on a change)
• These capabilities free developers from accidental difficulties
  • having to remember the correct compilation order
  • correctly identifying all files dependent on a change

Unix Build Management

• In Unix environments, a common build management tool is “make”
  • Make provides very powerful capabilities via three types of specification styles
    • declarative
    • imperative
    • relational
  • These styles are combined into one specification
    • “the make file”

Specification/Modeling Styles

• Operational (or Imperative)
  • Described according to desired actions
  • Usually given in terms of an execution model
• Descriptive (or Declarative)
  • Described according to desired properties
  • Usually given in terms of axioms or algebras
• Structural (or Relational)
  • Described according to desired relationships
  • Usually given in terms of a graph
    • e.g. entity-relationship diagrams from the field of databases

Make Specification Language

• Hybrid Declarative/Imperative/Relational
  • Dependencies are Relational
    • Make specifies dependencies between artifacts
  • Rules are Declarative
    • Make specifies rules for creating new artifacts
  • Actions are Imperative
    • Make specifies actions to carry out rules

Example Makefile

---

Target1: Target2 Target3 … TargetN
  \t     Action1
  \t     Action2
  \t     …
  \t     ActionN

Target2: Target5 Target6
  \t     Action3

Target3: Target5 Target7
  \t     Action4

---

• A Makefile consists of a set of rules.
• Each rule contains a target followed by a colon followed by a list of dependencies
• Each subsequent line of a rule begins with a tab character (required) followed by an action
• If a dependency changes, make invokes a rule's action to recreate the target
• Thus, if Target5 were to change in some way, make would
  • run Action3 to recreate Target2
  • run Action4 to recreate Target3
  • run Action1–ActionN to recreate Target1

More on Make

• Make is well-integrated into the Unix environment
  • Targets and Dependencies are file names
  • Actions are shell commands
• Rules are placed in a file and denote the “specification”
• Rules make explicit the dependencies of a system and how to manage them
• Note: make is not just for source code!

Why use make?

• Why not write a shell script for building a program? e.g.

#!/bin/bash
g++ -c main.cpp
g++ -c input.cpp
g++ -c output.cpp
g++ main.o input.o output.o -o program

• Answers
  • A shell script will compile each file every time its run… even if the file has not changed since the last compilation!
  • When building large systems, such an approach does not scale!
  • You only want to recompile changed files and the files that depend on them

  Make is much “smarter”; by only recompiling changed files and their dependencies, make can scale to building large software systems

A “Real” Makefile

program: main.o input.o output.o
    g++ main.o input.o output.o -o program

main.o: main.cpp defs.h
    g++ -c main.cpp

input.o: input.cpp defs.h
    g++ -c input.cpp

output.o: output.cpp defs.h
    g++ -c output.cpp

• This makefile creates an executable called program from three C++ files and a single header file
• The default target of a makefile is its first rule (there are exceptions to this that we will see on Monday)
• So, when make interprets this file, it will attempt to create program by looking for the files main.o, input.o, and output.o
• Make will then search for rules that govern the creation of those files and then look at their dependencies and rules for those dependencies, etc.
• As soon as it finds a target that is “out of date”, it will invoke actions to bring them up to date.
• In this particular example, if none of the .o files existed, make would invoke the g++ compiler to create them and then invoke the linker (via g++) to create program

Examples

Given a directory with the following files and time stamps (higher number == newer)

File	Time Stamp
main.cpp	1
input.cpp	2
defs.h	3
main.o	4

Example 1

main.o: main.cpp defs.h
    g++ -c main.cpp

• What happens when you invoke make on this makefile?
  • make: "main.o" is up to date

Example 2

input.o: input.cpp defs.h
    g++ -c input.cpp

• What happens when you invoke make on this makefile?
  • g++ -c input.cpp

Example 3

output.o: output.cpp defs.h
    g++ -c output.cpp

• What happens when you invoke make on this makefile?
  • make: Fatal error: Don't know how to make target "output.cpp"

What would happen if you typed “make” again after you ran example 2 for the first time?

Make Dependency Graph

• A Makefile can be modeled as a dependency graph.
• The make algorithm performs a traversal over this graph.
• Each node is checked after all of its children have been checked
• And a node's actions are run if any child has a timestamp greater than its parent

make on the command line

• If you invoke make on the command line with no options it will
  • look for a file called makefile or Makefile and open it
  • It will then look for the first target in that file and make that its goal
  • It then checks that target's dependencies and invokes actions as necessary to make it up to date
• You can override this default behavior with command line arguments
  • -f filename: use the specified file as the makefile
  • targetname: you can list a specific target on the command line and then make sets that target as its goal
  • Example: make -f mymakefile main.o
    • Look for rules in the file called mymakefile and verify that main.o is up to date

More on Actions

• Actions do not have to invoke a compiler
  • they can be any shell command
• Additionally, targets do not have to be files:

clean:
    rm *.o

• Targets like “clean” with no dependencies and no files created in response to their actions are called “phony targets”
  • The actions of a phony target always execute if the phony target becomes the current goal
  • any target that depends on a phony target will always have its actions executed
    • why?
• Phony targets can be useful for deployment:

install: ~/csci3308/arch/i686/bin/program

~/csci3308/arch/i686/bin/program: program
    cp program ~/csci3308/arch/i686/bin

If you type “make install”, make checks to see if the file “program” in the current directory is newer than the one in the install directory. If it is, make copies (or installs) the new version into the install directory

Action/Target mismatch

• Actions do not have to create their target
  • but when this occurs, you have a mismatch

main.o: main.c
    lpr -Pakira ~/.cshrc

• What happens when you type make?
  • Assume that main.c is newer than main.o
• You typically do not want to do this; but make has no way to prevent the creation of this type of rule

Summary

• Software Tools: Brooks stresses the need for both generic and specific tools and the need for a tool builder on a development team
• Build Management: We want the process of building tools to be engineered. What are the steps and who is responsible for building the system under development
• Make: a build management tool that is well integrated into the Unix environment via file names and shell commands

Coming Up Next

• Lecture 9: Advanced Make
• Lecture 10: The Ant Build Management System
• Quiz 3: Available after this lecture. Don't forget to take it!!!
• Homework 4 Assigned