Makefiles and make give you benefits that cannot be matched by the best IDE tools:

  • You can see exactly what your toolchain looks like and the parameters passed to each step. You do not have to click through many menus and screens with no guarantee that there is yet one more screen that had parameters you wanted to adjust.

  • You can compile remotely; you can compile using a crontab in the middle of the night; you do not have to attend to the compilation; colleagues can run a diff and see what has recently changed in the build process; etc.

There are serious modern contenders to Makefiles, for example qmake and cmake. There are less modern but still widely adopted build systems, for example automake. But make/make/nmake on Linux, OS X, and Windows, respectively, is the tool that will most likely be available.

The catch, of course, is that the Makefile language is arcane. It is unlike any other language you have seen.

1. Hello World of Auto-Compilation

The most basic feature of make is timestamp comparisons. Suppose that you would like to compile the iconic helloworld.cpp file (I’ll use C++ examples, but the principles apply for any project that can be built from the command-line, including, as you’ll see towards the end, examples for processing LaTeX, HTML, and image files.):

#include <iostream>

int main() {
    std::cout << "Hello, World!" << std::endl;
}

The command to compile this program is g++ helloworld.cpp -o helloworld. One advantage of a Makefile is that it saves us from typing repeatedly the same lines, but the basic and crucial function is to say "only run this command if it is necessary".

Since the operating system helpfully saves a timestamp for each file, we will use that timestamp and implement "If the timestamp of the program helloworld.cpp is later than that of the executable helloworld, refresh the executable by running the build command."

Because the instruction "If timestamp(fileA) > timestamp(fileB)" is abundant in builds, it has the brief notation

fileB: fileA

You may want to read it as "fileB depends on fileA", and hence fileB should be generated if its timestamp is older than that of fileA.

fileB: fileA is a shorthand for "If fileA is more recent than fileB, execute the following commands."

Any number of commands can follow this implicit if-statement, but they must all be preceded by a TAB character—spaces will not do. Indeed long before Python adopted its signature of identifying blocks by indentation, Makefiles identified blocks by TABs.

In our case we are running a single command, hence the Makefile:


helloworld: helloworld.cpp
	g++ -o helloworld helloworld.cpp

Each such block is termed a rule. The syntax for a rule is:

target: prerequisites
<tab>    command
...
<tab>    command

The target is what is being built. The prerequisites are what will be used during the build. Note the plural. In our current example there is just one prerequisite, but as we’ll soon see, there can be many.

When we run make in a directory, it looks by default for a file called Makefile. It is seldom necessary to use a different filename, but doing so is possible with the -f command-line switch make -f alternativeMakefile.

make by default parses a file named Makefile.

A project will normally consist of many header files (.hpp) and compilation units (.cpp). We may write for instance

example.cpp
#include "factorial.hpp"
#include <iostream>

int main() {
    std::cout << "7!=" << factorial(7) << std::endl;
}

and depend on an implementation of a function factorial that is declared in a header file

factorial.hpp
int factorial(int n);

and defined in its own compilation unit.

factorial.cpp
#include "factorial.hpp"

int factorial(int n) {
// precondition: n>=0
    if(n)
	return n * factorial(n-1);
    else
	return 1;
}

A perfectly legitimate Makefile in this case is

build:
	g++ -c factorial.cpp
	g++ -c example.cpp
	g++ -o example example.o factorial.o

The three commands will be run in this order. The trouble is that all three will always be run regardless of whether each is necessary.

To test the timestamps to determine whether each step is necessary, we use three rules. The fourth rule (make clean) serves to delete the generated files.

example: example.o factorial.o
	g++ -o example example.o factorial.o

example.o: example.cpp factorial.hpp
	g++ -c example.cpp

factorial.o: factorial.hpp factorial.cpp
	g++ -c factorial.cpp

clean:
	rm example example.o factorial.o

The second rule, for example, says if the timestamp of either example.cpp or factorial.hpp is more recent than that of example.o, run the command g++ -c example.cpp.

One detail is important; make has a default target: the first one.

The default target is the first target appearing in a Makefile.

3. Implicit Rules

If the Makefile consists of just the one rule

Makefile
example: example.o factorial.o
	g++ -o example example.o factorial.o

the build will still complete. Running make will produce

c++    -c -o example.o example.cpp
c++    -c -o factorial.o factorial.cpp
g++ -o example example.o factorial.o

Let’s look at the diagram of dependencies

prerequisites
Figure 1. prerequisites

The chart will be a directed acyclic graph (DAG). We have only specified the rule for building example, but rules for building example.o and factorial.o were executed anyway. That’s because make has a built-in rule that says: "If a .o file is needed and its timestamp is later than a .cpp file with the same trunk name, run the command `c++ -c -o trunk.o trunk.cpp`".

We’re now missing an important dependency: if factorial.hpp (and no other file) is modified, make will not recompile.

4. Make Variables

Note that c++ is not the command we meant. The two commands c++ and g++ may or may not be the same. Even if they are the same, it is prudent and cleaner to be explicit about the command we want, which we do using a variable CXX=g++.


CXX = g++

example: example.o factorial.o
	$(CXX) -o example example.o factorial.o

clean:
	rm example example.o factorial.o

5. Dependencies

We reinsert the dependencies in the makefile. The rules remain implicit, and a dependency need not be the prelude to a rule! This is also a chance to illustrate that make variables serve more than just string substitution. They can store a list of items.


CXX = g++

OBJECTS = example.o factorial.o

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

example.o: factorial.hpp
factorial.o: factorial.hpp

clean:
	rm example $(OBJECTS)

6. Phonies

The choice of the "clean" target is arbitrary; "clean" is not a make keyword. This becomes a problem if we happen to also have a file named "clean". Because if the file already exists, make will determine that it need not be built, hence the rule will never run. We rectify this by signaling that our chosen keyword is a "phony" target.


CXX = g++

OBJECTS = example.o factorial.o

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

example.o: factorial.hpp
factorial.o: factorial.hpp

.PHONY: clean
clean:
	rm -f example $(OBJECTS)

7. Automatic Variables

Let’s backtrack all the way to our first Makefile.


helloworld: helloworld.cpp
	g++ -o helloworld helloworld.cpp

Even a build description this brief contains redundancies. Each of the two filenames (the source and the executable) appears twice. Within a given rule, we can refer to the target using $@ and to the prerequisite using $<, leading to the following simpler build steps.


helloworld: helloworld.cpp
	g++ -o $@ $<

8. Omitting Makefile

Occasionally, it remains convenient to use make even if the directory has no Makefile. If we have a directory with multiple, independent source files such as test1.cpp and test2.cpp, we may find it quite sufficient to omit the Makefile. The implicit rule will still apply.

Effectively, if our intended Makefile will consist of just:

test1: test1.cpp
	c++ -o $@ $<

test2: test2.cpp
	c++ -o $@ $<

then we can omit the Makefile altogether.

9. Auto Dependencies

Merely determining the list of dependencies is a tedious and error-prone process. Fortunately g++ provides a command-line switch that will generate the list of dependencies for a given source file. If we run g++ -MM example.cpp we will obtain a single line example.o: example.cpp factorial.hpp. We save this line in a file and use d as a suffix for the dependency files g++ -MM example.cpp > example.d.

But we need one such command for each source file. This will again quickly get unwieldy. An auto dependency includes a wildcard in the filenames. We use %.d: %.cpp to say that anytime a file with the extension .d is needed, its timestamp is compared to the file with the same trunk and with a .cpp extension.

Naturally, auto dependencies are the perfect place to use automatic variables.


CXX = g++

OBJECTS = example.o factorial.o

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

# automatic variables: $< and $@
%.d: %.cpp
	$(CXX) -MM $< > $@

# include example.d factorial.d
-include example.d factorial.d

.PHONY: clean
clean:
	rm -f example $(OBJECTS)

If we run make, we get:

make
Makefile:15: example.d: No such file or directory
Makefile:15: factorial.d: No such file or directory
g++ -MM factorial.cpp > factorial.d
g++ -MM example.cpp > example.d
g++    -c -o example.o example.cpp
g++    -c -o factorial.o factorial.cpp
g++ -o example example.o factorial.o

The include directive simply includes the two files that contain the dependencies. The -include variation performs the same task, but without echoing to the shell whether the inclusion has been successful:

make
g++ -MM factorial.cpp > factorial.d
g++ -MM example.cpp > example.d
g++    -c -o example.o example.cpp
g++    -c -o factorial.o factorial.cpp
g++ -o example example.o factorial.o

10. Substitution

The include directive in the last Makefile is evidently repeating filenames that exactly match those in the OBJECTS variables. It would be nice to avoid this repetition by performing string substitution instead.

Rather than include explicit filenames, we use instead -include $(DEPENDENCYFILES), where DEPENDENCYFILES is the variable that will hold the names of the dependency (.d) files.

We initialize DEPENDENCYFILES using the substitution DEPENDENCYFILES = $(OBJECTS:.o=.d), which replaces instances of .o by .d in the variable OBJECTS.


CXX = g++

OBJECTS = example.o factorial.o
DEPENDENCYFILES = $(OBJECTS:.o=.d)

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

%.d: %.cpp
	$(CXX) -MM $< > $@

-include $(DEPENDENCYFILES)

.PHONY: clean
clean:
	rm -f example $(OBJECTS)

11. Cleaning Dependencies

A flaw with the last Makefile is that dependency files continued to clutter the directory even after make clean. We will typically want to provide several levels of "cleaning", reserving the deepest clean for deleting the resulting executable itself. Regardless of our choice, the ability to delete dependency files is necessary. If we distribute the code, the dependencies may even be incorrect for alternative operating systems. To remedy this issue we will delete dependency files alongside .o files for the clean target.


CXX = g++

OBJECTS = example.o factorial.o
DEPENDENCYFILES = $(OBJECTS:.o=.d)

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

%.d: %.cpp
	$(CXX) -MM $< > $@

-include $(DEPENDENCYFILES)

.PHONY: clean
clean:
	rm -f example $(OBJECTS) $(DEPENDENCYFILES)

Running make clean will now duly delete dependency files.

make clean
rm -f example example.o factorial.o example.d factorial.d

But we’ve just introduced another problem. If we run make clean two times in a row—​hence deleting .d files—​the .d files be generated, only to be deleted right after. At the second invocation we get:

make clean
g++ -MM factorial.cpp > factorial.d
g++ -MM example.cpp > example.d
rm -f example example.o factorial.o example.d factorial.d

12. Make Command Goals

The solution is simple. We only include dependency files if the target is not clean. Make defines the built-in variable MAKECMDGOALS to reflect the parameters passed to make on the command-line, clean in our case. ifeq and ifneq test whether the two expressions passed in parentheses (do not) evaluate to the same string.


CXX = g++

OBJECTS = example.o factorial.o
DEPENDENCYFILES = $(OBJECTS:.o=.d)

example: $(OBJECTS)
	$(CXX) -o example $(OBJECTS)

%.d: %.cpp
	$(CXX) -MM $< > $@

.PHONY: clean
clean:
	rm -f example $(OBJECTS) $(DEPENDENCYFILES)

ifneq ($(MAKECMDGOALS),clean)
-include $(DEPENDENCYFILES)
endif

13. Two Convenience Variables

Let’s make a small modification for convenience by introducing a SOURCES variable that we use in two substitutions to determine the objects and the dependency files. We also define a PROGRAM variable for the name of the executable.


CXX = g++

PROGRAM = example

SOURCES = example.cpp factorial.cpp
OBJECTS = $(SOURCES:.cpp=.o)
DEPENDENCYFILES = $(SOURCES:.cpp=.d)

$(PROGRAM): $(OBJECTS)
	g++ -o example $(OBJECTS)

%.d: %.cpp
	g++ -MM $< > $@

.PHONY: clean
clean:
	rm -f example $(OBJECTS) $(DEPENDENCYFILES)

ifneq ($(MAKECMDGOALS),clean)
-include $(DEPENDENCYFILES)
endif

14. Pattern Substitution (and avoiding clutter)

We need an excuse to discuss a more sophisticated method for pattern substitution. After compiling, we will end up with dependency and object files in the same directory as our source files. Is there a way to store generated files in a separate directory? If our objective is just hyper-organization, that would be sufficient justification, but there is more.

Suppose that our program will happily run on multiple operating systems. Suppose also that the filesystem is shared. Thus we will see the same files and directories regardless of the operating system we are logged on and compiling for.

It would be wasteful to make clean simply to make room for new object files. This also precludes testing simultaneously on more than one operating system. Worst, the time needed to compile our project may necessitate keeping the object and dependency files indefinitely. (Keep in mind that C++ #include directives can make the dependencies diverge from one OS to another.)

The solution relies on the uname command. It will give us a brief string that identifies the OS, for example Darwin on OS X and i386-linux on Linux. We store the shell’s uname result in a variable HOSTTYPE and store all generated files in a directory with that name. For each target, we ensure that the directory exists by running mkdir -p. The -p option will silently skip creating the directory if it already exists.

We now need a more sophisticated pattern substitution. We’d like not just to replace the extension, but also to insert a prefix $(HOSTTYPE)/ to the trunkname.


HOSTTYPE = $(shell uname)
PROGRAM = $(HOSTTYPE)/example
SOURCES = example.cpp factorial.cpp

OBJECTS         = $(patsubst %.cpp,$(HOSTTYPE)/%.o,$(SOURCES))
DEPENDENCYFILES = $(patsubst %.cpp,$(HOSTTYPE)/%.d,$(SOURCES))

CXX = g++

$(PROGRAM): $(OBJECTS)
	@mkdir -p `dirname $@`
	$(CXX) -o $@ $(OBJECTS)

$(HOSTTYPE)/%.o: %.cpp
	@mkdir -p `dirname $@`
	$(CXX) -c -o $@ $<

$(HOSTTYPE)/%.d: %.cpp
	@mkdir -p `dirname $@`
	$(CXX) -MM $< > $@

.PHONY: clean
clean:
	rm -f $(OBJECTS) $(DEPENDENCYFILES)

ifneq ($(MAKECMDGOALS),clean)
-include $(DEPENDENCYFILES)
endif

15. Subdirectories

For our last step, let’s look at a necessary feature for a larger project. We divide the code into modules and subdirectories. For our example, we will store the source files in directories main and factorial and set the SOURCES variable accordingly: SOURCES = main/example.cpp factorial/factorial.cpp. Once we are content with the build steps and features of our Makefile, the SOURCES variable will typically be the only part of the Makefile that we will continuously adjust by hand, and even then we may want to automate that step, which forces us to declare unused source files as clutter and stow them elsewhere. The current change will only require adjusting the pattern substitution steps. For good measure, we also conclude by defining two levels of cleaning, with the addition of a distclean target for preparing our code for distribution by deleting also the executable.


HOSTTYPE = $(shell uname)
PROGRAM = $(HOSTTYPE)/example

CXX = g++
CXXFLAGS += -I. -O2

SOURCES = main/example.cpp factorial/factorial.cpp
OBJECTS  = $(patsubst %.cpp,$(HOSTTYPE)/%.o,$(SOURCES))
DEPFILES = $(patsubst %.cpp,$(HOSTTYPE)/%.d,$(SOURCES))

$(PROGRAM): $(OBJECTS)
	mkdir -p `dirname $@`
	$(CXX) $(CXXFLAGS) -o $(PROGRAM) $(OBJECTS)

$(HOSTTYPE)/%.o: %.cpp
	mkdir -p `dirname $@`
	$(CXX) $(CXXFLAGS) -c -o $@ $<

$(HOSTTYPE)/%.d: %.cpp
	mkdir -p `dirname $@`
	$(CXX) $(CXXFLAGS) -MM -MT $@ $< > $@

ifneq ($(MAKECMDGOALS),clean)
ifneq ($(MAKECMDGOALS),distclean)
-include $(DEPFILES)
endif
endif

.PHONY: clean distclean
clean:
	rm -f $(OBJECTS) $(DEPFILES)

distclean: clean
	rm -f $(PROGRAM)