NAME

ccache - a fast C/C++ compiler cache

SYNOPSIS

ccache [options] ccache compiler [compiler options] compiler [compiler options] (via symbolic link)

DESCRIPTION

ccache is a compiler cache. It speeds up recompilation of C/C++ code by caching the result of previous compilations and detecting when the same compilation is being done again.

ccache has been carefully written to always produce exactly the same compiler output that you would get without the cache. The only way you should be able to tell that you are using ccache is the speed. Currently known exceptions to this goal are listed under BUGS. If you ever discover an undocumented case where ccache changes the output of your compiler, please let us know.

FEATURES

  • Keeps statistics on hits/misses.

  • Automatic cache size management.

  • Can cache compilations that generate warnings.

  • Easy installation.

  • Low overhead.

  • Optionally uses hard links where possible to avoid copies.

  • Optionally compresses files in the cache to reduce disk space.

LIMITATIONS

  • Only knows how to cache the compilation of a single C/C++ file. Other types of compilations (multi-file compilation, linking, etc) will silently fall back to running the real compiler.

  • Only works with GCC and compilers that behave similar enough.

  • Some compiler flags are not supported. If such a flag is detected, ccache will silently fall back to running the real compiler.

RUN MODES

There are two ways to use ccache. You can either prefix your compilation commands with ccache or you can create a symbolic link (named as your compiler) to ccache. The first method is most convenient if you just want to try out ccache or wish to use it for some specific projects. The second method is most useful for when you wish to use ccache for all your compilations.

To install for usage by the first method just make sure ccache is in your path.

To install for the second method, do something like this:

cp ccache /usr/local/bin/
ln -s ccache /usr/local/bin/gcc
ln -s ccache /usr/local/bin/g++
ln -s ccache /usr/local/bin/cc
ln -s ccache /usr/local/bin/c++

And so forth. This will work as long as /usr/local/bin comes before the path to the compiler (which is usually in /usr/bin). After installing you may wish to run “which gcc” to make sure that the correct link is being used.

Note
Do not use a hard link, use a symbolic link. A hard link will cause “interesting” problems.

OPTIONS

These options only apply when you invoke ccache as “ccache”. When invoked as a compiler (via a symlink as described in the previous section), none of these options apply. In that case your normal compiler options apply and you should refer to the compiler’s documentation.

-c, --cleanup

Clean the cache and recalculate the cache file count and size totals. Normally the -c option should not be necessary as ccache keeps the cache below the specified limits at runtime and keeps statistics up to date on each compile. This option is mostly useful if you manually modify the cache contents or believe that the cache size statistics may be inaccurate.

-C, --clear

Clear the entire cache, removing all cached files.

-F, --max-files=N

Set the maximum number of object files allowed in the cache. The value is stored inside the cache directory and applies to all future compilations. Due to the way the value is stored the actual value used is always rounded down to the nearest multiple of 16.

-h, --help

Print an options summary page.

-M, --max-size=SIZE

Set the maximum size of the object files stored in the cache. You can specify a value in gigabytes, megabytes or kilobytes by appending a G, M or K to the value. The default is gigabytes. The actual value stored is rounded down to the nearest multiple of 16 kilobytes.

-s, --show-stats

Print the current statistics summary for the cache.

-V, --version

Print version and copyright information.

-z, --zero-stats

Zero the cache statistics (but not the configured limits).

EXTRA OPTIONS

When run as a compiler, ccache usually just takes the same command line options as the compiler you are using. The only exception to this is the option --ccache-skip. That option can be used to tell ccache that the next option is definitely not a input filename, and should be passed along to the compiler as-is.

The reason this can be important is that ccache does need to parse the command line and determine what is an input filename and what is a compiler option, as it needs the input filename to determine the name of the resulting object file (among other things). The heuristic ccache uses when parsing the command line is that any argument that exists as a file is treated as an input file name (usually a C/C++ file). By using --ccache-skip you can force an option to not be treated as an input file name and instead be passed along to the compiler as a command line option.

ENVIRONMENT VARIABLES

ccache uses a number of environment variables to control operation. In most cases you won’t need any of these as the defaults will be fine.

CCACHE_BASEDIR

If you set the environment variable CCACHE_BASEDIR to an absolute path, ccache rewrites absolute paths into relative paths before computing the hash that identifies the compilation, but only for paths under the specified directory. See the discussion under COMPILING IN DIFFERENT DIRECTORIES.

CCACHE_CC

You can optionally set CCACHE_CC to force the name of the compiler to use. If you don’t do this then ccache works it out from the command line.

CCACHE_COMPRESS

If you set the environment variable CCACHE_COMPRESS then ccache will compress object files and other compiler output it puts in the cache. However, this setting has no effect on how files are retrieved from the cache; compressed and uncompressed results will still be usable regardless of this setting.

CCACHE_COMPILERCHECK

By default, ccache includes the modification time (mtime) and size of the compiler in the hash to ensure that results retrieved from the cache are accurate. The CCACHE_COMPILERCHECK environment variable can be used to select another strategy. Possible values are: none (don’t hash anything), mtime (hash the compiler’s mtime and size) and content (hash the content of the compiler binary). Setting the variable to content makes ccache very slightly slower, but makes it cope better with compiler upgrades during a build bootstrapping process. The none setting may be good for situations where you can safely use the cached results even though the compiler’s mtime or size has changed (e.g. if the compiler is built as part of your build system and the compiler’s source has not changed, or if the compiler only has changes that don’t affect code generation). You should only set the variable to none if you know what you are doing.

CCACHE_CPP2

If you set the environment variable CCACHE_CPP2 then ccache will not use the optimisation of avoiding the second call to the preprocessor by compiling the preprocessed output that was used for finding the hash in the case of a cache miss. This is primarily a debugging option, although it is possible that some unusual compilers will have problems with the intermediate filename extensions used in this optimisation, in which case this option could allow ccache to be used.

CCACHE_DIR

The CCACHE_DIR environment variable specifies where ccache will keep its cached compiler output. The default is $HOME/.ccache.

CCACHE_DISABLE

If you set the environment variable CCACHE_DISABLE then ccache will just call the real compiler, bypassing the cache completely.

CCACHE_EXTENSION

Normally ccache tries to automatically determine the extension to use for intermediate C preprocessor files based on the type of file being compiled. Unfortunately this sometimes doesn’t work, for example when using the “aCC” compiler on HP-UX. On systems like this you can use the CCACHE_EXTENSION option to override the default. On HP-UX set this environment variable to i if you use the “aCC” compiler.

CCACHE_EXTRAFILES

If you set the environment variable CCACHE_EXTRAFILES to a colon-separated list of paths then ccache will include the contents of those files when calculating the hash sum.

CCACHE_HARDLINK

If you set the environment variable CCACHE_HARDLINK then ccache will attempt to use hard links from the cache directory when creating the compiler output rather than using a file copy. Using hard links may be slightly faster in some situations, but can confuse programs like “make” that rely on modification times. Hard links are never made for compressed cache files. This means that you should not set the CCACHE_COMPRESS variable if you want to use hard links.

CCACHE_HASHDIR

This tells ccache to hash the current working directory when calculating the hash that is used to distinguish two compiles. This prevents a problem with the storage of the current working directory in the debug info of a object file, which can lead ccache to give a cached object file that has the working directory in the debug info set incorrectly. This option is off by default as the incorrect setting of this debug info rarely causes problems. If you strike problems with GDB not using the correct directory then enable this option.

CCACHE_LOGFILE

If you set the CCACHE_LOGFILE environment variable then ccache will write some information on what it is doing to the log. This is useful for tracking down problems.

CCACHE_NLEVELS

The environment variable CCACHE_NLEVELS allows you to choose the number of levels of hash in the cache directory. The default is 2. The minimum is 1 and the maximum is 8.

CCACHE_NODIRECT

If you set the environment variable CCACHE_NODIRECT then ccache will not use the direct mode.

CCACHE_NOSTATS

If you set the environment variable CCACHE_NOSTATS then ccache will not update the statistics files on each compilation.

CCACHE_PATH

You can optionally set CCACHE_PATH to a colon-separated path where ccache will look for the real compilers. If you don’t do this then ccache will look for the first executable matching the compiler name in the normal PATH that isn’t a symbolic link to ccache itself.

CCACHE_PREFIX

This option adds a prefix to the command line that ccache runs when invoking the compiler. Also see the section below on using ccache with “distcc”.

CCACHE_READONLY

The CCACHE_READONLY environment variable tells ccache to attempt to use existing cached object files, but not to try to add anything new to the cache. If you are using this because your CCACHE_DIR is read-only, then you may find that you also need to set CCACHE_TEMPDIR as otherwise ccache will fail to create temporary files.

CCACHE_RECACHE

This forces ccache to not use any cached results, even if it finds them. New results are still cached, but existing cache entries are ignored.

CCACHE_TEMPDIR

The CCACHE_TEMPDIR environment variable specifies where ccache will put temporary files. The default is $CCACHE_DIR/tmp. (Note: In previous versions of ccache, CCACHE_TEMPDIR had to be on the same filesystem as the CCACHE_DIR path, but this requirement has been relaxed.)

CCACHE_UMASK

This sets the umask for ccache and all child processes (such as the compiler). This is mostly useful when you wish to share your cache with other users. Note that this also affects the file permissions set on the object files created from your compilations.

CCACHE_UNIFY

If you set the environment variable CCACHE_UNIFY then ccache will use a C/C++ unifier when hashing the preprocessor output if the -g option is not used. The unifier is slower than a normal hash, so setting this environment variable loses a little bit of speed, but it means that ccache can take advantage of not recompiling when the changes to the source code consist of reformatting only. Note that using CCACHE_UNIFY changes the hash, so cached compilations with CCACHE_UNIFY set cannot be used when CCACHE_UNIFY is not set and vice versa. The reason the unifier is off by default is that it can give incorrect line number information in compiler warning messages. Enabling the unifier implies turning off the direct mode.

CCACHE_VERBOSE

If you set the environment variable CCACHE_VERBOSE, ccache prints (to the standard output) the executed command lines when running the preprocessor and the compiler.

CACHE SIZE MANAGEMENT

By default ccache has a one gigabyte limit on the total size of object files in the cache and no maximum number of object files. You can set different limits using the -M/--max-size and -F/--max-files options. Use ccache -s to see the cache size and the currently configured limits (in addition to other various statistics).

CACHE COMPRESSION

ccache can optionally compress all files it puts into the cache using the compression library zlib. While this involves a negligible performance slowdown, it significantly increases the number of files that fit in the cache. You can turn on compression by setting the CCACHE_COMPRESS environment variable.

HOW IT WORKS

The basic idea is to detect when you are compiling exactly the same code a second time and reuse the previously produced output. The detection is done by hashing different kinds of information that should be unique for the compilation and then using the hash sum to identify the cached output. ccache uses MD4, a very fast cryptographic hash algorithm, for the hashing. (MD4 is nowadays too weak to be useful in cryptographic contexts, but it should be safe enough to be used to identify recompilations.) On a cache hit, ccache is able to supply all of the correct compiler outputs (including all warnings, dependency file, etc) from the cache.

ccache has two ways of doing the detection:

  • the direct mode (hashes the source code and include files directly)

  • the preprocessor mode (hashes output from the preprocessor)

THE DIRECT MODE

In the direct mode, the hash is formed of:

  • the input source file

  • the command line options

  • the real compiler’s size and modification time (unless CCACHE_COMPILERCHECK says something else)

Based on the hash, a data structure called “manifest” is looked up in the cache. The manifest contains:

  • references to cached compilation results (object file, dependency file, etc) that were produced by previous compilations that matched the hash

  • paths to the include files that were read at the time the compilation results were stored in the cache

  • hash sums of the include files at the time the compilation results were stored in the cache

The current contents of the include files are then hashed and compared to the information in the manifest. If there is a match, ccache knows the result of the compilation. If there is no match, ccache falls back to running the preprocessor. The output from the preprocessor is parsed to find the include files that were read. The paths and hash sums of those include files are then stored in the manifest along with information about the produced compilation result.

The direct mode will be disabled if any of the following holds:

  • the environment variable CCACHE_NODIRECT is set

  • a modification time of one of the include files is too new (needed to avoid a race condition)

  • the unifier is enabled (the environment variable CCACHE_UNIFY is set)

  • a -Wp,X compiler option other than -Wp,-MD,path and -Wp,-MMD,path is used

  • the string “__TIME__” is present outside comments and string literals in the source code

THE PREPROCESSOR MODE

In the preprocessor mode, the hash is formed of:

  • the preprocessor output from running the compiler with -E

  • the command line options except options that affect include files (-I, -include, -D, etc; the theory is that these options will change the preprocessor output if they have any effect at all)

  • the real compiler’s size and modification time (unless CCACHE_COMPILERCHECK says something else)

  • any standard error output generated by the preprocessor

Based on the hash, the cached compilation result can be looked up directly in the cache.

COMPILING IN DIFFERENT DIRECTORIES

Some information included in the hash that identifies a unique compilation may contain absolute paths:

  • The preprocessed source code may contain absolute paths to include files if the compiler option -g is used or if absolute paths are given to -I and similar compiler options.

  • Paths specified by compiler options (such as -I, -MF, etc) may be absolute.

  • The source code file path may be absolute, and that path may substituted for __FILE__ macros in the source code or included in warnings emitted to standard error by the preprocessor.

This means that if you compile the same code in different locations, you can’t share compilation results between the different build directories since you get cache misses because of the absolute build directory paths that are part of the hash. To mitigate this problem, you can specify a “base directory” by setting the CCACHE_BASEDIR variable to an absolute path. ccache will then rewrite absolute paths that are under the base directory (i.e., paths that have the base directory as a prefix) to relative paths when constructing the hash. The typical path to use as base directory is your home directory or another directory that is a parent of your build directories. (Don’t use / as the base directory since that will make ccache also rewrite paths to system header files, which doesn’t gain anything.)

The drawbacks of using CCACHE_BASEDIR are:

  • If you specify an absolute path to the source code file, __FILE__ macros will be expanded to a relative path instead.

  • If you specify an absolute path to the source code file and compile with -g, the source code path stored in the object file may point to the wrong directory, which may prevent debuggers like GDB from finding the source code. Sometimes, a work-around is to change the directory explicitly with the “cd” command in GDB.

SHARING A CACHE

A group of developers can increase the cache hit rate by sharing a cache directory. To share a cache without unpleasant side effects, the following conditions should to be met:

  • Use the same CCACHE_DIR environment variable setting.

  • Unset the CCACHE_HARDLINK environment variable.

  • Make sure everyone sets the CCACHE_UMASK environment variable to 002. This ensure that cached files are accessible to everyone in the group.

  • Make sure that all users have write permission in the entire cache directory (and that you trust all users of the shared cache).

  • Make sure that the setgid bit is set on all directories in the cache. This tells the filesystem to inherit group ownership for new directories. The command “find $CCACHE_DIR -type d | xargs chmod g+s” might be useful for this.

The reason to avoid the hard link mode is that the hard links cause unwanted side effects, as all links to a cached file share the file’s modification timestamp. This results in false dependencies to be triggered by timestamp-based build systems whenever another user links to an existing file. Typically, users will see that their libraries and binaries are relinked without reason.

You may also want to make sure that the developers have CCACHE_BASEDIR set appropriately, as discussed in the previous section.

SHARING A CACHE ON NFS

It is possible to put the cache directory on an NFS filesystem (or similar filesystems), but keep in mind that:

  • Having the cache on NFS may slow down compilation. Make sure to do some benchmarking to see if it’s worth it.

  • ccache hasn’t been tested very thoroughly on NFS.

A tip is to set CCACHE_TEMPDIR to a directory on the local host to avoid NFS traffic for temporary files.

USING CCACHE WITH DISTCC

“distcc” is a very useful program for distributing compilation across a range of compiler servers. It is often useful to combine distcc with ccache so that compilations that are done are sped up by distcc, but ccache avoids the compilation completely where possible.

The recommended way of combining distcc and ccache is by using the CCACHE_PREFIX option. You just need to set the environment variable CCACHE_PREFIX to distcc and ccache will prefix the command line used with the compiler with the command “distcc”.

BUGS

ccache doesn’t handle the GNU Assembler’s .incbin directive correctly. This directive can be embedded in the source code inside an asm statement in order to include a file verbatim in the object file. If the included file is changed, ccache doesn’t pick up the change since the inclusion isn’t done by the preprocessor. A workaround of this problem is to set CCACHE_EXTRAFILES to the path of the included file.

MORE INFORMATION

Credits, mailing list information, bug reporting instructions, source code, etc, can be found on ccache’s web site: http://ccache.samba.org.

AUTHOR

ccache was originally written by Andrew Tridgell and is currently maintained by Joel Rosdahl. See http://ccache.samba.org/credits.html for a list of contributors.