Contents
The ExternalProject_Add() function creates a custom target to drive download, update/patch, configure, build, install and test steps of an external project:
ExternalProject_Add(<name> [<option>...])
The individual steps within the process can be driven independently if required (e.g. for CDash submission) and extra custom steps can be defined, along with the ability to control the step dependencies. The directory structure used for the management of the external project can also be customized. The function supports a large number of options which can be used to tailor the external project behavior.
Most of the time, the default directory layout is sufficient. It is largely an implementation detail that the main project usually doesn’t need to change. In some circumstances, however, control over the directory layout can be useful or necessary. The directory options are potentially more useful from the point of view that the main build can use the ExternalProject_Get_Property() command to retrieve their values, thereby allowing the main project to refer to build artifacts of the external project.
Source directory into which downloaded contents will be unpacked, or for non-URL download methods, the directory in which the repository should be checked out, cloned, etc. If no download method is specified, this must point to an existing directory where the external project has already been unpacked or cloned/checked out.
Note
If a download method is specified, any existing contents of the source directory may be deleted. Only the URL download method checks whether this directory is either missing or empty before initiating the download, stopping with an error if it is not empty. All other download methods silently discard any previous contents of the source directory.
If any of the above ..._DIR options are not specified, their defaults are computed as follows. If the PREFIX option is given or the EP_PREFIX directory property is set, then an external project is built and installed under the specified prefix:
TMP_DIR = <prefix>/tmp
STAMP_DIR = <prefix>/src/<name>-stamp
DOWNLOAD_DIR = <prefix>/src
SOURCE_DIR = <prefix>/src/<name>
BINARY_DIR = <prefix>/src/<name>-build
INSTALL_DIR = <prefix>
LOG_DIR = <STAMP_DIR>
Otherwise, if the EP_BASE directory property is set then components of an external project are stored under the specified base:
TMP_DIR = <base>/tmp/<name>
STAMP_DIR = <base>/Stamp/<name>
DOWNLOAD_DIR = <base>/Download/<name>
SOURCE_DIR = <base>/Source/<name>
BINARY_DIR = <base>/Build/<name>
INSTALL_DIR = <base>/Install/<name>
LOG_DIR = <STAMP_DIR>
If no PREFIX, EP_PREFIX, or EP_BASE is specified, then the default is to set PREFIX to <name>-prefix. Relative paths are interpreted with respect to CMAKE_CURRENT_BINARY_DIR at the point where ExternalProject_Add() is called.
A download method can be omitted if the SOURCE_DIR option is used to point to an existing non-empty directory. Otherwise, one of the download methods below must be specified (multiple download methods should not be given) or a custom DOWNLOAD_COMMAND provided.
Specify whether the .netrc file is to be used for operation. If this option is not specified, the value of the CMAKE_NETRC variable will be used instead (see file(DOWNLOAD)) Valid levels are:
NOTE: A git version of 1.6.5 or later is required if this download method is used.
Git branch name, tag or commit hash. Note that branch names and tags should generally be specified as remote names (i.e. origin/myBranch rather than simply myBranch). This ensures that if the remote end has its tag moved or branch rebased or history rewritten, the local clone will still be updated correctly. In general, however, specifying a commit hash should be preferred for a number of reasons:
If GIT_SHALLOW is enabled then GIT_TAG works only with branch names and tags. A commit hash is not allowed.
When GIT_TAG refers to a remote branch, this option can be used to specify how the update step behaves. The <strategy> must be one of the following:
The variable CMAKE_EP_GIT_REMOTE_UPDATE_STRATEGY can be set to override the default strategy. This variable should not be set by a project, it is intended for the user to set. It is primarily intended for use in continuous integration scripts to ensure that when history is rewritten on a remote branch, the build doesn’t end up with unintended changes or failed builds resulting from conflicts during rebase operations.
Whenever CMake is re-run, by default the external project’s sources will be updated if the download method supports updates (e.g. a git repository would be checked if the GIT_TAG does not refer to a specific commit).
When enabled, this option causes the update step to be skipped. It does not, however, prevent the download step. The update step can still be added as a step target (see ExternalProject_Add_StepTargets()) and called manually. This is useful if you want to allow developers to build the project when disconnected from the network (the network may still be needed for the download step though).
When this option is present, it is generally advisable to make the value a cache variable under the developer’s control rather than hard-coding it. If this option is not present, the default value is taken from the EP_UPDATE_DISCONNECTED directory property. If that is also not defined, updates are performed as normal. The EP_UPDATE_DISCONNECTED directory property is intended as a convenience for controlling the UPDATE_DISCONNECTED behavior for an entire section of a project’s directory hierarchy and may be a more convenient method of giving developers control over whether or not to perform updates (assuming the project also provides a cache variable or some other convenient method for setting the directory property).
The configure step is run after the download and update steps. By default, the external project is assumed to be a CMake project, but this can be overridden if required.
This is the same as the CMAKE_CACHE_ARGS option except the set() commands do not include the FORCE keyword. This means the values act as initial defaults only and will not override any variables already set from a previous run. Use this option with care, as it can lead to different behavior depending on whether the build starts from a fresh build directory or re-uses previous build contents.
If the CMake generator is the Green Hills MULTI and not overridden then the original project’s settings for the GHS toolset and target system customization cache variables are propagated into the external project.
If the configure step assumed the external project uses CMake as its build system, the build step will also. Otherwise, the build step will assume a Makefile-based build and simply run make with no arguments as the default build step. This can be overridden with custom build commands if required.
If the configure step assumed the external project uses CMake as its build system, the install step will also. Otherwise, the install step will assume a Makefile-based build and simply run make install as the default build step. This can be overridden with custom install commands if required.
The test step is only defined if at least one of the following TEST_... options are provided.
Each of the following LOG_... options can be used to wrap the relevant step in a script to capture its output to files. The log files will be created in LOG_DIR if supplied or otherwise the STAMP_DIR directory with step-specific file names.
Steps can be given direct access to the terminal in some cases. Giving a step access to the terminal may allow it to receive terminal input if required, such as for authentication details not provided by other options. With the Ninja generator, these options place the steps in the console job pool. Each step can be given access to the terminal individually via the following options:
Any of the other ..._COMMAND options can have additional commands appended to them by following them with as many COMMAND ... options as needed (generator expressions are supported). For example:
ExternalProject_Add(example
... # Download options, etc.
BUILD_COMMAND ${CMAKE_COMMAND} -E echo "Starting $<CONFIG> build"
COMMAND ${CMAKE_COMMAND} --build <BINARY_DIR> --config $<CONFIG>
COMMAND ${CMAKE_COMMAND} -E echo "$<CONFIG> build complete"
)
It should also be noted that each build step is created via a call to ExternalProject_Add_Step(). See that command’s documentation for the automatic substitutions that are supported for some options.
The ExternalProject_Get_Property() function retrieves external project target properties:
ExternalProject_Get_Property(<name> <prop1> [<prop2>...])
The function stores property values in variables of the same name. Property names correspond to the keyword argument names of ExternalProject_Add(). For example, the source directory might be retrieved like so:
ExternalProject_Get_property(myExtProj SOURCE_DIR)
message("Source dir of myExtProj = ${SOURCE_DIR}")
The ExternalProject_Add() function on its own is often sufficient for incorporating an external project into the main build. Certain scenarios require additional work to implement desired behavior, such as adding in a custom step or making steps available as manually triggerable targets. The ExternalProject_Add_Step(), ExternalProject_Add_StepTargets() and ExternalProject_Add_StepDependencies functions provide the lower level control needed to implement such step-level capabilities.
The ExternalProject_Add_Step() function specifies an additional custom step for an external project defined by an earlier call to ExternalProject_Add():
ExternalProject_Add_Step(<name> <step> [<option>...])
<name> is the same as the name passed to the original call to ExternalProject_Add(). The specified <step> must not be one of the pre-defined steps (mkdir, download, update, patch, configure, build, install or test). The supported options are:
The command line, comment, working directory and byproducts of every standard and custom step are processed to replace the tokens <SOURCE_DIR>, <SOURCE_SUBDIR>, <BINARY_DIR>, <INSTALL_DIR> <TMP_DIR>, <DOWNLOAD_DIR> and <DOWNLOADED_FILE> with their corresponding property values defined in the original call to ExternalProject_Add().
The ExternalProject_Add_StepTargets() function generates targets for the steps listed. The name of each created target will be of the form <name>-<step>:
ExternalProject_Add_StepTargets(<name> [NO_DEPENDS] <step1> [<step2>...])
Creating a target for a step allows it to be used as a dependency of another target or to be triggered manually. Having targets for specific steps also allows them to be driven independently of each other by specifying targets on build command lines. For example, you may be submitting to a sub-project based dashboard where you want to drive the configure portion of the build, then submit to the dashboard, followed by the build portion, followed by tests. If you invoke a custom target that depends on a step halfway through the step dependency chain, then all the previous steps will also run to ensure everything is up to date.
If the NO_DEPENDS option is specified, the step target will not depend on the dependencies of the external project (i.e. on any dependencies of the <name> custom target created by ExternalProject_Add()). This is usually safe for the download, update and patch steps, since they do not typically require that the dependencies are updated and built. Using NO_DEPENDS for any of the other pre-defined steps, however, may break parallel builds. Only use NO_DEPENDS where it is certain that the named steps genuinely do not have dependencies. For custom steps, consider whether or not the custom commands require the dependencies to be configured, built and installed.
Internally, ExternalProject_Add() calls ExternalProject_Add_Step() to create each step. If any STEP_TARGETS or INDEPENDENT_STEP_TARGETS were specified, then ExternalProject_Add_StepTargets() will also be called after ExternalProject_Add_Step(). INDEPENDENT_STEP_TARGETS have the NO_DEPENDS option set, whereas STEP_TARGETS do not. Other than that, the two options result in ExternalProject_Add_StepTargets() being called in the same way. Even if a step is not mentioned in either of those two options, ExternalProject_Add_StepTargets() can still be called later to manually define a target for the step.
The STEP_TARGETS and INDEPENDENT_STEP_TARGETS options for ExternalProject_Add() are generally the easiest way to ensure targets are created for specific steps of interest. For custom steps, ExternalProject_Add_StepTargets() must be called explicitly if a target should also be created for that custom step. An alternative to these two options is to populate the EP_STEP_TARGETS and EP_INDEPENDENT_STEP_TARGETS directory properties. These act as defaults for the step target options and can save having to repeatedly specify the same set of step targets when multiple external projects are being defined.
The ExternalProject_Add_StepDependencies() function can be used to add dependencies to a step. The dependencies added must be targets CMake already knows about (these can be ordinary executable or library targets, custom targets or even step targets of another external project):
ExternalProject_Add_StepDependencies(<name> <step> <target1> [<target2>...])
This function takes care to set both target and file level dependencies and will ensure that parallel builds will not break. It should be used instead of add_dependencies() whenever adding a dependency for some of the step targets generated by the ExternalProject module.
The following example shows how to download and build a hypothetical project called FooBar from github:
include(ExternalProject)
ExternalProject_Add(foobar
GIT_REPOSITORY git@github.com:FooCo/FooBar.git
GIT_TAG origin/release/1.2.3
)
For the sake of the example, also define a second hypothetical external project called SecretSauce, which is downloaded from a web server. Two URLs are given to take advantage of a faster internal network if available, with a fallback to a slower external server. The project is a typical Makefile project with no configure step, so some of the default commands are overridden. The build is only required to build the sauce target:
find_program(MAKE_EXE NAMES gmake nmake make)
ExternalProject_Add(secretsauce
URL http://intranet.somecompany.com/artifacts/sauce-2.7.tgz
https://www.somecompany.com/downloads/sauce-2.7.zip
URL_HASH MD5=d41d8cd98f00b204e9800998ecf8427e
CONFIGURE_COMMAND ""
BUILD_COMMAND ${MAKE_EXE} sauce
)
Suppose the build step of secretsauce requires that foobar must already be built. This could be enforced like so:
ExternalProject_Add_StepDependencies(secretsauce build foobar)
Another alternative would be to create a custom target for foobar‘s build step and make secretsauce depend on that rather than the whole foobar project. This would mean foobar only needs to be built, it doesn’t need to run its install or test steps before secretsauce can be built. The dependency can also be defined along with the secretsauce project:
ExternalProject_Add_StepTargets(foobar build)
ExternalProject_Add(secretsauce
URL http://intranet.somecompany.com/artifacts/sauce-2.7.tgz
https://www.somecompany.com/downloads/sauce-2.7.zip
URL_HASH MD5=d41d8cd98f00b204e9800998ecf8427e
CONFIGURE_COMMAND ""
BUILD_COMMAND ${MAKE_EXE} sauce
DEPENDS foobar-build
)
Instead of calling ExternalProject_Add_StepTargets(), the target could be defined along with the foobar project itself:
ExternalProject_Add(foobar
GIT_REPOSITORY git@github.com:FooCo/FooBar.git
GIT_TAG origin/release/1.2.3
STEP_TARGETS build
)
If many external projects should have the same set of step targets, setting a directory property may be more convenient. The build step target could be created automatically by setting the EP_STEP_TARGETS directory property before creating the external projects with ExternalProject_Add():
set_property(DIRECTORY PROPERTY EP_STEP_TARGETS build)
Lastly, suppose that secretsauce provides a script called makedoc which can be used to generate its own documentation. Further suppose that the script expects the output directory to be provided as the only parameter and that it should be run from the secretsauce source directory. A custom step and a custom target to trigger the script can be defined like so:
ExternalProject_Add_Step(secretsauce docs
COMMAND <SOURCE_DIR>/makedoc <BINARY_DIR>
WORKING_DIRECTORY <SOURCE_DIR>
COMMENT "Building secretsauce docs"
ALWAYS TRUE
EXCLUDE_FROM_MAIN TRUE
)
ExternalProject_Add_StepTargets(secretsauce docs)
The custom step could then be triggered from the main build like so:
cmake --build . --target secretsauce-docs