Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Derivations and Deriving Paths #12290

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/manual/source/SUMMARY.md.in
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
- [Store Object](store/store-object.md)
- [Content-Addressing Store Objects](store/store-object/content-address.md)
- [Store Path](store/store-path.md)
- [Derivation and Deriving Path](store/drv.md)
- [Building](store/drv/building.md)
- [Store Types](store/types/index.md)
{{#include ./store/types/SUMMARY.md}}
- [Nix Language](language/index.md)
Expand Down
6 changes: 3 additions & 3 deletions doc/manual/source/command-ref/nix-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ directory containing at least a file named `default.nix`.

`nix-build` is essentially a wrapper around
[`nix-instantiate`](nix-instantiate.md) (to translate a high-level Nix
expression to a low-level [store derivation]) and [`nix-store
expression to a low-level [derivation]) and [`nix-store
--realise`](@docroot@/command-ref/nix-store/realise.md) (to build the store
derivation).

[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-derivation

> **Warning**
>
Expand Down Expand Up @@ -80,7 +80,7 @@ except for `--arg` and `--attr` / `-A` which are passed to [`nix-instantiate`](n

```console
$ nix-build '<nixpkgs>' --attr firefox
store derivation is /nix/store/qybprl8sz2lc...-firefox-1.5.0.7.drv
derivation is /nix/store/qybprl8sz2lc...-firefox-1.5.0.7.drv
/nix/store/d18hyl92g30l...-firefox-1.5.0.7

$ ls -l result
Expand Down
4 changes: 2 additions & 2 deletions doc/manual/source/command-ref/nix-copy-closure.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ When using public key authentication, you can avoid typing the passphrase with `

- `--include-outputs`

Also copy the outputs of [store derivation]s included in the closure.
Also copy the outputs of [derivation]s included in the closure.

[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-store-derivation

- `--use-substitutes` / `-s`

Expand Down
8 changes: 4 additions & 4 deletions doc/manual/source/command-ref/nix-env/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The arguments *args* map to store paths in a number of possible ways:
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.

[derivation]: @docroot@/glossary.md#gloss-derivation
[derivation expression]: @docroot@/glossary.md#gloss-derivation-expression
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise

Expand Down Expand Up @@ -66,9 +66,9 @@ The arguments *args* map to store paths in a number of possible ways:
This can be used to override the priority of the derivations being installed.
This is useful if *args* are [store paths], which don't have any priority information.

- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store paths] to [derivations](@docroot@/glossary.md#gloss-derivation), then those derivations are [realised], and the resulting output paths are installed.

- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] not to derivations, then these are [realised] and installed.

- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Expand Down Expand Up @@ -198,7 +198,7 @@ To copy the store path with symbolic name `gcc` from another profile:
$ nix-env --install --from-profile /nix/var/nix/profiles/foo gcc
```

To install a specific [store derivation] (typically created by
To install a specific [derivation] (typically created by
`nix-instantiate`):

```console
Expand Down
5 changes: 4 additions & 1 deletion doc/manual/source/command-ref/nix-env/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,10 @@ derivation is shown unless `--no-name` is specified.

- `--drv-path`

Print the path of the [store derivation](@docroot@/glossary.md#gloss-store-derivation).
Print the [store path] to the [derivation].

[store path]: @docroot@/glossary.md#gloss-store-path
[derivation]: @docroot@/glossary.md#gloss-derivation

- `--out-path`

Expand Down
14 changes: 7 additions & 7 deletions doc/manual/source/command-ref/nix-instantiate.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Name

`nix-instantiate` - instantiate store derivations from Nix expressions
`nix-instantiate` - instantiate derivations from Nix expressions

# Synopsis

Expand All @@ -17,13 +17,13 @@

# Description

The command `nix-instantiate` produces [store derivation]s from (high-level) Nix expressions.
The command `nix-instantiate` produces [derivation]s from (high-level) Nix expressions.
It evaluates the Nix expressions in each of *files* (which defaults to
*./default.nix*). Each top-level expression should evaluate to a
derivation, a list of derivations, or a set of derivations. The paths
of the resulting store derivations are printed on standard output.
of the resulting derivations are printed on standard output.

[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-derivation

If *files* is the character `-`, then a Nix expression will be read from
standard input.
Expand All @@ -42,8 +42,8 @@ standard input.
- `--eval`

Just parse and evaluate the input files, and print the resulting
values on standard output. No instantiation of store derivations
takes place.
values on standard output.
Derivations are not serialized and written to the store, but instead just discarded.
Ericson2314 marked this conversation as resolved.
Show resolved Hide resolved

> **Warning**
>
Expand Down Expand Up @@ -133,7 +133,7 @@ standard input.

# Examples

Instantiate [store derivation]s from a Nix expression, and build them using `nix-store`:
Instantiate [derivation]s from a Nix expression, and build them using `nix-store`:

```console
$ nix-instantiate test.nix (instantiate)
Expand Down
14 changes: 7 additions & 7 deletions doc/manual/source/command-ref/nix-store/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ symlink.

- `--use-output` / `-u`

For each argument to the query that is a [store derivation], apply the
For each argument to the query that is a [derivation], apply the
query to the output path of the derivation instead.

- `--force-realise` / `-f`

Realise each argument to the query first (see [`nix-store --realise`](./realise.md)).

[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-derivation

# Queries

Expand All @@ -54,12 +54,12 @@ symlink.
This query has one option:

- `--include-outputs`
Also include the existing output paths of [store derivation]s,
Also include the existing output paths of [derivation]s,
and their closures.

This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
store derivation. A *binary deployment* is obtained by distributing
derivation. A *binary deployment* is obtained by distributing
the closure of an output path. A *cache deployment* (combined
source/binary deployment, including binaries of build-time-only
dependencies) is obtained by distributing the closure of a store
Expand Down Expand Up @@ -112,7 +112,7 @@ symlink.
of the `dot` tool of AT\&T's [Graphviz
package](http://www.graphviz.org/). This can be used to visualise
dependency graphs. To obtain a build-time dependency graph, apply
this to a store derivation. To obtain a runtime dependency graph,
this to a derivation. To obtain a runtime dependency graph,
apply it to an output path.

- `--tree`
Expand All @@ -128,13 +128,13 @@ symlink.
Prints the references graph of the store paths *paths* in the
[GraphML](http://graphml.graphdrawing.org/) file format. This can be
used to visualise dependency graphs. To obtain a build-time
dependency graph, apply this to a [store derivation]. To obtain a
dependency graph, apply this to a [derivation]. To obtain a
runtime dependency graph, apply it to an output path.

- `--binding` *name* / `-b` *name*

Prints the value of the attribute *name* (i.e., environment
variable) of the [store derivation]s *paths*. It is an error for a
variable) of the [derivation]s *paths*. It is an error for a
derivation to not have the specified attribute.

- `--hash`
Expand Down
12 changes: 6 additions & 6 deletions doc/manual/source/command-ref/nix-store/realise.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@

Each of *paths* is processed as follows:

- If the path leads to a [store derivation]:
1. If it is not [valid], substitute the store derivation file itself.
- If the path leads to a [derivation]:
1. If it is not [valid], substitute the derivation file itself.
2. Realise its [output paths]:
- Try to fetch from [substituters] the [store objects] associated with the output paths in the store derivation's [closure].
- Try to fetch from [substituters] the [store objects] associated with the output paths in the derivation's [closure].
- With [content-addressed derivations] (experimental):
Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database].
- For any store paths that cannot be substituted, produce the required store objects:
Expand All @@ -23,11 +23,11 @@ Each of *paths* is processed as follows:
<!-- TODO: Link to build process page #8888 -->
- Otherwise, and if the path is not already valid: Try to fetch the associated [store objects] in the path's [closure] from [substituters].

If no substitutes are available and no store derivation is given, realisation fails.
If no substitutes are available and no derivation is given, realisation fails.

[store paths]: @docroot@/store/store-path.md
[valid]: @docroot@/glossary.md#gloss-validity
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-derivation
[output paths]: @docroot@/glossary.md#gloss-output-path
[store objects]: @docroot@/store/store-object.md
[closure]: @docroot@/glossary.md#gloss-closure
Expand Down Expand Up @@ -71,7 +71,7 @@ For non-derivation arguments, the argument itself is printed.

# Examples

This operation is typically used to build [store derivation]s produced by
This operation is typically used to build [derivation]s produced by
[`nix-instantiate`](@docroot@/command-ref/nix-instantiate.md):

```console
Expand Down
40 changes: 15 additions & 25 deletions doc/manual/source/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,31 +21,26 @@

- [derivation]{#gloss-derivation}

A description of a build task. The result of a derivation is a
store object. Derivations declared in Nix expressions are specified
using the [`derivation` primitive](./language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-build`, or explicitly by `nix-instantiate`).
A single build task.
See [Derivation](@docroot@/store/drv.md#derivation) for details.

[derivation]: #gloss-derivation

- [store derivation]{#gloss-store-derivation}
- [derivation expression]{#gloss-derivation}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I don't think this term is precise enough.
The only expressions that produce derivations are those that evaluate to builtins.derivation{,Strict} x for some attrset x at some point during their evaluation, returning a [package attribute set] or output path string referencing the instantiated derivation.

The current definition describes a high-level goal that a human might have, but not an accurate term for the technical process, and not a common term that actual humans use; it'd just be "derivation" or even "package".

I guess we can't just refer tobuiltins.derivation, because we also have derivationStrict, so I guess we could define it more reasonably as:

An expression that instantiates one or more derivations by using `derivation` or `builtins.derivationStrict`, usually through some indirections like [`mkDerivation`](https://nixos.org/manual/nixpkgs/stable/#sec-using-stdenv).

Glossaries are all about relating the terminology to the outside world, so the link to Nixpkgs is ok.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this text was mostly moved around? Could be misremembering that , however.

I don't really want to have a "derivation expression" concept at all (if we make a new, improved, derivation primop, that would make for 3 primops, each with different interfaces, that make derivations. Best to just dispense with the notion and embrace #6507.) But I don't really want to shave that yak in this PR either.


A [derivation] represented as a `.drv` file in the [store].
It has a [store path], like any [store object].
It is the [instantiated][instantiate] form of a derivation.
A description of a [derivation] in the Nix language.
The result of a derivation is a store object.
Ericson2314 marked this conversation as resolved.
Show resolved Hide resolved
Derivations are typically specified in Nix expressions using the [`derivation` primitive](./language/derivations.md).
These are translated into low-level *derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
Ericson2314 marked this conversation as resolved.
Show resolved Hide resolved

Example: `/nix/store/g946hcz4c8mdvq2g8vxx42z51qb71rvp-git-2.38.1.drv`

See [`nix derivation show`](./command-ref/new-cli/nix3-derivation-show.md) (experimental) for displaying the contents of store derivations.

[store derivation]: #gloss-store-derivation
[derivation expression]: #gloss-derivation-expression

- [instantiate]{#gloss-instantiate}, instantiation

Save an evaluated [derivation] as a [store derivation] in the Nix [store].
Translate a [derivation expression] into a [derivation].

See [`nix-instantiate`](./command-ref/nix-instantiate.md), which produces a store derivation from a Nix expression that evaluates to a derivation.
See [`nix-instantiate`](./command-ref/nix-instantiate.md), which produces a derivation from a Nix expression that evaluates to a derivation.

[instantiate]: #gloss-instantiate

Expand Down Expand Up @@ -188,7 +183,7 @@
>
> The contents of a `.nix` file form a Nix expression.

Nix expressions specify [derivations][derivation], which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
Nix expressions specify [derivation expressions][derivation expression], which are [instantiated][instantiate] into the Nix store as [derivations][derivation].
These derivations can then be [realised][realise] to produce [outputs][output].

> **Example**
Expand Down Expand Up @@ -246,18 +241,13 @@

- [deriving path]{#gloss-deriving-path}

Deriving paths are a way to refer to [store objects][store object] that ar not yet [realised][realise].
This is necessary because, in general and particularly for [content-addressed derivations][content-addressed derivation], the [output path] of an [output] is not known in advance.
There are two forms:

- *constant*: just a [store path]
It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store.
Deriving paths are a way to refer to [store objects][store object] that might not be yet [realised][realise].
Ericson2314 marked this conversation as resolved.
Show resolved Hide resolved

- *output*: a pair of a [store path] to a [derivation] and an [output] name.
See [Deriving Path](./store/drv.md#deriving-path) for details.

- [deriver]{#gloss-deriver}

The [store derivation] that produced an [output path].
The [derivation] that produced an [output path].

The deriver for an output path can be queried with the `--deriver` option to
[`nix-store --query`](@docroot@/command-ref/nix-store/query.md).
Expand Down
28 changes: 12 additions & 16 deletions doc/manual/source/language/derivations.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Derivations

The most important built-in function is `derivation`, which is used to describe a single derivation:
a specification for running an executable on precisely defined input files to repeatably produce output files at uniquely determined file system paths.
The most important built-in function is `derivation`, which is used to describe a single store-layer [derivation].
Consult the [store chapter](@docroot@/store/drv.md) for what a derivation is;
this section just concerns how to create one from the Nix language.

It takes as input an attribute set, the attributes of which specify the inputs to the process.
It outputs an attribute set, and produces a [store derivation] as a side effect of evaluation.
This builtin function takes as input an attribute set, the attributes of which specify the inputs to the process.
It outputs an attribute set, and produces a [derivation] as a side effect of evaluation.

[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[derivation]: @docroot@/glossary.md#gloss-derivation

## Input attributes

Expand All @@ -15,7 +16,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
- [`name`]{#attr-name} ([String](@docroot@/language/types.md#type-string))

A symbolic name for the derivation.
It is added to the [store path] of the corresponding [store derivation] as well as to its [output paths](@docroot@/glossary.md#gloss-output-path).
See [derivation outputs](@docroot@/store/drv.md#outputs) for what this is affects.

[store path]: @docroot@/store/store-path.md

Expand All @@ -28,17 +29,12 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> }
> ```
>
> The store derivation's path will be `/nix/store/<hash>-hello.drv`.
> The derivation's path will be `/nix/store/<hash>-hello.drv`.
> The [output](#attr-outputs) paths will be of the form `/nix/store/<hash>-hello[-<output>]`

- [`system`]{#attr-system} ([String](@docroot@/language/types.md#type-string))

The system type on which the [`builder`](#attr-builder) executable is meant to be run.

A necessary condition for Nix to build derivations locally is that the `system` attribute matches the current [`system` configuration option].
It can automatically [build on other platforms](@docroot@/language/derivations.md#attr-builder) by forwarding build requests to other machines.

[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
See [system](@docroot@/store/drv.md#system).

> **Example**
>
Expand Down Expand Up @@ -68,7 +64,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect

- [`builder`]{#attr-builder} ([Path](@docroot@/language/types.md#type-path) | [String](@docroot@/language/types.md#type-string))

Path to an executable that will perform the build.
See [builder](@docroot@/store/drv.md#builder).

> **Example**
>
Expand Down Expand Up @@ -117,7 +113,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect

Default: `[ ]`

Command-line arguments to be passed to the [`builder`](#attr-builder) executable.
See [args](@docroot@/store/drv.md#args).

> **Example**
>
Expand Down Expand Up @@ -182,7 +178,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> }
> ```
>
> The store derivation path will be `/nix/store/<hash>-example.drv`.
> The derivation path will be `/nix/store/<hash>-example.drv`.
> The output paths will be
> - `/nix/store/<hash>-example-lib`
> - `/nix/store/<hash>-example-dev`
Expand Down
7 changes: 4 additions & 3 deletions doc/manual/source/language/import-from-derivation.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,10 @@ Boxes are data structures, arrow labels are transformations.
| evaluate | | |
| | | | |
| V | | |
| .------------. | | .------------------. |
| | derivation |----|-instantiate-|->| store derivation | |
| '------------' | | '------------------' |
| .------------. | | |
| | derivation | | | .------------. |
| | expression |----|-instantiate-|---->| derivation | |
| '------------' | | '------------' |
| | | | |
| | | realise |
| | | | |
Expand Down
Loading
Loading