|
| 1 | +# Bootstrapping the Compiler |
| 2 | + |
| 3 | +This subchapter is about the bootstrapping process. |
| 4 | + |
| 5 | +When running `x.py` you will see output such as: |
| 6 | + |
| 7 | +```txt |
| 8 | +Building stage0 std artifacts |
| 9 | +Copying stage0 std from stage0 |
| 10 | +Building stage0 compiler artifacts |
| 11 | +Copying stage0 rustc from stage0 |
| 12 | +Building LLVM for x86_64-apple-darwin |
| 13 | +Building stage0 codegen artifacts |
| 14 | +Assembling stage1 compiler |
| 15 | +Building stage1 std artifacts |
| 16 | +Copying stage1 std from stage1 |
| 17 | +Building stage1 compiler artifacts |
| 18 | +Copying stage1 rustc from stage1 |
| 19 | +Building stage1 codegen artifacts |
| 20 | +Assembling stage2 compiler |
| 21 | +Uplifting stage1 std |
| 22 | +Copying stage2 std from stage1 |
| 23 | +Generating unstable book md files |
| 24 | +Building stage0 tool unstable-book-gen |
| 25 | +Building stage0 tool rustbook |
| 26 | +Documenting standalone |
| 27 | +Building rustdoc for stage2 |
| 28 | +Documenting book redirect pages |
| 29 | +Documenting stage2 std |
| 30 | +Building rustdoc for stage1 |
| 31 | +Documenting stage2 whitelisted compiler |
| 32 | +Documenting stage2 compiler |
| 33 | +Documenting stage2 rustdoc |
| 34 | +Documenting error index |
| 35 | +Uplifting stage1 rustc |
| 36 | +Copying stage2 rustc from stage1 |
| 37 | +Building stage2 tool error_index_generator |
| 38 | +``` |
| 39 | + |
| 40 | +A deeper look into `x.py`'s phases can be seen here: |
| 41 | + |
| 42 | +<img alt="A diagram of the rustc compilation phases" src="img/rustc_stages.svg" class="center" /> |
| 43 | + |
| 44 | +Keep in mind this diagram is a simplification, i.e. `rustdoc` can be built at |
| 45 | +different stages, the process is a bit different when passing flags such as |
| 46 | +`--keep-stage`, or if there are non-host targets. |
| 47 | + |
| 48 | +The following tables indicate the outputs of various stage actions: |
| 49 | + |
| 50 | +| Stage 0 Action | Output | |
| 51 | +|-----------------------------------------------------------|----------------------------------------------| |
| 52 | +| `beta` extracted | `build/HOST/stage0` | |
| 53 | +| `stage0` builds `bootstrap` | `build/bootstrap` | |
| 54 | +| `stage0` builds `libstd` | `build/HOST/stage0-std/TARGET` | |
| 55 | +| copy `stage0-std` (HOST only) | `build/HOST/stage0-sysroot/lib/rustlib/HOST` | |
| 56 | +| `stage0` builds `rustc` with `stage0-sysroot` | `build/HOST/stage0-rustc/HOST` | |
| 57 | +| copy `stage0-rustc (except executable)` | `build/HOST/stage0-sysroot/lib/rustlib/HOST` | |
| 58 | +| build `llvm` | `build/HOST/llvm` | |
| 59 | +| `stage0` builds `codegen` with `stage0-sysroot` | `build/HOST/stage0-codegen/HOST` | |
| 60 | +| `stage0` builds `rustdoc` with `stage0-sysroot` | `build/HOST/stage0-tools/HOST` | |
| 61 | + |
| 62 | +`--stage=0` stops here. |
| 63 | + |
| 64 | +| Stage 1 Action | Output | |
| 65 | +|-----------------------------------------------------|---------------------------------------| |
| 66 | +| copy (uplift) `stage0-rustc` executable to `stage1` | `build/HOST/stage1/bin` | |
| 67 | +| copy (uplift) `stage0-codegen` to `stage1` | `build/HOST/stage1/lib` | |
| 68 | +| copy (uplift) `stage0-sysroot` to `stage1` | `build/HOST/stage1/lib` | |
| 69 | +| `stage1` builds `libstd` | `build/HOST/stage1-std/TARGET` | |
| 70 | +| copy `stage1-std` (HOST only) | `build/HOST/stage1/lib/rustlib/HOST` | |
| 71 | +| `stage1` builds `rustc` | `build/HOST/stage1-rustc/HOST` | |
| 72 | +| copy `stage1-rustc` (except executable) | `build/HOST/stage1/lib/rustlib/HOST` | |
| 73 | +| `stage1` builds `codegen` | `build/HOST/stage1-codegen/HOST` | |
| 74 | + |
| 75 | +`--stage=1` stops here. |
| 76 | + |
| 77 | +| Stage 2 Action | Output | |
| 78 | +|-------------------------------------------|-----------------------------------------------------------------| |
| 79 | +| copy (uplift) `stage1-rustc` executable | `build/HOST/stage2/bin` | |
| 80 | +| copy (uplift) `stage1-sysroot` | `build/HOST/stage2/lib and build/HOST/stage2/lib/rustlib/HOST` | |
| 81 | +| `stage2` builds `libstd` (except HOST?) | `build/HOST/stage2-std/TARGET` | |
| 82 | +| copy `stage2-std` (not HOST targets) | `build/HOST/stage2/lib/rustlib/TARGET` | |
| 83 | +| `stage2` builds `rustdoc` | `build/HOST/stage2-tools/HOST` | |
| 84 | +| copy `rustdoc` | `build/HOST/stage2/bin` | |
| 85 | + |
| 86 | +`--stage=2` stops here. |
| 87 | + |
| 88 | +Note that the convention `x.py` uses is that: |
| 89 | +- A "stage N artifact" is an artifact that is _produced_ by the stage N compiler. |
| 90 | +- The "stage (N+1) compiler" is assembled from "stage N artifacts". |
| 91 | +- A `--stage N` flag means build _with_ stage N. |
| 92 | + |
| 93 | +In short, _stage 0 uses the stage0 compiler to create stage0 artifacts which |
| 94 | +will later be uplifted to stage1_. |
| 95 | + |
| 96 | +Every time any of the main artifacts (`std` and `rustc`) are compiled, two |
| 97 | +steps are performed. |
| 98 | +When `std` is compiled by a stage N compiler, that `std` will be linked to |
| 99 | +programs built by the stage N compiler (including `rustc` built later |
| 100 | +on). It will also be used by the stage (N+1) compiler to link against itself. |
| 101 | +This is somewhat intuitive if one thinks of the stage (N+1) compiler as "just" |
| 102 | +another program we are building with the stage N compiler. In some ways, `rustc` |
| 103 | +(the binary, not the `rustbuild` step) could be thought of as one of the few |
| 104 | +`no_core` binaries out there. |
| 105 | + |
| 106 | +So "stage0 std artifacts" are in fact the output of the downloaded stage0 |
| 107 | +compiler, and are going to be used for anything built by the stage0 compiler: |
| 108 | +e.g. `rustc` artifacts. When it announces that it is "building stage1 |
| 109 | +std artifacts" it has moved on to the next bootstrapping phase. This pattern |
| 110 | +continues in latter stages. |
| 111 | + |
| 112 | +Also note that building host `std` and target `std` are different based on the |
| 113 | +stage (e.g. see in the table how stage2 only builds non-host `std` targets. |
| 114 | +This is because during stage2, the host `std` is uplifted from the "stage 1" |
| 115 | +`std` -- specifically, when "Building stage 1 artifacts" is announced, it is |
| 116 | +later copied into stage2 as well (both the compiler's `libdir` and the |
| 117 | +`sysroot`). |
| 118 | + |
| 119 | +This `std` is pretty much necessary for any useful work with the compiler. |
| 120 | +Specifically, it's used as the `std` for programs compiled by the newly compiled |
| 121 | +compiler (so when you compile `fn main() { }` it is linked to the last `std` |
| 122 | +compiled with `x.py build --stage 1 src/libstd`). |
| 123 | + |
| 124 | +The `rustc` generated by the stage0 compiler is linked to the freshly-built |
| 125 | +`libstd`, which means that for the most part only `std` needs to be cfg-gated, |
| 126 | +so that `rustc` can use featured added to std immediately after their addition, |
| 127 | +without need for them to get into the downloaded beta. The `libstd` built by the |
| 128 | +`stage1/bin/rustc` compiler, also known as "stage1 std artifacts", is not |
| 129 | +necessarily ABI-compatible with that compiler. |
| 130 | +That is, the `rustc` binary most likely could not use this `std` itself. |
| 131 | +It is however ABI-compatible with any programs that the `stage1/bin/rustc` |
| 132 | +binary builds (including itself), so in that sense they're paired. |
| 133 | + |
| 134 | +This is also where `--keep-stage 1 src/libstd` comes into play. Since most |
| 135 | +changes to the compiler don't actually change the ABI, once you've produced a |
| 136 | +`libstd` in stage 1, you can probably just reuse it with a different compiler. |
| 137 | +If the ABI hasn't changed, you're good to go, no need to spend the time |
| 138 | +recompiling that `std`. |
| 139 | +`--keep-stage` simply assumes the previous compile is fine and copies those |
| 140 | +artifacts into the appropriate place, skipping the cargo invocation. |
| 141 | + |
| 142 | +The reason we first build `std`, then `rustc`, is largely just |
| 143 | +because we want to minimize `cfg(stage0)` in the code for `rustc`. |
| 144 | +Currently `rustc` is always linked against a "new" `std` so it doesn't |
| 145 | +ever need to be concerned with differences in std; it can assume that the std is |
| 146 | +as fresh as possible. |
| 147 | + |
| 148 | +The reason we need to build it twice is because of ABI compatibility. |
| 149 | +The beta compiler has it's own ABI, and then the `stage1/bin/rustc` compiler |
| 150 | +will produce programs/libraries with the new ABI. |
| 151 | +We used to build three times, but because we assume that the ABI is constant |
| 152 | +within a codebase, we presume that the libraries produced by the "stage2" |
| 153 | +compiler (produced by the `stage1/bin/rustc` compiler) is ABI-compatible with |
| 154 | +the `stage1/bin/rustc` compiler's produced libraries. |
| 155 | +What this means is that we can skip that final compilation -- and simply use the |
| 156 | +same libraries as the `stage2/bin/rustc` compiler uses itself for programs it |
| 157 | +links against. |
| 158 | + |
| 159 | +This `stage2/bin/rustc` compiler is shipped to end-users, along with the |
| 160 | +`stage 1 {std,rustc}` artifacts. |
| 161 | + |
0 commit comments