Feature/Stability staging is flaky

[nagisa]

NB: This refers to the feature of making #[unstable] APIs unusable in release channels.

Currently, to allow bootstrap compilers to build standard library, the CFG_BOOTSTRAP_KEY environment variable (see #20663) should match some secret value decided at compile time of rustc. This has several problems:

1. The key is very easy to find out (looking at build logs, running strings etc);
2. Key generation is not cross platform. On BSDs and OS Xs the generated key is well known and currently is equal to N;
3. It broke snapshots.

The proposed solution is to build two different versions of rustc for stage2/3 instead, depending on the situation:

1. stage2/3 src/driver/driver.rs which can process unstable libraries (e.g. using `--cfg enable_unstable);
2. stage2/3 libraries are built;
3. stage2/3 src/driver/driver.rs which cannot process unstable libraries;
4. Proper binary is packaged/shipped depending on the channel.

This has benefits of not having to do any environment variable dance during the build. It is also not really workaround-able without manually rebuilding rustc, unlike the current approach.

(also see logs)

cc @brson, @eddyb

[eddyb]

You don't actually need to build librustc_driver twice, you can have two rustc binaries built with the same libs, but different configs for src/driver/driver.rs.
I would suggest having the default set a TLS key before calling into librustc_driver, while the release version wouldn't do that extra step. The build system could then package bin/rustc-stable (or whatever name it would have) as bin/rustc for the release.

[nagisa]

This approach is made harder by the fact rustdoc test driver depends on get_unstable_features_settings(). This means that to use the proposed approach not only two versions of rustc, but also two versions of rustdoc have to be built.

(alternatively two versions of librustc_driver can be built)

[eddyb]

Building two versions of both rustc and rustdoc shouldn't be a problem - in fact, it probably makes some things simpler by using the same release packaging logic for both binaries.

[brson]

The issue of whether it can be worked around does not really matter to me - if people really want to cheat then it's in their hands.

It seems more complicated to me to teach the build system to build a second driver than to do what we're presently doing.

Can you tell me how it broke snapshots?

[eddyb]

@brson Can you really claim the release can't compile unstable code when I have a grep/sed one-liner to do it on linux (without patching the binary) and on OSX you can turn off the unstable warnings in the alpha with RUSTC_BOOTSTRAP_KEY=N.
I wouldn't care much for any of this (@bstrie might point out the problem of negative reactions to what is essentially a "security by obscurity" scheme with all its typical issues and no upsides), if it didn't break snapshots.
Which happened because the keys on the snapshot builder are unique to each build, amd without them, the snapshot behaves like a nightly, warning on unstable (which turns into errors in some cases).
Oh, and getting the build system to build two versions of the same binary is easier than the key stuff: foreach stable, 0 1 in the last part of mk/target.mk, and then have TARGET_TOOL change the file name and add a --cfg flag when the stable argument is 1. @nagisa is already working on this, and seems to have everything else ready in the latest commit mentioned on this issue.
What's a bit more tricky is getting make install and the release process to pick the right versions of the files.

[brson]

@eddyb I'm sorry you are feeling so hurt about this. I will fix the snapshot situation.

Don't confuse this with a security issue. There is no security involved here. The bootstrap key is just a token effort do discourage people from breaking the feature gates - it was a fun lark. We do not need to implement some kind of DRM scheme in the rust compiler.

[alexcrichton]

This is essentially obsoleted by #32731, so closing