Create tar balls of save-analysis-api metadata for the standard libra… #38102

nrc · 2016-12-01T04:57:59Z

…ries as part of make dist.

alexcrichton · 2016-12-01T19:32:40Z

configure

@@ -731,6 +731,8 @@ case "$CFG_RELEASE_CHANNEL" in
    nightly )
 	msg "overriding settings for $CFG_RELEASE_CHANNEL"
 	CFG_ENABLE_LLVM_ASSERTIONS=1
+    CFG_STAGE2_SAVE_ANALYSIS=true


Looks like tabs/spaces leaked in here by accident...

alexcrichton · 2016-12-01T19:33:26Z

src/bootstrap/dist.rs

+pub fn analysis(build: &Build, compiler: &Compiler, target: &str) {
+    println!("Dist analysis");
+
+    if env::var("CFG_STAGE2_SAVE_ANALYSIS") != Ok("true".to_string()) {


Have you confirmed this works? I don't think these environment variables are set by the configure script for the invocation of rustbuild. (this'd want to get parsed in src/bootstrap/config.rs and then you can check it here via build.config.xxx

You may also want to add a check here for:

if compiler.stage != 2 { return }

alexcrichton · 2016-12-01T19:35:05Z

cc @brson, first time I think we've added new distribution packages?

nrc · 2016-12-02T04:25:01Z

updated...

alexcrichton · 2016-12-02T17:39:10Z

src/bootstrap/dist.rs

+    let image_src = src.join("save-analysis");
+    let dst = image.join("lib/rustlib").join(target).join("save-analysis");
+    t!(fs::create_dir_all(&dst));
+    cp_r(&image_src, &dst);


In theory we want the rust-analysis package to just have the csv files for save-analysis, right? Isn't this copying in the whole set of target libraries?

No, on line 304 we append save-analysisto the path, which means on this line we only copy the contents of that directory, which is the json files (btw, we stopped using csv a while back).

brson

Since this doesn't include changes to the makefiles I take it the plan is to wait out their removal and the conversion of the release process to use only rustbuild.

brson · 2016-12-06T18:55:59Z

src/bootstrap/bin/rustc.rs

+        // Emit save-analysis info.
+        if env::var("RUSTC_SAVE_ANALYSIS") == Ok("api".to_string()) {
+            cmd.arg("-Zsave-analysis-api");
+        }


Why is this conditionalized on an env var?

Huh, this env var is also set by bootstrap. What all does this env var do?

Oh nvm I see what's going on. bootstrap is using this to communicate with the shim.

brson · 2016-12-06T18:57:52Z

src/bootstrap/dist.rs

+    if build.config.channel != "nightly" {
+        println!("Skipping dist-analysis - not on nightly channel");
+        return;
+    }


Doing this on nightlies I guess is just a conservative precaution?

brson · 2016-12-06T19:08:51Z

src/bootstrap/dist.rs

+    let src = build.stage_out(compiler, Mode::Libstd).join(target).join("release").join("deps");
+
+    let image_src = src.join("save-analysis");
+    let dst = image.join("lib/rustlib").join(target).join("save-analysis");


I'm not crazy about the name "save-analysis" here. The verb "save" sounds a bit wrong as the name of the folder containing the analysis.

What is the directory structure under "save-analysis"? I would expect that under this directory there are a series of directories named after target triples, since the analysis is per-target. Based on that assumption I would prefer if the analysis were stored under lib/rustlib/$target/analysis since we already have the organizational structure to distinguish targets under the rustlib dir. I'd also slightly prefer to put the analysis files side-by-side with the libraries they contain analysis for, directly under lib/rustlib/$target/lib, but can't articulate why, so maybe it doesn't matter.

nrc · 2016-12-08T22:26:44Z

@brson - changed save-analysis to analysis in the output names, I think this addresses everything we talked about yesterday.

r? @brson or @alexcrichton

brson · 2016-12-09T00:08:03Z

@bors r+

bors · 2016-12-09T00:08:04Z

📌 Commit 9adaff0 has been approved by brson

bors · 2016-12-09T00:23:29Z

⌛ Testing commit 9adaff0 with merge 5e750fa...

alexcrichton · 2016-12-09T01:54:31Z

@bors: retry force clean

bors · 2016-12-09T09:55:17Z

🔒 Merge conflict

bors · 2016-12-09T10:02:03Z

☔ The latest upstream changes (presumably #38256) made this pull request unmergeable. Please resolve the merge conflicts.

…ries as part of `make dist`.

nrc · 2016-12-09T18:40:01Z

@bors: r=brson

bors · 2016-12-09T18:40:02Z

📌 Commit c49ba05 has been approved by brson

bors · 2016-12-09T21:32:19Z

⌛ Testing commit c49ba05 with merge fa45d0b...

@alexcrichton

Create tar balls of save-analysis-api metadata for the standard libra… …ries as part of `make dist`. r? @alexcrichton

bors · 2016-12-10T00:29:04Z

alexcrichton · 2016-12-12T17:05:41Z

@nrc hm it looks like this is one reason nightlies are broken, failing logs end with:

Dist analysis
gen-installer: looking for programs
gen-installer: 
gen-installer: found tar
gen-installer: found cp
gen-installer: found rm
gen-installer: found mkdir
gen-installer: found echo
gen-installer: found tr
gen-installer: found awk
gen-installer: 
gen-installer: processing arguments
gen-installer: 
gen-installer: CFG_PRODUCT_NAME     := Rust 
gen-installer: CFG_COMPONENT_NAME   := rust-analysis-i686-pc-windows-msvc 
gen-installer: CFG_PACKAGE_NAME     := rust-analysis-nightly-i686-pc-windo ...
gen-installer: CFG_REL_MANIFEST_DIR := rustlib 
gen-installer: CFG_SUCCESS_MESSAGE  := save-analysis-saved. 
gen-installer: CFG_LEGACY_MANIFEST_DIRS := rustlib,cargo 
gen-installer: CFG_NON_INSTALLED_OVERLAY :=  
gen-installer: CFG_BULK_DIRS        :=  
gen-installer: CFG_IMAGE_DIR        := /C/bot/slave/nightly-dist-rustc-win ...
gen-installer: CFG_WORK_DIR         := /C/bot/slave/nightly-dist-rustc-win ...
gen-installer: CFG_OUTPUT_DIR       := /C/bot/slave/nightly-dist-rustc-win ...
gen-installer: 
gen-installer: validating arguments
gen-installer: 
gen-install-script: looking for install programs
gen-install-script: 
gen-install-script: found sed
gen-install-script: found chmod
gen-install-script: found cat
gen-install-script: 
gen-install-script: processing arguments
gen-install-script: 
gen-install-script: CFG_PRODUCT_NAME     := Rust 
gen-install-script: CFG_REL_MANIFEST_DIR := rustlib 
gen-install-script: CFG_SUCCESS_MESSAGE  := save-analysis-saved. 
gen-install-script: CFG_OUTPUT_SCRIPT    := /C/bot/slave/nightly-dist-rustc-win ...
gen-install-script: CFG_LEGACY_MANIFEST_DIRS := rustlib,cargo 
gen-install-script: 
gen-install-script: validating arguments
gen-install-script: 
tar: analysis: Cannot stat: No such file or directory


command did not execute successfully: "tar" "-czf" "/C/bot/slave/nightly-dist-rustc-win-msvc-32-cross/build/obj/build/dist/rust-analysis-nightly.tar.gz" "analysis"
expected success, got: exit code: 2

I'm currently investigating, but out of curiosity did this succeed locally for you?

nrc · 2016-12-13T06:21:12Z

@alexcrichton it did, but I changed a directory name and I think it only worked then because an artefact was left around from a previous run. I have, I think a fix, but waiting on a full build run...

rust-highfive assigned alexcrichton Dec 1, 2016

alexcrichton reviewed Dec 1, 2016

View reviewed changes

nrc force-pushed the save-bots branch from c2e3b19 to 907133f Compare December 2, 2016 04:24

nrc force-pushed the save-bots branch from 907133f to d103e95 Compare December 2, 2016 06:22

alexcrichton reviewed Dec 2, 2016

View reviewed changes

brson reviewed Dec 6, 2016

View reviewed changes

nrc force-pushed the save-bots branch from d103e95 to 9adaff0 Compare December 8, 2016 22:25

rust-highfive assigned brson and unassigned alexcrichton Dec 8, 2016

Create tar balls of save-analysis-api metadata for the standard libra…

c49ba05

…ries as part of `make dist`.

nrc force-pushed the save-bots branch from 9adaff0 to c49ba05 Compare December 9, 2016 18:38

bors added a commit that referenced this pull request Dec 9, 2016

Auto merge of #38102 - nrc:save-bots, r=brson

fa45d0b

Create tar balls of save-analysis-api metadata for the standard libra… …ries as part of `make dist`. r? @alexcrichton

bors merged commit c49ba05 into rust-lang:master Dec 10, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create tar balls of save-analysis-api metadata for the standard libra… #38102

Create tar balls of save-analysis-api metadata for the standard libra… #38102

nrc commented Dec 1, 2016

alexcrichton Dec 1, 2016

alexcrichton Dec 1, 2016

alexcrichton Dec 1, 2016

alexcrichton commented Dec 1, 2016

nrc commented Dec 2, 2016

alexcrichton Dec 2, 2016

nrc Dec 2, 2016

brson left a comment

brson Dec 6, 2016

brson Dec 6, 2016

brson Dec 6, 2016

brson Dec 6, 2016

nrc Dec 6, 2016

brson Dec 6, 2016

nrc commented Dec 8, 2016

brson commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

alexcrichton commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

nrc commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 10, 2016

alexcrichton commented Dec 12, 2016

nrc commented Dec 13, 2016

Create tar balls of save-analysis-api metadata for the standard libra… #38102

Create tar balls of save-analysis-api metadata for the standard libra… #38102

Conversation

nrc commented Dec 1, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Dec 1, 2016

nrc commented Dec 2, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nrc commented Dec 8, 2016

brson commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

alexcrichton commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

nrc commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 9, 2016

bors commented Dec 10, 2016

alexcrichton commented Dec 12, 2016

nrc commented Dec 13, 2016