WIP: Migrate some test jobs to GitHub actions. Refs #192 #201

atodorov · 2020-04-20T13:17:02Z

Linux test jobs look fine except the fact that GitHub doesn't seem to support yaml anchors.

edit: Windows jobs PASS with the last commit, but I'm not entirely confident the specified Python version is used. I see references to the Python version installed with Visual Studio.

MacOS test job fails with ./.travis/lib-setup.sh: line 72: pyenv: command not found.

For the list of installed software check here:
https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md
https://github.com/actions/virtual-environments/blob/master/images/macos/macos-10.15-Readme.md

atodorov · 2020-04-20T13:18:23Z

Example of the latest test execution: atodorov#1

atodorov · 2020-04-27T10:50:05Z

@frozencemetery, @aiudirog - can you check this out and comment on it? I'd like to move this into a state where I can enable wheel building for Linux.

frozencemetery

Sure, I was going to wait until you had marked it not WIP.

A couple nits inline. In general, I think our goal here should be to retire Travis integration as much as possible. This means that CI scripts should probably not be in a directory with "travis" in the name at the end.

It also means that everything should move - this includes the deployments. The hard part, as it were :) I recognize this isn't really going to be doable without the credentials, but if you can get it to a point where you think things will likely work, I can take it from there. Improvements exist on the status quo in this area - from not using hacky scripts for deployment to only building once per platform - but none of that's required.

Finally, I'm probably going to squash this into a single commit, if that makes your development any easier. (Then we can handle Linux wheels as a separate change.)

frozencemetery · 2020-04-27T20:40:30Z

.github/workflows/run-on-linux.sh

@@ -0,0 +1,6 @@
+#!/bin/bash -ex
+
+sudo sed -i '1i 127.0.0.1 test.box' /etc/hosts;


I know these were in the original, but we can lose the semicolons here.

.github/workflows/run-on-linux.sh

frozencemetery · 2020-04-27T20:52:36Z

.travis/lib-setup.sh

@@ -78,7 +78,7 @@ setup::macos::install() {

 setup::windows::install() {
    # Install the right Python version and MIT Kerberos
-    choco install python"${PYENV:0:1}" --version $PYENV
+    choco install python"${PYENV:0:1}" --version $PYENV || true


Question on this - do we know what's actually gone wrong here? I see the error logs, and it seems Windows passess...

From the log linked in the commit log:

Progress: Downloading python3 3.8.1... 100% python3 v3.8.1 [Approved] python3 package files install completed. Performing other installation steps. Installing 64-bit python3... WARNING: Generic MSI Error. This is a local environment error, not an issue with a package or the MSI itself - it could mean a pending reboot is necessary prior to install or something else (like the same version is already installed). Please see MSI log if available. If not, try again adding '--install-arguments="'/l*v c:\python3_msi_install.log'"'. Then search the MSI Log for "Return Value 3" and look above that for the error. ERROR: Running ["C:\ProgramData\chocolatey\lib\python3\tools\python-3.8.1-amd64.exe" /quiet InstallAllUsers=1 PrependPath=1 TargetDir="C:\Python38" ] was not successful. Exit code was '1603'. Exit code indicates the following: Generic MSI Error. This is a local environment error, not an issue with a package or the MSI itself - it could mean a pending reboot is necessary prior to install or something else (like the same version is already installed). Please see MSI log if available. If not, try again adding '--install-arguments="'/l*v c:\python3_msi_install.log'"'. Then search the MSI Log for "Return Value 3" and look above that for the error.. The install of python3 was NOT successful. Error while running 'C:\ProgramData\chocolatey\lib\python3\tools\chocolateyInstall.ps1'. See log for details. Chocolatey installed 0/1 packages. 1 packages failed. See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

More that I could find on this particular error code:
https://support.microsoft.com/en-us/help/834484/you-receive-an-error-1603-a-fatal-error-occurred-during-installation

I have seen something similar when testing on Windows (AppVeyor, GitHub CI) but installing msi files directly, not via Chocolatey. The ones I've seen have always been an installer which wants to reboot the system. IIRC Python's installer wants to reboot Windows so that changes to %PATH% are taken into account (or something along those lines).

The exit code from choco here is not 0 so I'm ignoring it, otherwise we can't proceed.

- rename .travis/ -> .ci/ - use a small script for running the test job b/c GitHub's parser doesn't support yaml anchors, see: https://github.community/t5/GitHub-Actions/Support-for-YAML-anchors/m-p/30336 - do not fail on Python installs in Windows environment, for example: https://github.com/atodorov/python-gssapi/pull/1/checks?check_run_id=601883334. Likely caused by the Python installer wanting to reboot Windows

atodorov · 2020-05-01T21:22:24Z

@frozencemetery I have hit a snag here and need an advice.

Travis CI supports stages and the jobs in a certain stage will execute only after the ones before have all passed. This ensures that docs and/or artifacts are deployed only if the tests before them pass. Stages are keyed off based on branch name or tags.

GitHub actions doesn't support stages but does support job dependencies where you've got to list all of the previous jobs the current one depends on, see https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idneeds. However this appears to be valid if all related jobs are defined in the same file, so testing.yml in this case.

GitHub does support specifying when a specific workflow should be executed, see https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#onpushpull_requestbranchestags and the current specification in both yml files in this PR.

This is on a per workflow level though, not per job. I can use conditionals for every job but combined with the fact that each deployment job must specify all of the test jobs as dependencies makes all of this very fragile.

There is also another trigger for workflows which is based on specific events, see:
https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#onevent_nametypes and https://help.github.com/en/actions/reference/events-that-trigger-workflows

From these 2 I can point out:

on.release.published - this is when you create a new release via the GitHub UI; releases can be created from tags but are not the same
on.create - when branch or tag is created. There are no examples in the docs but the API references shows ref_type branch or tag which I'm hoping to be valid for the types keyword in the workflow, see https://help.github.com/en/actions/reference/events-that-trigger-workflows#create-event-create and https://developer.github.com/v3/activity/events/types/#createevent.

I'll make an independent experiment with the last one b/c it sounds most promising.

However the question still remains - do you want to complicate things and only deploy when tests are passing or potentially deploy bogus docs/artifacts in case somebody tags a bad version (much more easier to model with the new syntax) ?

frozencemetery · 2020-05-04T20:19:28Z

Hmm. I'll allocate some time to play with it this week and see if I can figure out a way forward. NEEDINFO(me) :)

frozencemetery · 2020-05-11T18:34:37Z

(Superseded by #204)

atodorov force-pushed the github_actions branch 2 times, most recently from 56931cd to c751b51 Compare April 20, 2020 13:25

frozencemetery reviewed Apr 27, 2020

View reviewed changes

atodorov force-pushed the github_actions branch from 15b552a to 655ffb9 Compare May 1, 2020 20:29

atodorov force-pushed the github_actions branch from 655ffb9 to dfb8402 Compare May 1, 2020 21:08

frozencemetery mentioned this pull request May 8, 2020

Migrate CI and automation to GitHub actions #204

Merged

frozencemetery closed this May 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Migrate some test jobs to GitHub actions. Refs #192 #201

WIP: Migrate some test jobs to GitHub actions. Refs #192 #201

Uh oh!

atodorov commented Apr 20, 2020 •

edited

Loading

Uh oh!

atodorov commented Apr 20, 2020 •

edited

Loading

Uh oh!

atodorov commented Apr 27, 2020

Uh oh!

frozencemetery left a comment

Uh oh!

frozencemetery Apr 27, 2020

Uh oh!

Uh oh!

frozencemetery Apr 27, 2020

Uh oh!

atodorov May 1, 2020

Uh oh!

atodorov commented May 1, 2020

Uh oh!

frozencemetery commented May 4, 2020

Uh oh!

frozencemetery commented May 11, 2020

Uh oh!

Uh oh!

		@@ -0,0 +1,6 @@
		#!/bin/bash -ex

		sudo sed -i '1i 127.0.0.1 test.box' /etc/hosts;

WIP: Migrate some test jobs to GitHub actions. Refs #192 #201

WIP: Migrate some test jobs to GitHub actions. Refs #192 #201

Uh oh!

Conversation

atodorov commented Apr 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atodorov commented Apr 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atodorov commented Apr 27, 2020

Uh oh!

frozencemetery left a comment

Choose a reason for hiding this comment

Uh oh!

frozencemetery Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

frozencemetery Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

atodorov May 1, 2020

Choose a reason for hiding this comment

Uh oh!

atodorov commented May 1, 2020

Uh oh!

frozencemetery commented May 4, 2020

Uh oh!

frozencemetery commented May 11, 2020

Uh oh!

Uh oh!

atodorov commented Apr 20, 2020 •

edited

Loading

atodorov commented Apr 20, 2020 •

edited

Loading