Skip to content

Reuse build setup using a dedicated github action #2563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jan 5, 2022
Merged

Reuse build setup using a dedicated github action #2563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ce57b1d
Extract out ci build setup
jneira Jan 4, 2022
67f9b92
Correct action path
jneira Jan 4, 2022
2c88508
Add required shell property
jneira Jan 4, 2022
3bbc8b7
Remove unused cabal version
jneira Jan 4, 2022
378c1ce
Update .github/actions/setup-build/action.yml
jneira Jan 5, 2022
c0e1903
Update .github/actions/setup-build/action.yml
jneira Jan 5, 2022
b422d5f
Update .github/actions/setup-build/action.yml
jneira Jan 5, 2022
8cd5e7f
Update .github/actions/setup-build/action.yml
jneira Jan 5, 2022
186d93d
Copy alt project file unconditionally
jneira Jan 5, 2022
2719110
Make freeze strict
jneira Jan 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions .github/actions/setup-build/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
name: "Cached build"
description: "Setup the build using cache"
inputs:
ghc:
description: "Ghc version"
required: true
cabal:
description: "Cabal version"
required: false
default: "3.6"
os:
description: "Operating system: Linux, Windows or macOS"
required: true
runs:
using: "composite"
steps:
- uses: haskell/actions/setup@v1
id: HaskEnvSetup
with:
ghc-version : ${{ inputs.ghc }}
cabal-version: ${{ inputs.cabal }}
enable-stack: false

- if: inputs.os == 'Windows'
name: (Windows) Platform config
run: |
echo "CABAL_PKGS_DIR=C:\\cabal\\packages" >> $GITHUB_ENV
shell: bash
- if: ( inputs.os == 'Linux' ) || ( inputs.os == 'macOS' )
name: (Linux,macOS) Platform config
run: |
echo "CABAL_PKGS_DIR=~/.cabal/packages" >> $GITHUB_ENV
shell: bash

# This copy an alternative cabal-ghc${GHCVER}.project (for example cabal-ghc921.project)
# as main cabal-project, for not fully supported ghc versions
# Needs to be before the caching step so that the cache can detect changes to the modified cabal.project file
- name: Use possible modified `cabal.project`
env:
GHCVER: ${{ inputs.ghc }}
run: |
# File has some protections preventing regular `rm`.
# (most probably sticky bit is set on $HOME)
# `&&` insures `rm -f` return is positive.
# Many platforms aslo have `alias cp='cp -i'`.
ALT_PROJECT_FILE=cabal-ghc${GHCVER//./}.project
if [[ -f "$ALT_PROJECT_FILE" ]]; then
rm -f -v cabal.project && cp -v "$ALT_PROJECT_FILE" cabal.project
fi
shell: bash

- if: inputs.os == 'Windows' && inputs.ghc == '8.8.4'
name: (Windows,GHC 8.8) Modify `cabal.project` to workaround segfaults
run: |
echo "package floskell" >> cabal.project
echo " ghc-options: -O0" >> cabal.project
shell: bash

# Shorten binary names as a workaround for filepath length limits in Windows,
# but since tests are hardcoded on this workaround -
# all platforms (in 2021-12-07) need it.
# All workflows which distinquishes cache on `cabal.project` needs this.
- name: Workaround shorten binary names
run: |
sed -i.bak -e 's/haskell-language-server/hls/g' \
-e 's/haskell_language_server/hls/g' \
haskell-language-server.cabal cabal.project
sed -i.bak -e 's/Paths_haskell_language_server/Paths_hls/g' \
src/**/*.hs exe/*.hs
shell: bash

- name: Retrieving `cabal.project` Hackage timestamp
run: |
# Form: index-state: 2021-11-29T08:11:08Z
INDEX_STATE_ENTRY=$(grep index-state cabal.project)
# Form: 2021-11-29T08-11-08Z
INDEX_STATE1=$(echo "$INDEX_STATE_ENTRY" | cut -d' ' -f2 | tr ':' '-')
echo "INDEX_STATE=$INDEX_STATE1" >> $GITHUB_ENV
shell: bash

# We have to restore package sources before `cabal update`
# because it overwrites the hackage index with the cached one
- name: Hackage sources cache
uses: actions/cache@v2
env:
cache-name: hackage-sources
with:
path: ${{ env.CABAL_PKGS_DIR }}
key: ${{ env.cache-name }}-${{ env.INDEX_STATE }}
restore-keys: ${{ env.cache-name }}-

# To ensure we get the latest hackage index without relying on the haskell action logic
# It has to be done before `cabal freeze` to make it aware of the new index
- run: cabal update
shell: bash

- name: Form the package list ('cabal.project.freeze')
run: |
cabal v2-freeze && \
echo "" && \
echo 'Output:' && \
echo "" && \
cat 'cabal.project.freeze'
shell: bash

- name: Compiled deps cache
id: compiled-deps
uses: actions/cache@v2
env:
cache-name: compiled-deps
with:
path: ${{ steps.HaskEnvSetup.outputs.cabal-store }}
key: ${{ env.cache-name }}-${{ inputs.os }}-${{ inputs.ghc }}-${{ env.INDEX_STATE }}-${{ hashFiles('cabal.project.freeze') }}
restore-keys: |
${{ env.cache-name }}-${{ inputs.os }}-${{ inputs.ghc }}-${{ env.INDEX_STATE }}-
${{ env.cache-name }}-${{ inputs.os }}-${{ inputs.ghc }}-
${{ env.cache-name }}-${{ inputs.os }}-

# We remove the freeze file because it could interfere with the build
- name: "Remove freeze file"
run: rm -f cabal.project.freeze
shell: bash
68 changes: 3 additions & 65 deletions .github/workflows/bench.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ jobs:
matrix:
ghc: ['8.10.7']
os: [ubuntu-latest]
cabal: ['3.6']

# This code is fitted to the strategy: assumes Linux is used ... etc,
# change of the strategy may require changing the bootstrapping/run code
Expand All @@ -57,71 +56,10 @@ jobs:

- run: git fetch origin master # check the master branch for benchmarking

- uses: haskell/actions/setup@v1
id: HaskEnvSetup
with:
ghc-version : ${{ matrix.ghc }}
cabal-version: ${{ matrix.cabal }}
enable-stack: false

- name: Linux Platform config
run: |
echo "CABAL_PKGS_DIR=~/.cabal/packages" >> $GITHUB_ENV

# All workflows which distinquishes cache on `cabal.project` needs this.
- name: Workaround shorten binary names
run: |
sed -i.bak -e 's/haskell-language-server/hls/g' \
-e 's/haskell_language_server/hls/g' \
haskell-language-server.cabal cabal.project
sed -i.bak -e 's/Paths_haskell_language_server/Paths_hls/g' \
src/**/*.hs exe/*.hs

- name: Retrieving `cabal.project` Hackage timestamp
run: |
# Form: index-state: 2021-11-29T08:11:08Z
INDEX_STATE_ENTRY=$(grep index-state cabal.project)
# Form: 2021-11-29T08-11-08Z
INDEX_STATE1=$(echo "$INDEX_STATE_ENTRY" | cut -d' ' -f2 | tr ':' '-')
echo "INDEX_STATE=$INDEX_STATE1" >> $GITHUB_ENV

# We have to restore package sources before `cabal update`
# cause it overwrites the hackage index with the cached one
- name: Hackage sources cache
uses: actions/cache@v2
env:
cache-name: hackage-sources
with:
path: ${{ env.CABAL_PKGS_DIR }}
key: ${{ env.cache-name }}-${{ env.INDEX_STATE }}
restore-keys: ${{ env.cache-name }}-

# To ensure we get the lastest hackage index and not relying on haskell action logic
# It has to be done before `cabal freeze` to make it aware of the new index
- run: cabal update

- name: Form the package list ('cabal.project.freeze')
run: |
cabal v2-freeze && \
echo "" && \
echo 'Output:' && \
echo "" && \
cat 'cabal.project.freeze' && \
echo '' || \
echo 'WARNING: Could not produce the `freeze`.'

- name: Compiled deps cache
id: compiled-deps
uses: actions/cache@v2
env:
cache-name: compiled-deps
- uses: ./.github/actions/setup-build
with:
path: ${{ steps.HaskEnvSetup.outputs.cabal-store }}
key: ${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-${{ env.INDEX_STATE }}-${{ hashFiles('cabal.project.freeze') }}
restore-keys: |
${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-${{ env.INDEX_STATE }}-
${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-
${{ env.cache-name }}-${{ runner.os }}-
ghc: ${{ matrix.ghc }}
os: ${{ runner.os }}

# max-backjumps is increased as a temporary solution
# for dependency resolution failure
Expand Down
96 changes: 3 additions & 93 deletions .github/workflows/caching.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,104 +86,14 @@ jobs:
, "macOS-latest"
, "windows-latest"
]
cabal: ['3.6']

steps:
- uses: actions/checkout@v2

- uses: haskell/actions/setup@v1
id: HaskEnvSetup
- uses: ./.github/actions/setup-build
with:
ghc-version : ${{ matrix.ghc }}
cabal-version: ${{ matrix.cabal }}
enable-stack: false

- if: runner.os == 'Windows'
name: (Windows) Platform config
run: |
echo "CABAL_PKGS_DIR=C:\\cabal\\packages" >> $GITHUB_ENV
- if: ( runner.os == 'Linux' ) || ( runner.os == 'macOS' )
name: (Linux,macOS) Platform config
run: |
echo "CABAL_PKGS_DIR=~/.cabal/packages" >> $GITHUB_ENV

# Needs to be before Cache Cabal so the cache can detect changes to the modified cabal.project file
- if: matrix.ghc == '9.0.1'
name: (GHC 9.0.1) Use modified `cabal.project`
run: |
# File has some protections preventing regular `rm`.
# (most probably sticky bit is set on $HOME)
# `&&` insures `rm -f` return is positive.
# Many platforms also have `alias cp='cp -i'`.
rm -f -v cabal.project && cp -v cabal-ghc901.project cabal.project
- if: runner.os == 'Windows' && matrix.ghc == '8.8.4'
name: (Windows,GHC 8.8) Modify `cabal.project` to workaround segfaults
run: |
echo "package floskell" >> cabal.project
echo " ghc-options: -O0" >> cabal.project

# Shorten binary names as a workaround for filepath length limits in Windows,
# but since tests are hardcoded on this workaround -
# all platforms (in 2021-12-07) need it.
# All workflows which distinquishes cache on `cabal.project` needs this.
- name: Workaround shorten binary names
run: |
sed -i.bak -e 's/haskell-language-server/hls/g' \
-e 's/haskell_language_server/hls/g' \
haskell-language-server.cabal cabal.project
sed -i.bak -e 's/Paths_haskell_language_server/Paths_hls/g' \
src/**/*.hs exe/*.hs

- name: Retrieving `cabal.project` Hackage timestamp
run: |
# Form: index-state: 2021-11-29T08:11:08Z
INDEX_STATE_ENTRY=$(grep index-state cabal.project)
# Form: 2021-11-29T08-11-08Z
INDEX_STATE1=$(echo "$INDEX_STATE_ENTRY" | cut -d' ' -f2 | tr ':' '-')
echo "INDEX_STATE=$INDEX_STATE1" >> $GITHUB_ENV

# 2021-12-02: NOTE: Cabal Hackage source tree storage does not depend on OS or GHC really,
# but can depend on `base`.
# But this caching is happens only inside `master` for `master` purposes of compiling the deps
# so having a shared pool here that depends only on Hackage pin & does not depend on `base` is "good enough"
# & used such because it preserves 10% of a global cache storage pool.
# We have to restore package sources before `cabal update`
# cause it overwrites the hackage index with the cached one
- name: Hackage sources cache
uses: actions/cache@v2
env:
cache-name: hackage-sources
with:
path: ${{ env.CABAL_PKGS_DIR }}
key: ${{ env.cache-name }}-${{ env.INDEX_STATE }}
restore-keys: ${{ env.cache-name }}-

# To ensure we get the lastest hackage index and not relying on haskell action logic
# It has to be done before `cabal freeze` to make it aware of the new index
- run: cabal update

- name: Form the package list ('cabal.project.freeze')
run: |
cabal v2-freeze && \
echo "" && \
echo 'Output:' && \
echo "" && \
cat 'cabal.project.freeze' && \
echo '' || \
echo 'WARNING: Could not produce the `freeze`.'

- name: Compiled deps cache
id: compiled-deps
uses: actions/cache@v2
env:
cache-name: compiled-deps
with:
path: ${{ steps.HaskEnvSetup.outputs.cabal-store }}
key: ${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-${{ env.INDEX_STATE }}-${{ hashFiles('cabal.project.freeze') }}
restore-keys: |
${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-${{ env.INDEX_STATE }}-
${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-
${{ env.cache-name }}-${{ runner.os }}-
ghc: ${{ matrix.ghc }}
os: ${{ runner.os }}

- if: steps.compiled-deps.outputs.cache-hit != 'true' && runner.os == 'Linux' && matrix.ghc == '8.10.7'
name: Download sources for bench
Expand Down
Loading