Skip to content

⚡Improve Build Performance by Using Invoke-Build Incremental Builds #3619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

⚡Improve Build Performance by Using Invoke-Build Incremental Builds #3619

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 77 additions & 28 deletions vscode-powershell.build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,88 +13,129 @@ $script:IsPreviewExtension = $script:PackageJson.name -like "*preview*" -or $scr
Write-Host "`n### Extension Version: $($script:PackageJson.version) Extension Name: $($script:PackageJson.name)`n" -ForegroundColor Green

function Get-EditorServicesPath {
$psesRepoPath = if ($EditorServicesRepoPath) {
$EditorServicesRepoPath
if ($EditorServicesRepoPath) {
Resolve-Path $EditorServicesRepoPath -ErrorAction Stop
} else {
"$PSScriptRoot/../PowerShellEditorServices/"
Resolve-Path "$PSScriptRoot/../PowerShellEditorServices/" -ErrorAction Stop
}
# NOTE: The ErrorActionPreference for both Invoke-Build and Azure DevOps
# scripts is Stop, but we want to continue and return false here.
return Resolve-Path "$psesRepoPath/PowerShellEditorServices.build.ps1" -ErrorAction Continue
}

task Restore -If { !(Test-Path "$PSScriptRoot/node_modules") } {
function Get-EditorServicesBuildScriptPath {
return Resolve-Path (Join-Path (Get-EditorServicesPath) 'PowerShellEditorServices.build.ps1') -ErrorAction Stop
}
try {
$editorServicesPath = Get-EditorServicesPath
$editorServicesBuildScriptPath = Get-EditorServicesBuildScriptPath
} catch {
throw 'Powershell Editor Services repo was not detected. Specify its path via the -EditorServicesRepoPath parameter or ensure it is available in the directory above {0}' -f $PSScriptRoot
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so this change should no longer be necessary with #3623.

}

$restoreCheckpoint = "$PSScriptRoot/node_modules/restored"
#This will run when node_modules is empty or when package.json is updated
Task Restore -Input $PSScriptRoot/package.json -Output { $RestoreCheckpoint } {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just -Input package.json -Output node_modules?

$ErrorActionPreference = 'stop'
Write-Host "`n### Restoring vscode-powershell dependencies`n" -ForegroundColor Green
# When in a CI build use the --loglevel=error parameter so that
# package install warnings don't cause PowerShell to throw up
if ($env:TF_BUILD) {
exec { & npm ci --loglevel=error }
Exec { & npm ci --loglevel=error }
} else {
exec { & npm install }
Exec { & npm install }
}
New-Item $restoreCheckpoint
}


#region Clean tasks

task Clean {
Task Clean {
Write-Host "`n### Cleaning vscode-powershell`n" -ForegroundColor Green
Remove-Item ./modules -Exclude "README.md" -Recurse -Force -ErrorAction Ignore
Remove-Item ./modules -Exclude 'README.md' -Recurse -Force -ErrorAction Ignore
Remove-Item ./out -Recurse -Force -ErrorAction Ignore
Remove-Item ./node_modules -Recurse -Force -ErrorAction Ignore
}

task CleanEditorServices -If (Get-EditorServicesPath) {
Task CleanEditorServices -If $editorServicesBuildScriptPath {
Write-Host "`n### Cleaning PowerShellEditorServices`n" -ForegroundColor Green
Invoke-Build Clean (Get-EditorServicesPath)
Invoke-Build Clean $editorServicesBuildScriptPath
}

#endregion
#region Build tasks

task BuildEditorServices -If (Get-EditorServicesPath) {
Task BuildEditorServices -Input {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is going to work. This task and build script is specifically setup to allow the build to work when the PSES repo does not exist, but a drop of PSES (like from the signing build) has been placed in the appropriate location.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't something like substituting in a separate PSES be explicit? I could add a -UseExistingPSES switch or something to the script and the appropriate same place on the CI for that scenario

Get-ChildItem (Join-Path $editorServicesPath 'src') |
Get-ChildItem -Exclude 'bin', 'obj' |
Get-ChildItem -File -Recurse |
Where-Object Name -NotMatch 'BuildInfo.cs'
} -Output {
#Some copied files preserve timestamps which breaks the incremental build process as the files are seen as "older" than the source
#We use these files as a freshness indicator since they always gets updated on a build
#TODO: More granular incremental build on PSES tasks
Join-Path $editorServicesPath 'module/PowerShellEditorServices/bin/Core/Microsoft.PowerShell.EditorServices.Hosting.dll'
} -If $editorServicesBuildScriptPath {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this shouldn't be necessary any more.

Write-Host "`n### Building PowerShellEditorServices`n" -ForegroundColor Green
Invoke-Build Build (Get-EditorServicesPath)
Invoke-Build Build $editorServicesBuildScriptPath
}

task CopyEditorServices -If { !(Test-Path ./modules/PowerShellEditorServices) -and (Get-EditorServicesPath) } BuildEditorServices, {
#This will run when the editorservices module is empty or has been updated
$PSESModulePath = (Join-Path $editorServicesPath 'module')
Task CopyEditorServices -Partial -Input {
$SCRIPT:PSESFiles = Get-ChildItem -File -Recurse $PSESModulePath
$PSESFiles
} -Output {
#Get the relative path. Resolve-Path -Relative requires you to be in the directory you want to be relative
$modulesDir = Join-Path $PSScriptRoot 'modules'
[string]$stripModulesDirRegex = '^' + [Regex]::Escape($PSESModulePath)
($SCRIPT:PSESFiles.fullname -replace $stripModulesDirRegex, '').foreach{ Join-Path $modulesDir $PSItem }
} -If {
$editorServicesPath
} BuildEditorServices, {
Write-Host "`n### Copying PowerShellEditorServices module files" -ForegroundColor Green
Copy-Item -Recurse -Force "$(Split-Path (Get-EditorServicesPath))/module/*" ./modules
Copy-Item $PSESModulePath/* -Recurse -Force -Destination (Join-Path $PSScriptRoot 'modules')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

}

task Build CopyEditorServices, Restore, {
Task Build -Input {
Get-ChildItem -File -Recurse (Join-Path $PSScriptRoot 'src')
} -Output {
Join-Path $PSScriptRoot 'out/main.js'
Get-ChildItem -File -Recurse (Join-Path $PSScriptRoot 'out')
} CopyEditorServices, Restore, {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, since esbuild is soooo quick, what if we moved the calls of tsc all to the lint step and did the lint when running the tests instead? Then rebuilding the extension doesn't need to be incremental, and the tests can be the "slower" part. (Note that esbuild does not build the test files, just the extension.)

Write-Host "`n### Building vscode-powershell" -ForegroundColor Green
# TODO: TSLint is deprecated and we need to switch to ESLint.
# https://github.com/PowerShell/vscode-powershell/pull/3331
exec { & npm run lint }
Exec { & npm run lint }

# TODO: When supported we should use `esbuild` for the tests too. Although
# we now use `esbuild` to transpile, bundle, and minify the extension, we
# still use `tsc` to transpile everything in `src` and `test` because the VS
# Code test runner expects individual files (and globs them at runtime).
# Unfortunately `esbuild` doesn't support emitting 1:1 files (yet).
# https://github.com/evanw/esbuild/issues/944
exec { & npm run build }
Exec { & npm run build }
}

#endregion
#region Test tasks

task Test -If (!($env:TF_BUILD -and $global:IsLinux)) Build, {
Task Test -If (!($env:TF_BUILD -and $global:IsLinux)) Build, {
Write-Host "`n### Running extension tests" -ForegroundColor Green
exec { & npm run test }
if ($ENV:TERM_PROGRAM -eq 'vscode') {
Write-Warning 'E2E Tests cannot be performed from the CLI while vscode is running, please close vscode and run again or use the "Launch Extension Tests" launch config'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure they can, that's the whole reason it opens up a copy of Insiders.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe because I was running on insiders, it threw an error for me every time and I couldn't find a way to specific like a "data-only" copy of insiders.

return
}
Exec { & npm run test }
}

task TestEditorServices -If (Get-EditorServicesPath) {
Task TestEditorServices -If $editorServicesBuildPath {
Write-Host "`n### Testing PowerShellEditorServices`n" -ForegroundColor Green
Invoke-Build Test (Get-EditorServicesPath)
Invoke-Build Test $editorServicesBuildPath
}

#endregion

#region Package tasks

task UpdateReadme -If { $script:IsPreviewExtension } {
Task UpdateReadme -If { $script:IsPreviewExtension } {
# Add the preview text
$newReadmeTop = '# PowerShell Language Support for Visual Studio Code

Expand All @@ -104,13 +145,21 @@ task UpdateReadme -If { $script:IsPreviewExtension } {
$readmePath = (Join-Path $PSScriptRoot README.md)

$readmeContent = Get-Content -Path $readmePath
if (!($readmeContent -match "This is the PREVIEW version of the PowerShell extension")) {
if (!($readmeContent -match 'This is the PREVIEW version of the PowerShell extension')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we save changes like this for a different PR?

$readmeContent[0] = $newReadmeTop
$readmeContent | Set-Content $readmePath -Encoding utf8
}
}

task Package UpdateReadme, Build, {
Task Package -Input {
Get-ChildItem $PSScriptRoot -Exclude '.vscode-test', 'logs', 'sessions', 'node_modules', '.git' |
Get-ChildItem -Recurse -File
} -Output {
$vsix = Get-ChildItem $PSScriptRoot/*.vsix
if (-not $VSIX) {
'No VSIX Created Yet'
} else { $vsix }
Comment on lines +155 to +161
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this is necessary. The VSIX is only really created by the release pipeline, and only occasionally by a developer (and doesn't need to be incremental).

} UpdateReadme, Build, {
assert { Test-Path ./modules/PowerShellEditorServices }
Write-Host "`n### Packaging $($script:PackageJson.name)-$($script:PackageJson.version).vsix`n" -ForegroundColor Green
exec { & npm run package }
Expand Down