Go To Definition does not work (Bash IDE in VS Code)

[timoteicampian]

Go To Definition does not work starting with version 1.19.0

[skovhus]

Please include an example script. Unfortunately I can’t reproduce this error.

[timoteicampian]

I am using it in Visual Studio Code: Right Click > "Go to Definition" or CMD + Click
Unfortunately, VSC updates the Bash IDE extension to the latest version after every restart. So I have to downgrade it again to 1.18.0.

[skovhus]

A code example would still be great to know if this is an issue across files or jump to definition in a single file.

Note that we recently https://github.com/bash-lsp/bash-language-server/blob/main/vscode-client/CHANGELOG.md#1210 started using source statements to narrow down the completions. If this is an issue across multiple files, then that could be the culprit.

[timoteicampian]

it does not work even inside a single file

below a sample of code to check/test

`#!/usr/bin/env bash
VAR1=a
VAR2=b

func1() {
echo $VAR1
}
func2() {
echo $VAR2
}
main() {
func1
func2
}`

[skovhus]

Thanks for the example here. I couldn't reproduce any issues with that example using the latest extension. So I'm curious what is going on. Can I get you to upgrade to the latest version and paste the output from the extension? You can find the log output when opening a bash file and click "View -> Output -> Bash IDE". Try clicking jump to definition and paste the entire log output here.

Thanks 🙏

[timoteicampian]

sure...sorry for the late answer...many thx for your effort

[timoteicampian]

onDefinition 77:7 word="config_openssh"
onHover 77:6 word="config_openssh"
onHover 77:5 word="config_openssh"
onDefinition 77:15 word=null
onHover 77:11 word="config_openssh"

[timoteicampian]

onDocumentHighlight 79:7 word="config_avahi"
onDefinition 79:13 word=null
onHover 79:14 word="||"
onHover 79:7 word="config_avahi"
onReferences 79:13 word=null
onHover 81:18 word="get_status"

[timoteicampian]

for the small sample text file was working....
I find out that Go to Definition works only after scrolling down all the pages and reading all the functions from the single file

[timoteicampian]

I started "ShellCheck: Run Linting" and it seems that Go to Reference is working again (using latest version 1.23.0)!

[skovhus]

ShellCheck: Run Linting

Hm. That isn’t a Bash IDE feature/option.

But you see jump to definition working as expected now with the latest extension?

[timoteicampian]

Yes, working fine after executing "ShellCheck: Run Linting" - thank you

[timoteicampian]

Could you please reopen this case as "Go to Definition" only works up to 1.16.0 for projects opened locally or on a remote (ssh) server (for me and my college)
I have tried it a few times on different projects...

[skovhus]

@timoteicampian could you provide a reproducible example that always fails for you? The example above consistently works.

Does jump to definition start working after you modify the file?

[timoteicampian]

Dear Kenneth
I was able to reproduce something using this two sample files

ls -1
func.sh
init.0

cat func.sh

#!/usr/bin/env bash

VAR_A1=true
VAR_A2=true
a1() {
    :
}
a2() {
    :
}
a3() {
    :
}
a4() {
    :
}
a5() {
    :
}
a6() {
    :
}

cat init.0

#!/usr/bin/env bash

VAR_B1=true
b1() {
    :
}
b2() {
    :
}
b3() {
    :
}
b4() {
    :
}

main() {
    # source func.sh
    echo $VAR_A1
    echo $VAR_A2
    echo $VAR_B1
    echo $VAR_B2
    a1
    a2
    a3
    a4
    b1
    b2
    b3
    b4
    a5
    a6
}
main

Observation:

• Go to definition & jump to function/variable within the same file always works
• if the line source func.sh is active in the main function, the reference works also for functions/variables across the files mentioned by the source line

Suggestion:
the reference should always work even if the source action is not so obvious, as we sometimes use loops to iterate multiple source files...

[skovhus]

the reference should always work even if the source action is not so obvious, as we sometimes use loops to iterate multiple source files...

That means the we would suggest symbols an across the entire workspace, which if it contains more than a few bash files does give a rather cluttered and false suggestions/jump to definition.

But the bahaviour is configurable. We default to analyze simple source statements, a feature that I wish to extend to analyze some dynamic sourcing and sourcing done inside functions. But if you prefer resolving symbols from the entire workspace you can use the includeAllWorkspaceSymbols configuration option. See https://github.com/bash-lsp/bash-language-server/blob/main/server/src/config.ts#L18

I would like to understand how we can make this behaviour more clear to the end user. I believe the current default is sane, but that can be debated. But if we keep the current default we could show a warning on source statements that we cannot parse (I.e. “failed to analyze sourced file, resolution of symbols can be configured using…”)

let me know how that sounds.

[skovhus]

And I would like to see examples of source statements from you project. :)

[timoteicampian]

_source() { for i in $*; do [[ -r $i ]] && echo $i && source $i; done; }

[timoteicampian]

the reference should always work even if the source action is not so obvious, as we sometimes use loops to iterate multiple source files...

That means the we would suggest symbols an across the entire workspace, which if it contains more than a few bash files does give a rather cluttered and false suggestions/jump to definition.

But the bahaviour is configurable. We default to analyze simple source statements, a feature that I wish to extend to analyze some dynamic sourcing and sourcing done inside functions. But if you prefer resolving symbols from the entire workspace you can use the includeAllWorkspaceSymbols configuration option. See https://github.com/bash-lsp/bash-language-server/blob/main/server/src/config.ts#L18

I would like to understand how we can make this behaviour more clear to the end user. I believe the current default is sane, but that can be debated. But if we keep the current default we could show a warning on source statements that we cannot parse (I.e. “failed to analyze sourced file, resolution of symbols can be configured using…”)

let me know how that sounds.

setting true includeAllWorkspaceSymbols: z.boolean().default(true) does not help for the latest version.
only switching back to 1.18.0 does help + Disable Auto Update for all extensions, otherwise after each vs code restart the extension get updated even "Ignore Update" for Shell IDE was activated.
the bash code is stored on a common linux server...

[skovhus]

If I create the two files mentioned in #646 (comment) in a new workspace and enable includeAllWorkspaceSymbols from the vscode settings then I do get jump to definition and jump to references. That isn't working for you using the latest version? Or do you have another example script that isn't working? Is it related to your setup of having a common linux server or can you also reproduce this locally? 🤔

I'm also curious which exact version that started breaking. You mention 1.18.0 working, but could you try 1.20.1? The behaviour of parsing source commands was first introduced in 1.21.0.

I would like to understand how we can make this behaviour more clear to the end user. I believe the current default is sane, but that can be debated. But if we keep the current default we could show a warning on source statements that we cannot parse (I.e. “failed to analyze sourced file, resolution of symbols can be configured using…”)

I've created #658 to capture this. Feel free to add additional input and ideas.

[skovhus]

@timoteicampian can you provide an example where enabling includeAllWorkspaceSymbols wouldn't make jump to definition work again? I would really like to have a reproducible example so we can fix this issue.

[skovhus]

@timoteicampian A fix has now been released as [email protected] and the latest vscode extension 1.29.0. Let me know if it works for you and if you have ideas for future improvements.

[andreicaba]

@timoteicampian A fix has now been released as [email protected] and the latest vscode extension 1.29.0. Let me know if it works for you and if you have ideas for future improvements.

A feature that could benefit Bash IDE users greatly would be if the needed library source files to run a specific script would be automatically listed by Bash IDE - given a specific library folder where to choose from.

[skovhus]

@andreicaba sounds interesting. Could you write up an issue with additional detail? E.g. would these be suggestions after a source command or? https://github.com/bash-lsp/bash-language-server/issues/new?assignees=&labels=enhancement&template=2-feature-request.yml

[andreicaba]

@andreicaba sounds interesting. Could you write up an issue with additional detail? E.g. would these be suggestions after a source command or? https://github.com/bash-lsp/bash-language-server/issues/new?assignees=&labels=enhancement&template=2-feature-request.yml

OK, I did it :)
#687

[timoteicampian]

@timoteicampian A fix has now been released as [email protected] and the latest vscode extension 1.29.0. Let me know if it works for you and if you have ideas for future improvements.

works perfectly now - T H X ! ! -

[timoteicampian]

Sorry @skovhus to disappoint you, but I had to switch back to the stable (for me) version 1.18 again
on the same day I confirmed that the fix version is working, as I had no time to inform you or test the new version properly to understand why does not work for me...

[skovhus]

@timoteicampian could you share an example script that works in 1.18 but not in the latest version? The error logs just shows some files that fail to parse (I assume you removed the file names?) – I'm thinking they also failed to parse with version 1.18, so I'm curious to see some example code.

[timoteicampian]

they are working nice with 1.18 all the time :)

[skovhus]

they are working nice with 1.18 all the time :)

That is great, but I would still like some example code to debug this. :)

[timoteicampian]

sure...I have uploaded a example file please check, I'll put soon much more if you need them

[timoteicampian]

I uploaded 2 script files + output of Bash IDE

[skovhus]

Thanks for the additional scripts. It turns out that for files where the treesitter parser fails, we didn't gracefully handle this in the later version of bash-language-server. I believe the error handling in the latest extension (1.30.0) is now the same as 1.18 – but including all the new improvements.

Let me know if it works for you.

[timoteicampian]

same error for the same two files
updated errors there

[skovhus]

same error for the same two files

Note that the error logs are expected. In older versions this was shown as diagnostics (in the editor) if you enabling highlight parsing errors. So the syntax errors in the logs are expected.

Anything that is working in the old version that isn’t working in the new version? If so please be specific with an example like “jump to definition on line X”.

[skovhus]

This is my findings:

progress_bar.sh:

• 1.18 jump to defintion and on hover works for _fill, _empty, but not for _progress as we do not parse let commands
• 1.30 same behaviour + improvement that hovering on _fill returns documentation "Build progressbar string lengths"

output_print.sh

• 1.18 same as 1.30, although jump to definition might not work if the variables are duplicated in the workspace
• 1.30 jump to definition, hover, and click works for everything but jd_COUNTER_PRINT, random_op, b and c due to parse problems.

So I conclude that 1.30 is an improvement to 1.18. Let me know if you find anything that works in 1.18, but not in 1.30.

[timoteicampian]

Dear @skovhus 1.18 works in my case for all scripts all the time :)
but the files provided and for almost all other in my project the later version do not work
but it is true that i have a lot of duplicate files in my workspace as I am using symlinks for files and folders

[skovhus]

Okay. The only thing I need in order to investigate and potentially fix the issue is concrete reproducible examples including what isn’t working in the latest version (steps to reproduce). As mentioned, then the two files here actually works better in the newest version.

So I would appreciate more detailed examples of what isn’t working including some source files, then I’ll work on fixing the issues.

[timoteicampian]

Dear @skovhus,
we found something :)

Let's say we want to source "file1" inside "run.sh".
Because "file1" does not end in .sh, it won't parse it for possible functions even if I source it's full path.
Now, if I manually open the "file1" source file, it automagically know the function name as a workspace symbol.
But if we close VS Code and reopen it -> and then open only the "run.sh", once again, it won't know the functions defined in "file1" source file, even if we do "source ../file1" inside "run.sh".

cat file1

f() {
    echo this is a test
    echo "$@"
}

cat run.sh

#!/usr/bin/bash
source ../file1
f test

[skovhus]

That specific problem of including symbols from a sourced file without a file extension also doesn't work in 1.18. We use the background analysis glob (configurable) to analyze the workspace.

But with the latest extension we actually do support parsing sourced files in case the the includeAllWorkspaceSymbols configuration flag is disabled (this is the default). We should probably make analyzing sourced files work no matter how the includeAllWorkspaceSymbols configuration flag is set.

Example

File: x.sh

#!/bin/sh
. ./extension

echo "It works when includeAllWorkspaceSymbols is false as we follow sourced commands ${XXX}"

File: ./extension

#!/bin/sh

# documentation for XXX
export XXX=1

[skovhus]

A fix has been released in the vscode extension version 1.32, so in case includeAllWorkspaceSymbols we still use source commands to find additional files not covered by the background analysis based on the globPattern. This means that sourced files not matching **/*@(.sh|.inc|.bash|.command) should still be analyzed and used for completions, jump to definition, etc.

[andreicaba]

Go to definition also doesn't work if a function is defined under where it's called. But - if it's about a library file, it shouldn't really be a problem to be called before the function is defined. And - actually - if it ever is to be signaled, this potential problem can be signaled by shellcheck and it doesn't seem to be a real need to be signaled by "Bash IDE" too (while I can understand that some people might want to use Bash IDE as a language server for this :) )

short example:

f() {
    echo another test
    g
}

g() {
    echo the following function
}

h() {
    echo last test
    g
}

here g can be found inside h, but not inside f.

[skovhus]

Go to definition also doesn't work if a function is defined under where it's called.

Yeah that is a limitation of the current scope aware heuristic, implemented in #649 … The new heuristic should overall be better, where the previous one just found all symbols no matter the scope. That meant that jump to definition would jump to the latest definition in many cases.

That being said: I would love to improve the heuristic based on your examples and feedback! Thanks for providing these. This specific case should be easy to fix.

And - actually - if it ever is to be signaled, this potential problem can be signaled by shellcheck and it doesn't seem to be a real need to be signaled by "Bash IDE" too

My intention with improving the scope aware symbol resolution was never to signal problems to the end user but improve the overall experience by providing more accurate information — correct the previous sometimes confusing behaviour. See the different issues we fixed with the new heuristic: #649

What shellcheck does internally is actually flow tracing the scripts. I was hoping to avoid that complexity and provide something that would be “good enough” based on a few AST traversals.

So any examples of scripts with incorrect behaviour is very welcome!

[andreicaba]

Maybe it would be providing more accurate information if, after checking that the symbol is nowhere before, it would still provide a hyperlink to the function if it's defined afterward.
Also... Another (possibly better) implementation would be to just sort the results by where the definition was found.
This would push the choice of picking a definition to the end user. I would prefer something like that - and maybe there would be others like me (while going through some bash "library files" that are being sourced in other files).

[skovhus]

@andreicaba can I get you to create a new issue capturing your suggestions for improving this or the bugs that you experience? That would be a huge help.