Multiple test failures in Gentoo's network sandbox #3126

hololeap · 2022-08-27T16:54:33Z

I am getting a lot of test failures from the hls-eval-plugin test suite using 55d9024:

62 out of 65 tests failed (272.92s)

Full build log is here.

Your environment

GHC 9.0.2 on Gentoo Linux. All packages are installed through the system package manager.

I am building the following packages from 55d9024:

ghcide
haskell-language-server
hls-alternate-number-format-plugin
hls-change-type-signature-plugin
hls-code-range-plugin
hls-eval-plugin
hls-floskell-plugin
hls-fourmolu-plugin
hls-graph
hls-hlint-plugin
hls-plugin-api
hls-rename-plugin
hls-retrie-plugin
hls-test-utils

I am building the rest from Hackage with the following versions (let me know if you need more):

aeson-2.0.3.0
data-default-0.7.1.1
diff-0.4.0
dlist-1.0
extra-1.7.10
ghc-paths-0.1.0.12
hashable-1.4.0.2
lens-5.1.1
lsp-1.5.0.0
lsp-types-1.5.0.0
megaparsec-9.2.1
parser-combinators-1.3.0
pretty-simple-4.1.1.0
quickcheck-2.14.2
safe-exceptions-0.1.7.1
unliftio-0.2.18
unordered-containers-0.2.17.0

The text was updated successfully, but these errors were encountered:

hololeap · 2022-09-02T17:06:58Z

Are these tests passing elsewhere? If so, can someone give me a link to where they are passing?

pepeiborra · 2022-09-02T17:57:39Z

They are passing in the Github CI. link

wz1000 · 2022-09-07T11:40:04Z

Perhaps you want to test using the environment variables LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 to get more information on why it fails.

hololeap · 2022-09-23T18:23:03Z

@wz1000, thanks for the hint about the environment variables.

It looks like this has something to do with Gentoo's build sandboxing. All the tests pass when run outside the sandbox.

I am seeing blocked network calls being logged. It looks like cabal is being run as a part of the test, which is trying to download something.

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-eval-plugin-1.3.0.0/work/hls-eval-plugin-1.3.0.0/test/testdata/TLocalImport.hs",
        "version": 0
    }
}

Full build log with LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 is here.

hololeap · 2022-09-23T21:49:03Z

It looks like I'm running into the same thing for hls-module-rename-plugin-1.1.0.0:

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-module-name-plugin-1.1.0.0/work/hls-module-name-plugin-1.1.0.0/test/testdata/canonicalize/Lib/A.hs",
        "version": 0
    }
}

FAIL (16.59s)
    Received an unexpected message from the server:
    Was parsing: Pattern match failure in do expression at test/Main.hs:48:7-38
    But the last message received was:
    {
        "id": 2,
        "jsonrpc": "2.0",
        "result": []
    }
    Use -p '/Fix#3047/' to rerun this test only.

1 out of 6 tests failed (24.41s)
Test suite tests: FAIL

Full build log

This test passes outside of the sandbox.

hololeap · 2022-09-24T15:51:23Z

There are more seemingly-related test failures, this time from ghcide:

typecheck-all-parents-of-interest

FAIL (2.13s)
      test/exe/Main.hs:724:
      Incorrect number of diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353299/A.hs"}, expected [] but got [Diagnostic {_range = Range {_start = Positio
n {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Couldn't match e
xpected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing},Diagnostic {_range = Range {_sta
rt = Position {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Coul
dn't match expected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing}]
      Use -p '/typecheck-all-parents-of-interest/' to rerun this test only.

simple plugin

FAIL (7.77s)
    test/exe/Main.hs:1196:
    Could not find (DsError,(9,15),"Variable not in scope: c",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
    Use -p '/simple plugin/' to rerun this test only.

notification handlers run in priority order

FAIL
      Exception: Timed out waiting to receive a message from the server.
      Use -p '/notification handlers run in priority order/' to rerun this test only.

ignore-fatal-warning

FAIL (9.15s)
        test/exe/Main.hs:2512:
        Got unexpected diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353468/IgnoreFatal.hs"} got [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
        Use -p '/ignore-fatal-warning/' to rerun this test only.

simple-multi-test

FAIL (7.49s)
        test/exe/Main.hs:2532:
        A should typecheck
        Use -p '$0=="ghcide.cradle.multi.simple-multi-test"' to rerun this test only.

simple-multi-test2

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-test2/' to rerun this test only.

simple-multi-test3

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-test3/' to rerun this test only.

simple-multi-def-test

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-def-test/' to rerun this test only.

simple-subdirectory

FAIL (5.22s)
        test/exe/Main.hs:2520:
        Could not find (DsWarning,(2,0),"Top-level binding",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
        Use -p '/simple-subdirectory/' to rerun this test only.

ghcide command line.works

  ghcide command line
    works:                                                                            FAIL (4.47s)
      test/exe/Main.hs:2814:
      expected: ExitSuccess
       but got: ExitFailure 1
      Use -p '/ghcide command line.works/' to rerun this test only.

13 out of 344 tests failed (2982.71s)

(Three tests fail regardless of the network-sandbox. See #3221.)

Full build log (raw file is ~15MB)

pepeiborra · 2022-09-24T21:45:15Z

The problem is that Tasty runs test in parallel by default, and most HLS test suites do not handle this well. Set TASTY_NUM_THREADS to 1

michaelpj · 2022-10-05T11:50:47Z

Did you try Pepe's suggestion? did it help?

michaelpj · 2024-01-16T18:20:28Z

Closing as old and quiet.

unhammer · 2025-03-29T08:31:21Z

Possibly related, I was unable to start HLS in emacs without being online:

[stderr]  [2025-03-29T08:22:37Z INFO  emacs_lsp_booster::app] Running server "haskell-language-server-wrapper" "--lsp"
[stderr]  [2025-03-29T08:22:37Z INFO  emacs_lsp_booster::app] Will convert server json to bytecode! bytecode options: BytecodeOptions { object_type: Plist, null_value: Nil, false_value: Keyword("json-false") }
[stderr]  nil
[stderr]  No 'hie.yaml' found. Try to discover the project type!
[stderr]  Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper) Version 2.0.0.0 x86_64 ghc-9.2.8
[stderr]  Current directory: /home/me/foo
[stderr]  Operating system: linux
[stderr]  Arguments: ["--lsp"]
[stderr]  Cradle directory: /home/me/foo
[stderr]  Cradle type: Cabal
[stderr]
[stderr]
[stderr]  Tool versions found on the $PATH
[stderr]  cabal:          3.12.1.0
[stderr]  stack:          3.1.1
[stderr]  ghc:            9.2.8
[stderr]
[stderr]
[stderr]  Consulting the cradle to get project GHC version...
[stderr]  2025-03-29T08:22:38.402607Z | Debug | executing command: cabal exec -v0 -- ghc --print-libdir
[stderr]  nil
[stderr]  Failed to find the GHC version of this Cabal project.
[stderr]  Error when calling cabal exec -v0 -- ghc --print-libdir
[stderr]
[stderr]  dieVerbatim: user error (Error: cabal: '/usr/bin/curl' exited with an error:
[stderr]  curl: (6) Could not resolve host: hackage.haskell.org
[stderr]  )

then I connected (now in the terminal) and saw that it complained about cabal not being updated:

$ haskell-language-server-wrapper
No 'hie.yaml' found. Try to discover the project type!
Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper) Version 2.0.0.0 x86_64 ghc-9.2.8
Current directory: /home/me/foo
Operating system: linux
Arguments: []
Cradle directory: /home/me/foo
Cradle type: Cabal

Tool versions found on the $PATH
cabal:          3.12.1.0
stack:          3.1.1
ghc:            9.2.8


Consulting the cradle to get project GHC version...
2025-03-29T08:24:48.403936Z | Debug | executing command: cabal exec -v0 -- ghc --print-libdir
Failed to find the GHC version of this Cabal project.
Error when calling cabal exec -v0 -- ghc --print-libdir

Error: [Cabal-7160]
The package list for 'hackage.haskell.org' does not exist. Run 'cabal update' to download it.

I don't understand why it needs to do that.

Anyway, I did cabal update and now I am able to use HLS in airplane mode or sketchy wifi's again.

(Yes I should update HLS, I haven't found time to test on the newest, but hopefully this helps anyone looking for the same error message.)

hololeap added status: needs triage type: bug Something isn't right: doesn't work as intended, documentation is missing/outdated, etc.. labels Aug 27, 2022

kokobd added component: hls-eval-plugin and removed status: needs triage labels Sep 7, 2022

wz1000 added the status: needs info Not actionable, because there's missing information label Sep 7, 2022

This was referenced Sep 23, 2022

hls-hlint-plugin test failure in Gentoo Linux #3217

Open

hls-gadt-plugin test failures in Gentoo Linux #3218

Open

hololeap changed the title ~~hls-eval-plugin test failures on Gentoo Linux~~ Multiple test failures in Gentoo's network sandbox Sep 23, 2022

hololeap mentioned this issue Sep 24, 2022

ghcide test failures in Gentoo Linux #3221

Open

3 tasks

hololeap mentioned this issue Sep 24, 2022

Flakey tests in Gentoo Linux #3224

Closed

hololeap mentioned this issue Sep 25, 2022

HLS-1.8 gentoo-haskell/gentoo-haskell#1332

Merged

10 tasks

hololeap mentioned this issue Nov 13, 2022

haskell-language-server failing tests gentoo-haskell/gentoo-haskell#1359

Open

10 tasks

michaelpj closed this as completed Jan 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multiple test failures in Gentoo's network sandbox #3126

Multiple test failures in Gentoo's network sandbox #3126

hololeap commented Aug 27, 2022

hololeap commented Sep 2, 2022

pepeiborra commented Sep 2, 2022

wz1000 commented Sep 7, 2022

hololeap commented Sep 23, 2022 •

edited

Loading

hololeap commented Sep 23, 2022

hololeap commented Sep 24, 2022 •

edited

Loading

pepeiborra commented Sep 24, 2022

michaelpj commented Oct 5, 2022

michaelpj commented Jan 16, 2024

unhammer commented Mar 29, 2025 •

edited

Loading

Multiple test failures in Gentoo's network sandbox #3126

Multiple test failures in Gentoo's network sandbox #3126

Comments

hololeap commented Aug 27, 2022

Your environment

hololeap commented Sep 2, 2022

pepeiborra commented Sep 2, 2022

wz1000 commented Sep 7, 2022

hololeap commented Sep 23, 2022 • edited Loading

hololeap commented Sep 23, 2022

hololeap commented Sep 24, 2022 • edited Loading

pepeiborra commented Sep 24, 2022

michaelpj commented Oct 5, 2022

michaelpj commented Jan 16, 2024

unhammer commented Mar 29, 2025 • edited Loading

hololeap commented Sep 23, 2022 •

edited

Loading

hololeap commented Sep 24, 2022 •

edited

Loading

unhammer commented Mar 29, 2025 •

edited

Loading