Skip to content

Checkov (3.2.368) crashes with KeyError: '__startline__' when scanning restructured serverless YAML file #7006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
vish-space opened this issue Feb 10, 2025 · 6 comments · May be fixed by #7115
Open

Checkov (3.2.368) crashes with KeyError: '__startline__' when scanning restructured serverless YAML file #7006

vish-space opened this issue Feb 10, 2025 · 6 comments · May be fixed by #7115
Labels
crash

Comments

@vish-space
Copy link

vish-space commented Feb 10, 2025 •
edited
Loading

We've been using Checkov for serverless checks in our Bitbucket Pipelines CI/CD, and it has been working fine with our standard serverless configurations, scanning everything properly. However, after recently modularizing our serverless YAML files—splitting them into separate files for functions and resources, and referencing them in the main serverless.yml — Checkov started throwing errors during scans. Specifically, it seems unable to process the file references within the main YAML file. Since restructuring, Checkov consistently throws a KeyError: '__startline__' when scanning. We've validated the YAML files using yamllint, and no syntax errors are present. The issue appears to be related to how Checkov handles modular serverless resources, and it persists even after addressing all reported YAML issues.

Examples

  1. Please refer this Example code to reproduce the error it contains code before and after restructuring serverless https://github.com/vish-space/test-examples
  2. Run Checkov: checkov -f serverless.yml

Exception Trace
ran LOG_LEVEL=debug checkov -d .

2025-02-10 11:22:41,323 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_10
2025-02-10 11:22:41,323 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_48
2025-02-10 11:22:41,323 [MainThread  ] [DEBUG]  Running check: Ensure that AWS Lambda function permissions delegated to AWS services are limited by SourceArn or SourceAccount on file /ResourceCF-CustomPermission.yml
2025-02-10 11:22:41,323 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_46
2025-02-10 11:22:41,323 [MainThread  ] [DEBUG]  File /ResourceCF-CustomPermission.yml, resource "AWS::Lambda::Permission.CustomFunctionInvokePermission" check "Ensure that AWS Lambda function permissions delegated to AWS services are limited by SourceArn or SourceAccount" Result: {'result': <CheckResult.PASSED: 'PASSED'>, 'evaluated_keys': ['Properties/Principal', 'Properties/SourceArn', 'Properties/SourceAccount']}
2025-02-10 11:22:41,323 [MainThread  ] [DEBUG]  Secret was not saved in CKV_AWS_364, can't omit
2025-02-10 11:22:41,324 [MainThread  ] [DEBUG]  Secret was not saved in CKV_AWS_364, can't omit
2025-02-10 11:22:41,325 [MainThread  ] [DEBUG]  skip_severity = None, explicit_skip = [], regex_match = False, suppressed_policies: []
2025-02-10 11:22:41,332 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_22
2025-02-10 11:22:41,331 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_32
2025-02-10 11:22:41,331 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AZURE_36
2025-02-10 11:22:41,335 [MainThread  ] [DEBUG]  Searching through [] and ['__init__.cpython-311.pyc']
2025-02-10 11:22:41,336 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_71
2025-02-10 11:22:41,348 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_DOCKER_6
2025-02-10 11:22:41,349 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_DOCKER_14
2025-02-10 11:22:41,351 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_53
2025-02-10 11:22:41,352 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_14
2025-02-10 11:22:41,350 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_44
2025-02-10 11:22:41,355 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_54
2025-02-10 11:22:41,355 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_27
2025-02-10 11:22:41,353 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_AWS_46
2025-02-10 11:22:41,355 [MainThread  ] [DEBUG]  Running function /home/vish/.local/lib/python3.11/site-packages/checkov/common/runners/object_runner.<lambda> with parallelization type 'fork'
2025-02-10 11:22:41,367 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_OCI_5
2025-02-10 11:22:41,367 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_OCI_6
2025-02-10 11:22:41,367 [ThreadPoolEx] [DEBUG]  Running graph check: CKV2_OCI_2
2025-02-10 11:22:41,411 [MainThread  ] [DEBUG]  Running function /home/vish/.local/lib/python3.11/site-packages/checkov/common/runners/object_runner.<lambda> with parallelization type 'fork'
2025-02-10 11:22:41,412 [MainThread  ] [DEBUG]  Running function /home/vish/.local/lib/python3.11/site-packages/checkov/common/runners/object_runner.<lambda> with parallelization type 'fork'
2025-02-10 11:22:41,392 [MainThread  ] [ERROR]  Exception traceback:
Traceback (most recent call last):
  File "/home/vish/.local/lib/python3.11/site-packages/checkov/main.py", line 532, in run
    self.scan_reports = runner_registry.run(
                        ^^^^^^^^^^^^^^^^^^^^
  File "/home/vish/.local/lib/python3.11/site-packages/checkov/common/runners/runner_registry.py", line 177, in run
    for result in parallel_runner_results:
  File "/home/vish/.local/lib/python3.11/site-packages/checkov/common/parallelizer/parallel_runner.py", line 110, in _run_function_multiprocess_fork
    raise v.internal_exception.with_traceback(v.internal_exception.__traceback__)
KeyError: '__startline__'
2025-02-10 11:22:41,435 [MainThread  ] [DEBUG]  Running function /home/vish/.local/lib/python3.11/site-packages/checkov/common/runners/object_runner.<lambda> with parallelization type 'fork'

checkov --version 3.2.368

Desktop (please complete the following information):

  • OS: Ubuntu 20.04.6
  • Checkov Version 3.2.368

Additional context

  • Rolling back to older Checkov versions resulted in the same behavior.
@jbabala
Copy link

jbabala commented Feb 11, 2025

I'm having the same issue (serverless checks fail with the above error). Through trial and error I have found last working version to be 3.2.332.

@Stephen-Cousins
Copy link

I'm having the same issue (serverless checks fail with the above error). Through trial and error I have found last working version to be 3.2.332.

I have made this change for another project affected by it.
https://github.com/costa-coffee/costa-digital-inventory-availability/pull/124

@vish-space
Copy link
Author

@jbabala I tested with 3.2.332 and while it resolves the issue for either resources or functions, it doesn’t work for both at the same time. If I focus on modernizing resources, functions fail, and vice versa.

@tsmithv11 tsmithv11 mentioned this issue Mar 31, 2025
Merged
5 tasks
@tsmithv11
Copy link
Collaborator

This crash no longer happens with 3.2.395. I'll close this Issue. If you run into the same problem again, please re-open this or open a new Issue.

@vish-space
Copy link
Author

vish-space commented Apr 4, 2025 •
edited
Loading

Hi @tsmithv11 Thanks for the update. I’ve just tested with 3.2.395 version and unfortunately, I’m still encountering the same issue/error as before (modularized Serverless files), hence would like to request to reopen this for further investigation.

@tsmithv11
Copy link
Collaborator

Hi @vish-space, thanks for the update. I'm not able to replicate the crash with the repo you provided. I'll reopen the issue to see if we can get a contribution.

@tsmithv11 tsmithv11 reopened this Apr 4, 2025
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 21, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
119a31b 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
@thentenaar thentenaar linked a pull request Apr 21, 2025 that will close this issue
Open
5 tasks
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 21, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
65304c2 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 22, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
7b5bc41 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 24, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
d5f53c0 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 24, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
d987230 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 26, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
eb96929 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 28, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
2e78162 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 28, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
a394af1 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 28, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
8acb06f 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 29, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
c623529 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 29, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
a087313 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue Apr 29, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
884259d 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 1, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
9d2935d 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 1, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
47173b1 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 5, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
b827283 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 6, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
5cbf81f 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 6, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
eb866b6 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 6, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
b892163 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 6, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
e879c35 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 6, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
ee017d5 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
thentenaar added a commit to thentenaar/checkov that referenced this issue May 9, 2025
@thentenaar
fix(serverless): Enhance yaml parsing, better support for file expans…
f804c81 
…ions

- Improve yaml parsing wrt file() expansion
- Add tests for file() expansion
- Add a '__file__' marker attribute to yaml nodes
- Utilize the '__file__' marker when generating reports for serverless
- Raise CfnParseError on circular inclusions
- Added a ``logger.error`` that logs specifics on pyyaml parse errors
- Updated the serverless graph builder to cope with file() expansion

Fixes bridgecrewio#7006
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(serverless): Enhance yaml parsing, better support for file expansion thentenaar/checkov
4 participants
@tsmithv11 @jbabala @Stephen-Cousins @vish-space