Skip to content

Pipeline parameters injected in Docker #5129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DRKolev-code opened this issue Apr 16, 2025 · 1 comment
Open

Pipeline parameters injected in Docker #5129

DRKolev-code opened this issue Apr 16, 2025 · 1 comment
Labels
component: pysdk-team Related to SageMaker Python SDK Core Issues type: bug

Comments

@DRKolev-code
Copy link

I will write this here too. Relates to:

#5124

This does not fix the issue. The offending code is:

                if isinstance(arg, PipelineVariable):
                    processed_value = json.dumps(arg.expr)
                    processed_arguments.append(processed_value)
                    print(processed_value)
                    print("YES")

This produces the broken arguments:

NO: --test_dict
YES: {"Get": "Parameters.test_dict"}
NO: --test_list
YES: {"Get": "Parameters.test_list"}
NO: --test_string
YES: {"Get": "Parameters.test_string"}
NO: --test_bool
YES: {"Std:Join": {"On": "", "Values": [{"Get": "Parameters.test_bool"}]}}
NO: --test_float
YES: {"Std:Join": {"On": "", "Values": [{"Get": "Parameters.test_float"}]}}
NO: --test_int
YES: {"Std:Join": {"On": "", "Values": [{"Get": "Parameters.test_int"}]}}
NO: --test_param
YES: {"Get": "Parameters.test_param"}

The Dockerimages look like this:

                                 - --test_dict
                                 - '{"Get": "Parameters.test_dict"}'
                                 - --test_list
                                 - '{"Get": "Parameters.test_list"}'
                                 - --test_string
                                 - '{"Get": "Parameters.test_string"}'
                                 - --test_bool
                                 - '{"Std:Join": {"On": "", "Values": [{"Get":
                             "Parameters.test_bool"}]}}'
                                 - --test_float
                                 - '{"Std:Join": {"On": "", "Values": [{"Get":
                             "Parameters.test_float"}]}}'
                                 - --test_int
                                 - '{"Std:Join": {"On": "", "Values": [{"Get":
                             "Parameters.test_int"}]}}'
                                 - --test_param
                                 - '{"Get": "Parameters.test_param"}'
@joshpulse
Copy link

joshpulse commented Apr 16, 2025

+1

We are facing the same issue. This breaks our production pipelines due to how we use the Pipeline parameters. By performing the json dumps, it is not allowing the pipeline variable to resolve to the value at runtime.

In the meantime, I have pinned the version to 2.243.0 as that is the last version that works with the pipeline variables I use.

@nargokul nargokul added type: bug component: pysdk-team Related to SageMaker Python SDK Core Issues labels Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: pysdk-team Related to SageMaker Python SDK Core Issues type: bug
Projects
None yet
Development

No branches or pull requests

3 participants