Skip to content

When providing custom code for MLflow for ModelBuilder (ie with mlflow.pyfunc.PythonModel) where JSON input is used, the input_fn will fail the request as bytes are parsed to string causing errors. #4959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
andjsmi opened this issue Dec 10, 2024 · 1 comment

Comments

@andjsmi
Copy link
Contributor

andjsmi commented Dec 10, 2024

Describe the bug
If I use a custom MLflow model with custom predict function and use a JSON or CSV input, the requests fail as the SageMaker Inference Toolkit converts bytes to string (source: ref) whereas the input_fn for ModelBuilder passes the input data directly into io.BytesIO (source: ref).

For scenarios where we might have a custom JSON input for a custom MLflow model, this doesn't work as JSON currently isn't supported.

To reproduce

  1. Create an MLflow model with a custom Python definition with a custom predict
  2. Deploy using ModelBuilder
  3. Invoke model with JSON model
  4. Request fails due to io.BytesIO input being a string instead of bytes

Expected behavior
Request succeeds and returns

Screenshots or logs
If applicable, add screenshots or logs to help explain your problem.

System information
A description of your system. Please provide:

  • SageMaker Python SDK version: 2.237.0
  • Framework name (eg. PyTorch) or algorithm (eg. KMeans): PyTorch / MLflow
  • Framework version: NA
  • Python version: Python 3.10.14
  • CPU or GPU: CPU
  • Custom Docker image (Y/N): NA

Additional context
Add any other context about the problem here.

@andjsmi
Copy link
Contributor Author

andjsmi commented Dec 21, 2024

Addressed with PR #4960

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant