diff --git a/README.md b/README.md index 2065a983342..48b247ecb23 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,4 @@ + # AWS Lambda Powertools for Python [![Build](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml/badge.svg)](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml) @@ -5,7 +6,7 @@ ![PythonSupport](https://img.shields.io/static/v1?label=python&message=3.6%20|%203.7|%203.8|%203.9&color=blue?style=flat-square&logo=python) ![PyPI version](https://badge.fury.io/py/aws-lambda-powertools.svg) ![PyPi monthly downloads](https://img.shields.io/pypi/dm/aws-lambda-powertools) [![Join our Discord](https://dcbadge.vercel.app/api/server/B8zZKbbyET)](https://discord.gg/B8zZKbbyET) -A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools [Java](https://github.com/awslabs/aws-lambda-powertools-java) and [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript) is also available). +A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools for [Java](https://github.com/awslabs/aws-lambda-powertools-java), [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript), and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"} are also available). **[📜Documentation](https://awslabs.github.io/aws-lambda-powertools-python/)** | **[🐍PyPi](https://pypi.org/project/aws-lambda-powertools/)** | **[Roadmap](https://awslabs.github.io/aws-lambda-powertools-python/latest/roadmap/)** | **[Detailed blog post](https://aws.amazon.com/blogs/opensource/simplifying-serverless-best-practices-with-lambda-powertools/)** diff --git a/docs/core/tracer.md b/docs/core/tracer.md index 8fbfc0e29f7..018af91797b 100644 --- a/docs/core/tracer.md +++ b/docs/core/tracer.md @@ -19,6 +19,14 @@ Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github. ???+ tip All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}. +### Install + +!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}" + +Add `aws-lambda-powertools[tracer]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_. + +This will ensure you have the required dependencies before using Tracer. + ### Permissions Before your use this utility, your AWS Lambda function [must have permissions](https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html#services-xray-permissions) to send traces to AWS X-Ray. diff --git a/docs/index.md b/docs/index.md index 1c6c4e80995..e83d5d806f6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,7 +11,7 @@ description: AWS Lambda Powertools for Python A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, and more. ???+ note - Lambda Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"} and [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}. + Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"}, [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}, and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"} ## Install @@ -20,11 +20,29 @@ Powertools is available in the following formats: * **Lambda Layer**: [**arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPython:33**](#){: .copyMe}:clipboard: * **PyPi**: **`pip install aws-lambda-powertools`** +???+ info "Some utilities require additional dependencies" + You can stop reading if you're using Lambda Layer. + + [Tracer](./core/tracer.md){target="_blank"}, [Validation](./utilities/validation.md){target="_blank"} and [Parser](./utilities/parser.md){target="_blank"} require additional dependencies. If you prefer to install all of them, use `pip install aws-lambda-powertools[all]`. + ???+ hint "Support this project by using Lambda Layers :heart:" Lambda Layers allow us to understand who uses this library in a non-intrusive way. This helps us justify and gain future investments for other Lambda Powertools languages. When using Layers, you can add Lambda Powertools as a dev dependency (or as part of your virtual env) to not impact the development process. +### Local development + +Powertools relies on the AWS SDK bundled in the Lambda runtime. This helps us achieve an optimal package size and initialization. + +This means you need to add AWS SDK as a development dependency (not as a production dependency). + +* **Pip**: `pip install aws-lambda-powertools[aws-sdk]` +* **Poetry**: `poetry add aws-lambda-powertools[aws-sdk] --dev` +* **Pipenv**: `pipenv install --dev "aws-lambda-powertools[aws-sdk]"` + +???+ note "Local emulation" + If you're running your code locally with [AWS SAM CLI](https://github.com/aws/aws-sam-cli){target="_blank"}, and not with your Python/IDE interpreter directly, this is not necessary. SAM CLI already brings the AWS SDK in its emulation image. + ### Lambda Layer [Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html){target="_blank"} is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. Layers promote code sharing and separation of responsibilities so that you can iterate faster on writing business logic. diff --git a/docs/utilities/parser.md b/docs/utilities/parser.md index e97395ae56c..c5112df737a 100644 --- a/docs/utilities/parser.md +++ b/docs/utilities/parser.md @@ -1,5 +1,5 @@ --- -title: Parser +title: Parser (Pydantic) description: Utility --- @@ -12,20 +12,25 @@ This utility provides data parsing and deep validation using [Pydantic](https:// * Built-in envelopes to unwrap, extend, and validate popular event sources payloads * Enforces type hints at runtime with user-friendly errors -**Extra dependency** +## Getting started -???+ warning +### Install + +!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}" + +Add `aws-lambda-powertools[parser]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_. +This will ensure you have the required dependencies before using Parser. + +???+ warning This will increase the compressed package size by >10MB due to the Pydantic dependency. To reduce the impact on the package size at the expense of 30%-50% of its performance [Pydantic can also be installed without binary files](https://pydantic-docs.helpmanual.io/install/#performance-vs-package-size-trade-off): - `SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[pydantic]` + Pip example: `SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[parser]` -Install parser's extra dependencies using **`pip install aws-lambda-powertools[pydantic]`**. - -## Defining models +### Defining models You can define models to parse incoming events by inheriting from `BaseModel`. @@ -47,11 +52,11 @@ class Order(BaseModel): These are simply Python classes that inherit from BaseModel. **Parser** enforces type hints declared in your model at runtime. -## Parsing events +### Parsing events You can parse inbound events using **event_parser** decorator, or the standalone `parse` function. Both are also able to parse either dictionary or JSON string as an input. -### event_parser decorator +#### event_parser decorator Use the decorator for fail fast scenarios where you want your Lambda function to raise an exception in the event of a malformed payload. @@ -104,7 +109,7 @@ handler(event=payload, context=LambdaContext()) handler(event=json.dumps(payload), context=LambdaContext()) # also works if event is a JSON string ``` -### parse function +#### parse function Use this standalone function when you want more control over the data validation process, for example returning a 400 error for malformed payloads. @@ -149,7 +154,7 @@ def my_function(): } ``` -## Built-in models +### Built-in models Parser comes with the following built-in models: @@ -171,7 +176,7 @@ Parser comes with the following built-in models: | **KafkaSelfManagedEventModel** | Lambda Event Source payload for self managed Kafka payload | | **KafkaMskEventModel** | Lambda Event Source payload for AWS MSK payload | -### extending built-in models +#### extending built-in models You can extend them to include your own models, and yet have all other known fields parsed along the way. @@ -236,7 +241,9 @@ for order_item in ret.detail.items: 3. Defined how part of our EventBridge event should look like by overriding `detail` key within our `OrderEventModel` 4. Parser parsed the original event against `OrderEventModel` -## Envelopes +## Advanced + +### Envelopes When trying to parse your payloads wrapped in a known structure, you might encounter the following situations: @@ -294,7 +301,7 @@ def handler(event: UserModel, context: LambdaContext): 3. Parser parsed the original event against the EventBridge model 4. Parser then parsed the `detail` key using `UserModel` -### Built-in envelopes +#### Built-in envelopes Parser comes with the following built-in envelopes, where `Model` in the return section is your given model. @@ -312,7 +319,7 @@ Parser comes with the following built-in envelopes, where `Model` in the return | **LambdaFunctionUrlEnvelope** | 1. Parses data using `LambdaFunctionUrlModel`.
2. Parses `body` key using your model and returns it. | `Model` | | **KafkaEnvelope** | 1. Parses data using `KafkaRecordModel`.
2. Parses `value` key using your model and returns it. | `Model` | -### Bringing your own envelope +#### Bringing your own envelope You can create your own Envelope model and logic by inheriting from `BaseEnvelope`, and implementing the `parse` method. @@ -377,7 +384,7 @@ Here's a snippet of how the EventBridge envelope we demonstrated previously is i 3. Then, we parsed the incoming data with our envelope to confirm it matches EventBridge's structure defined in `EventBridgeModel` 4. Lastly, we call `_parse` from `BaseEnvelope` to parse the data in our envelope (.detail) using the customer model -## Data model validation +### Data model validation ???+ warning This is radically different from the **Validator utility** which validates events against JSON Schema. @@ -394,7 +401,7 @@ Keep the following in mind regardless of which decorator you end up using it: * You must raise either `ValueError`, `TypeError`, or `AssertionError` when value is not compliant * You must return the value(s) itself if compliant -### validating fields +#### validating fields Quick validation to verify whether the field `message` has the value of `hello world`. @@ -439,7 +446,7 @@ class HelloWorldModel(BaseModel): parse(model=HelloWorldModel, event={"message": "hello universe", "sender": "universe"}) ``` -### validating entire model +#### validating entire model `root_validator` can help when you have a complex validation mechanism. For example finding whether data has been omitted, comparing field values, etc. @@ -470,7 +477,7 @@ parse(model=UserModel, event=payload) ???+ info You can read more about validating list items, reusing validators, validating raw inputs, and a lot more in Pydantic's documentation. -## Advanced use cases +### Advanced use cases ???+ tip "Tip: Looking to auto-generate models from JSON, YAML, JSON Schemas, OpenApi, etc?" Use Koudai Aono's [data model code generation tool for Pydantic](https://github.com/koxudaxi/datamodel-code-generator) @@ -535,55 +542,3 @@ If what you're trying to use isn't available as part of the high level import sy ```python title="Pydantic import escape hatch" from aws_lambda_powertools.utilities.parser.pydantic import ``` - -**What is the cold start impact in bringing this additional dependency?** - -No significant cold start impact. It does increase the final uncompressed package by **71M**, when you bring the additional dependency that parser requires. - -Artillery load test sample against a [hello world sample](https://github.com/aws-samples/cookiecutter-aws-sam-python) using Tracer, Metrics, and Logger with and without parser. - -**No parser** - -???+ info - **Uncompressed package size**: 55M, **p99**: 180.3ms - -```javascript -Summary report @ 14:36:07(+0200) 2020-10-23 -Scenarios launched: 10 -Scenarios completed: 10 -Requests completed: 2000 -Mean response/sec: 114.81 -Response time (msec): - min: 54.9 - max: 1684.9 - median: 68 - p95: 109.1 - p99: 180.3 -Scenario counts: - 0: 10 (100%) -Codes: - 200: 2000 -``` - -**With parser** - -???+ info - **Uncompressed package size**: 128M, **p99**: 193.1ms - -```javascript -Summary report @ 14:29:23(+0200) 2020-10-23 -Scenarios launched: 10 -Scenarios completed: 10 -Requests completed: 2000 -Mean response/sec: 111.67 -Response time (msec): - min: 54.3 - max: 1887.2 - median: 66.1 - p95: 113.3 - p99: 193.1 -Scenario counts: - 0: 10 (100%) -Codes: - 200: 2000 -``` diff --git a/docs/utilities/validation.md b/docs/utilities/validation.md index c9cd5813086..1b01c74f1e3 100644 --- a/docs/utilities/validation.md +++ b/docs/utilities/validation.md @@ -18,6 +18,14 @@ This utility provides JSON Schema validation for events and responses, including ???+ tip All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}. +### Install + +!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}" + +Add `aws-lambda-powertools[validation]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_. + +This will ensure you have the required dependencies before using Validation. + You can validate inbound and outbound events using [`validator` decorator](#validator-decorator). You can also use the standalone `validate` function, if you want more control over the validation process such as handling a validation error.