[Enhancement]: Implement AWS Bedrock prompt caching

[dvejsada]

What features would you like to see added?

Amazon Bedrock has made a prompt caching feature publicly available see here.

More details

The prompt caching works by adding a cache marker to a part of the prompt that remains static. In case of LibreChat, I believe that this would be beneficial for:

• system prompts - namely system prompts for agents with many tools can get quite long and remain static over each message
• documents added as text - if a user adds documents as text to conversation, this remains static over the conversation
• tool call results (?)

The prompt caching can lead to decrease of API usage costs as cache read input tokens are 10 times cheaper that regular input tokens.

Which components are impacted by your request?

Endpoints

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct