Allow users to configure initial buffer size #37786
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
70fe2aa to
acaa6d4
Compare
bogdandrutu
commented
Feb 7, 2025
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
acaa6d4 to
0ddbb6f
Compare
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
0ddbb6f to
ad85a6a
Compare
djaglowski
requested changes
Feb 10, 2025
This comment was marked as resolved.
This comment was marked as resolved.
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
ad85a6a to
39a098b
Compare
This comment was marked as resolved.
This comment was marked as resolved.
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
39a098b to
74d3f72
Compare
|
@djaglowski PTAL |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Signed-off-by: Bogdan Drutu <[email protected]>
Signed-off-by: Bogdan Drutu <[email protected]>
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
74d3f72 to
3c40e88
Compare
|
@djaglowski thanks for the pointers. All updates are in place. |
djaglowski
approved these changes
Feb 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using the filelogreceiver with a system that does not allow for in-progress file reads (holds a lock on the file while writing) and we want to read the entire file (can think of a big proto binary) then decode and construct logs.
Because of this use-case and because of the files being ~5MB we do a lot of re-size of the scanner buffer until we are able to read the whole file/token.
Proposal is to allow customizing the initial buffer size for use-cases like this.