ENH: simple patch for read_json compression #16750
Conversation
New feature, so it can go in
What's the output of but it may be named differently. |
I tried a few different forms, all failed with the same error, including: |
|
git remote add upstream https://github.com/pandas-dev/pandas should; then
you'll need to `git fetch upstream` before it'll work.
…On Wed, Jun 21, 2017 at 1:37 PM, colinhiggins ***@***.***> wrote:
@TomAugspurger <https://github.com/tomaugspurger>:
What's the output of git remote -v? That should include a couple lines like
$ git remote -v
origin ***@***.***:colinhiggins/pandas.git (fetch)
origin ***@***.***:colinhiggins/pandas.git (push)
I tried a few different forms, all failed with the same error, including:
https://github.com/pandas-dev/pandas
***@***.***:pandas-dev/pandas.git
git://github.com/pandas-dev/pandas
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#16750 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABQHIv9Xdh2gkVItDlLIY8ibza7-MsO0ks5sGWMAgaJpZM4OBSwm>
.
|
|
Ah, right, duh. It ran with no output. Is that a pass? |
colinhiggins
force-pushed
the
json_compression
branch
from
06aafa6 to
9393351
Compare
Codecov Report
@@ Coverage Diff @@
## master #16750 +/- ##
=======================================
Coverage 90.93% 90.93%
=======================================
Files 161 161
Lines 49282 49282
=======================================
Hits 44816 44816
Misses 4466 4466
Continue to review full report at Codecov.
|
|
Reverted back to the |
Addresses GH15644
colinhiggins
force-pushed
the
json_compression
branch
from
9393351 to
12b6012
Compare
|
Changes look good, but we'll need tests for the new behavior. You can compress https://github.com/pandas-dev/pandas/blob/master/pandas/tests/io/json/data/tsframe_iso_v012.json and add it to the repo, and then ensure we have basic coverage on this. We should also have a jsonlines version too probably. |
TomAugspurger
added
IO Data
|
see #13317 for how to do tests on this. This needs similar types of testing. |
| @@ -24,6 +24,7 @@ New features | |||
| <https://www.python.org/dev/peps/pep-0519/>`_ on most readers and writers (:issue:`13823`) | |||
| - Added ``__fspath__`` method to :class:`~pandas.HDFStore`, :class:`~pandas.ExcelFile`, | |||
| and :class:`~pandas.ExcelWriter` to work properly with the file system path protocol (:issue:`13823`) | |||
| - The ``read_json`` method now supports a ``compression`` keyword, which allows you to read compressed json directly. The behavior of this is identical to the ``read_csv`` keyword and defaults to ``infer``. (:issue:`15644`) | |||
There was a problem hiding this comment.
needs a new doc section in io.json (again see how the io.pickle compression section is done).
| @@ -258,6 +258,13 @@ def read_json(path_or_buf=None, orient=None, typ='frame', dtype=True, | |||
|
|
|||
| .. versionadded:: 0.19.0 | |||
|
|
|||
| compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None}, default 'infer' | |||
| For on-the-fly decompression of on-disk data. If 'infer', then use gzip, | |||
There was a problem hiding this comment.
needs version added. also needs to be indendted as this won't render I think.
There was a problem hiding this comment.
see pd.read_pickle doc-string as well.
|
Got it. I can take a crack at the to_json and tests, but it will have to wait for the weekend. |
| compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None}, default 'infer' | ||
| For on-the-fly decompression of on-disk data. If 'infer', then use gzip, | ||
| bz2, zip or xz if filepath_or_buffer is a string ending in '.gz', '.bz2', | ||
| '.zip', or 'xz', respectively, and no decompression otherwise. If using |
|
closing this. it needs comprehensive tests as indicated above. pls comment if you'd like to reopen. |
Addresses GH15644
git diff upstream/master --name-only -- '*.py' | flake8 --diffThe latest comment on #15644 from March suggest just adding compression to read_json. This does that.
Not sure what kind of tests should be added since its using pre-existing code. It works when I tested it manually, and test_fast.sh returned with:
This is my first contribution, and I'm a bit confused about the whatsnew entry. I'm not sure which version the changes should be recorded in.
In that vein,
git diff upstream/master --name-only -- '*.py' | flake8 --difffails with: