Skip to content

BUG: Fix float formatting when a string is passed as float_format arg #22308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 26, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1395,6 +1395,7 @@ Notice how we now instead output ``np.nan`` itself instead of a stringified form
- Bug in :meth:`read_excel()` in which extraneous header names were extracted, even though none were specified (:issue:`11733`)
- Bug in :meth:`read_excel()` in which ``index_col=None`` was not being respected and parsing index columns anyway (:issue:`20480`)
- Bug in :meth:`read_excel()` in which ``usecols`` was not being validated for proper column names when passed in as a string (:issue:`20480`)
- :func:`DataFrame.to_string()`, :func:`DataFrame.to_html()`, :func:`DataFrame.to_latex()` will correctly format output when a string is passed as the ``float_format`` argument (:issue:`21625`, :issue:`22270`)

Plotting
^^^^^^^^
Expand Down
2 changes: 2 additions & 0 deletions pandas/io/formats/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -960,6 +960,8 @@ def __init__(self, *args, **kwargs):
# float_format is expected to be a string
# formatter should be used to pass a function
if self.float_format is not None and self.formatter is None:
# GH21625, GH22270
self.fixed_width = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you see if u can change fixed_width to a cached property instead of setting it (need to remove from the signature as well)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a good chance I'm not understanding the comment, but I'm not sure we can set this as a cached property (using the cache_readonly decorator?) since it is set in the base class. As I say I might be missing something...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can set it in each of the subclasses. I just don't think we actually need this. try doing this as a property first to see if it works.

if callable(self.float_format):
self.formatter = self.float_format
self.float_format = None
Expand Down
14 changes: 14 additions & 0 deletions pandas/tests/io/formats/data/gh21625_expected_output.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<table border="1" class="dataframe">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't have a very strong opinion atm but wondering if we want dedicated files for tests this small. Could maybe include HTML as part of the test or alternately just test for the existence of the appropriate format in the result or the expression.

This doesn't need to be addressed in this PR but just bringing up as a general discussion point

<thead>
<tr style="text-align: right;">
<th></th>
<th>x</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.200</td>
</tr>
</tbody>
</table>
14 changes: 14 additions & 0 deletions pandas/tests/io/formats/data/gh22270_expected_output.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>x</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>100</td>
</tr>
</tbody>
</table>
12 changes: 12 additions & 0 deletions pandas/tests/io/formats/test_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -1359,6 +1359,18 @@ def test_to_string_float_formatting(self):
'1 2.512000e-01')
assert df_s == expected

def test_to_string_float_format_no_fixed_width(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you would need to add tests for to_latex and to_html in the appropriate files


# GH 21625
df = DataFrame({'x': [0.19999]})
expected = ' x\n0 0.200'
assert df.to_string(float_format='%.3f') == expected

# GH 22270
df = DataFrame({'x': [100.0]})
expected = ' x\n0 100'
assert df.to_string(float_format='%.0f') == expected

def test_to_string_small_float_values(self):
df = DataFrame({'a': [1.5, 1e-17, -5.5e-7]})

Expand Down
12 changes: 12 additions & 0 deletions pandas/tests/io/formats/test_to_html.py
Original file line number Diff line number Diff line change
Expand Up @@ -465,3 +465,15 @@ def test_to_html_with_id(self):
name='myindexname'))
result = df.to_html(index_names=False, table_id="TEST_ID")
assert ' id="TEST_ID"' in result

def test_to_html_float_format_no_fixed_width(self, datapath):

# GH 21625
df = DataFrame({'x': [0.19999]})
expected = expected_html(datapath, 'gh21625_expected_output')
assert df.to_html(float_format='%.3f') == expected

# GH 22270
df = DataFrame({'x': [100.0]})
expected = expected_html(datapath, 'gh22270_expected_output')
assert df.to_html(float_format='%.0f') == expected
26 changes: 26 additions & 0 deletions pandas/tests/io/formats/test_to_latex.py
Original file line number Diff line number Diff line change
Expand Up @@ -708,3 +708,29 @@ def test_to_latex_multiindex_empty_name(self):
\end{tabular}
"""
assert observed == expected

def test_to_latex_float_format_no_fixed_width(self):

# GH 21625
df = DataFrame({'x': [0.19999]})
expected = r"""\begin{tabular}{lr}
\toprule
{} & x \\
\midrule
0 & 0.200 \\
\bottomrule
\end{tabular}
"""
assert df.to_latex(float_format='%.3f') == expected

# GH 22270
df = DataFrame({'x': [100.0]})
expected = r"""\begin{tabular}{lr}
\toprule
{} & x \\
\midrule
0 & 100 \\
\bottomrule
\end{tabular}
"""
assert df.to_latex(float_format='%.0f') == expected