WIP: df rendering using templates + conditional formatting for HTML

[ghost]

closes #3190.

Provides conditional formatting and CSS styling of HTML tables for dataframes using code.
The plan is to clean up and package the functionality behind a nice API in some pandas namespace,
perhaps with a dotted API, and ship in 0.14.

Unlike my first attempt from 6 months ago which turned into a rabbit hole, this is a much simpler approach that came together in a few hours and supports everything I care about.

See the ipython notebook in the root directory, or on nbviewer.

Feedback welcome.

• Related Escape special characters in to_latex() output #5374. need to have format-specific escaping function as part of the process.
• Equivalent functionality to formatters
• Should be able to "freeze" constructed views and use them as functions of other df
  (Example: highlight 3 largest values in "price" column for each frame in a panel).
• Use to define default style for html output (bootstrap-like for visual integration with ipnb).

[ghost]

cc @takluyver

[jtratner]

I like this - should we prefix the classes pandas-row-heading or pandas-row0 to avoid name collisions?

Also I'm assuming the default names will be part of the standard pandas repr?

Finally are you planning to allow users to get at the underlying Jinja rendering object so they can add in filters to use with their template? That might be overkill though.

[ghost]

will do on the class name namespacing.

Not sure yet whether this API should live on the dataframe object or in it's own namespace.
Perhaps Stylist(df).zebra().highlight_col() would allows us to add more methods without
bloating the already obese dataframe method list.

Not sure what you mean by rendering object, but generally speaking sure, if the user wants
to step in between jinja and pandas he should be able to.

Also, still pondering whether this belongs in pandas or it's own package.
It turned out much smaller then I expected, so now I can't decide if it's feature
creep (I thought it would be) or a little bit that goes a long way.

[jtratner]

You can register template functions with Jinja. I thought you were letting people use their own templates? I'm not clear.

[ghost]

Yes, you'll be able to pass in an arbitrary template at construction time. But that means you may
lose the comforts of the styling functions included. Depends on the template.

In the previous attempt, I looked at using mako which supports arbitrary logic in templates,
but then the templates became messy.

The major use case of HTML/css output with conditional formatting is well-covered and with
a very small foortprint so I'm pleased with how that turned out.

[jtratner]

Okay, makes sense - the solution definitely looks good (and I think IPython
notebook already requires jinja, so it's not even that difficult of a dep).

[ghost]

Started arranging code into place, added nb of example usage for dotted API.
Renamed class names and hid them behind constants.
Added basic templates for latex and markdown (just checking) output. works.

		C_l1_g0	C_l1_g1
R_l0_g0	R_l1_g0	R0C0	R0C1
R_l0_g1	R_l1_g1	R1C0	R1C1
R_l0_g2	R_l1_g2	R2C0	R2C1

[cpcloud]

looking forward to this 👏

[ghost]

Another aborted attempt, sorry. Don't know when if ever I'll finish this,
closing rather then letting it rot on the vine.

[edoson]

I have a naive Q.. When I write an HTML table in a markup cell in the notebook, it is being printed just the same as a df.head() table. That made me think that the output of an df.head() operation is an HTML code which is govern by IPython default table styling (perhaps in a CSS file). I thought that in order to change the way df.head() is rendering, all I need is to hack into the IPython CSS file and change the appropriate styling code. Can you explain where I'm wrong? Im curious to know what really happening behind the scenes.. :)

[takluyver]

@edoson I don't think you are wrong. ;-)

When you do df.head(), pandas actually returns another data frame, with just the first few rows of your original. Then, if that's the last line of the cell, sys.displayhook() gets called on it. In the plain Python shell, that would calculate and display its repr. In IPython, it also calculates an HTML repr (you can do other formats like PNG or SVG, but they don't make much sense for a table). If there's an HTML repr, then IPython adds that to the page instead of the plain text repr, so you see a table. It's just part of the DOM, so any CSS rules that match that HTML will affect it.

[jreback]

@edoson

and DataFrame has a _repr_html, see here: http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/notebooks/Custom%20Display%20Logic.ipynb

so a way of doing this would be to override that display method and do exactly what you want, e.g.

def _repr_html(self):
    # return an awesome html repr here

DataFrame._repr_html = _repr_html

[edoson]

@takluyver so to change the appearance of table in IPython notebook, all one needs to do is to override the tables styling of IPython, right? Problem is I dont know where IPython tables styling config is (is it CSS? is it hardcoded?) I have a CSS file with nice styling, I have an HTML file with demo code, but I dont know how to tell IPython to use that style rules.

@jreback Is this an alternative why to using CSS's? because if the _repr_html returns an html code of a table, and IPython renders it by its own styling rules, we are in the same place. Or maybe I code return the HTML code I already have which uses nice table styles and have it refer to my custom.css file? Need some more guidance.. :)

[jreback]

@edoson I am not sure how ipython renders after the HTML is produced. You can pass classes to to_html but not CSS, so not sure how this woudld interact.

[takluyver]

If we have any styling of tables, it's in CSS. You can add CSS to the page by displaying HTML:

from IPython.display import HTML, display
display(HTML('<style type="text/css">' + style_rules + '</style>'))