diff --git a/README.md b/README.md index f089bd2..ba10bc6 100644 --- a/README.md +++ b/README.md @@ -1,59 +1,54 @@ # `greater_tables` Project -![GitHub commit activity](https://img.shields.io/github/commit-activity/y/mynl/greater_tables_project) - - -## TODO - -* Ratio cols with multi index columns -* % in tex output - never allow comments? -* center / left / right table output -> CSS -* ?Option to hide index -* Bring over the roll your own logger - - -## Greater Tables - -Display graphics vs. data tables - -no colors, sparklines, shading, ... - -Creating presentation quality tables is difficult. It is hard to left-align text and right-align numbers using pandas `display` or `df.to_html`. The `great_tables` package does a really nice job with pandas and polars dataframes but does not support indexes or TeX output. - -This package provides consistent HTML and TeX table output with flexible type-based formatting, and table rules. Neither output relies on the pandas `to_html` or `to_latex` functions. TeX output uses Tikz tables for very tight control over layout and grid lines. The package is designed for use in Jupyter Lab notebooks Quarto documents. - -Usage: the main class `GT` should be subclassed to set appropriate defaults for your project. `sGT` provides an example. - -The project is currently in **beta** status. HTML output is better developed than TeX. - -## The Name - -Obviously, the name is a play on the `great_tables` package. But, I have -been maintaining a set of macros called -[GREATools](https://www.mynl.com/old/GREAT/home.html) (generalized, -reusable, extensible actuarial tools) in VBA and Python since the late -1990s, and call all my macro packages "GREAT". - -## Documentation - +![](https://img.shields.io/github/commit-activity/y/mynl/greater_tables_project) +![](https://img.shields.io/pypi/format/greater_tables) ![](https://img.shields.io/readthedocs/greater_tables_project) -Available on -[readthedocs](https://greater-tables-project.readthedocs.io/en/latest). + +## Greater Tables + +Creating presentation quality tables is difficult. `greater_tables` provides +a flexible way to create consistent tables in HTML, LaTeX (PDF), and terminal +text outputs from Pandas dataframes. +It has many options but sensible defaults. It is designed +for use in Jupyter Lab and Quarto and will seamlessly return the correct format +for each output type. The basic usage is simply: + +```python +from greater_tables import GT +# ...create dataframe df... +GT(df) +``` + +or `display(GT(df))` if called within a Jupyter or Quarto code block. Once created `GT(df)` is immutable; to change options re-create. Presentation tables are small! They fit on one or two pages and, while `GT` does a lot of work to determine formating options, it still runs very quickly. + +`greater_tables` provides similar functionality to pandas `to_html`, `to_latex` and `to_markdown` +methods, without relying on them, and improves them in various ways. LaTeX output uses Tikz tables for very tight control over layout and grid lines. Arguments can be passed directly or set via a YAML configuration file. Validation is handled by `pydantic`. + +The package is tailored to more austere, black-and-white tables: no sparklines, colors or background shading. Tables can include a simple caption, but not more elaborate headers and footers. ## Installation -![](https://img.shields.io/pypi/format/greater_tables) - ```python pip install greater-tables ``` +## Documentation + +[ReadtheDocs](https://greater-tables-project.readthedocs.io/en/latest). + +## Source + +[GitHub](https://www.github.com/mynl/greater_tables_project). + +## Licence + +MIT. + ## Examples -The following example shows quite a hard table. It is formatted using -the `sGT` class, which is a subclass of `GT` with a few defaults set. +The following example shows a tricky hard table. ```python import pandas as pd @@ -63,49 +58,58 @@ level_1 = ["Group A", "Group A", "Group B", "Group B", 'Group C'] level_2 = ['Sub 1', 'Sub 2', 'Sub 2', 'Sub 3', 'Sub 3'] multi_index = pd.MultiIndex.from_arrays([level_1, level_2]) - -start = pd.Timestamp.today().normalize() # Today's date, normalized to midnight +start = pd.Timestamp.today().normalize() end = pd.Timestamp(f"{start.year}-12-31") # End of the year - -hard = pd.DataFrame( -{'x': np.arange(2020, 2025, dtype=int), +df = pd.DataFrame( +{'year': np.arange(2020, 2025, dtype=int), 'a': np.array((100, 105, 2000, 2025, 100000), dtype=int), 'b': 10. ** np.linspace(-9, 9, 5), 'c': np.linspace(601, 4000, 5), 'd': pd.date_range(start=start, end=end, periods=5), 'e': 'once upon a time, risk is hard to define, not in Kansas anymore, neutrinos are hard to detect, $\\int_\\infty^\\infty e^{-x^2/2}dx$ is a hard integral'.split(',') -}).set_index('x') -hard.columns = multi_index -sGT(hard, 'A hard table.') +}).set_index('year') +df.columns = multi_index +gtc.GT(df, caption='A simple GT table.', + year_cols='year', + vrule_widths=(1,.5, 0)) ``` -![HTML output.](img/hard-html.png) - -![TeX output.](img/hard-tex.png) +![](docs/img/simple-example.png) The output illustrates: -- Quarto or Jupyter automatically the class's `_repr_html_` method (or +- Quarto or Jupyter automatically calls the class's `_repr_html_` method (or `_repr_latex_` for pdf/TeX/Beamer output), providing seamless - integration across different output formats. -- Text is left-aligned, numbers are right-aligned. -- The index is displayed, was detected as likely years, and formatted - without a comma separator. -- The first column of integers does have a comma thousands separator. + integration across different output formats. `print()` produces fixed-pitch text output. +- Text is left-aligned, numbers are right-aligned, and dates are centered. +- The index is displayed, and formatted without a comma separator, being specified in `year_cols`. Columns specified in `ratio_col` use % formatting. Explicit control provided over all columns; these are just helpers. +- The first column of integers with a comma thousands separator and no decimals. - The second column of floats spans several orders of magnitude and is - formatted using Engineering format, n for nano through G for giga. + formatted using Engineering format, n for nano through k for kilo. - The third column of floats is formatted with a comma separator and two decimals, based on the average absolute value. -- The fourth column of date times is formatted as ISO standard dates - (not date times). -- The vertical lines separate the levels of the column multiindex. The - subgroups are a little tricky. +- The fourth column of date times is formatted as ISO standard dates. +- Text, in the last column, is sensibly wrapped and can include TeX. +- The vertical lines separate the levels of the column multiindex. -More coming soon. + + +## The Name + +Obviously, the name is a play on the `great_tables` package. I have +been maintaining a set of macros called +[GREATools](https://www.mynl.com/old/GREAT/home.html) (generalized, +reusable, extensible actuarial tools) in VBA and Python since the late +1990s, and call all my macro packages *GREAT*. ## History +3.3.0 +------- +* Added `tikz_` series of options to config: column and row separation, + container_env (for e.g., sidewaystable), hrule and vrule indices. + 3.2.0 ------- * Added more tex snippets! @@ -134,6 +138,7 @@ More coming soon. * Better column widths * Custom text output * Rich table output + 1.1.1 ------- * Added logo, updated docs. @@ -147,7 +152,6 @@ More coming soon. * Added ``caption_align='center'`` argument to set the caption alignment * Added ``large_ok=False`` argument, if ``False`` providing a dataframe with more than 100 rows throws an error. This function is expensive and is designed for small frames. - 1.0.0 ------ @@ -159,7 +163,6 @@ More coming soon. * Docs updated * Set tabs for width; use of width in HTML format. - 0.6.0 ------ @@ -172,10 +175,6 @@ Early development * tikz code from great.pres_manager - - - - ## 📁 Project Layout ``` @@ -222,17 +221,3 @@ greater_tables_project/ | hard-html.png | hard-tex.png ``` - - - -## 🧠 Design Principles You’re Following - -| Principle | Your Approach | -| ---------------------------- | ------------------------------------------- | -| Immutability | `GT(df, config)` is fixed once created | -| Separation of concerns | `GTConfigModel` holds defaults/types | -| Config as code/documentation | `config_template.yaml` generated from model | -| CLI-first mindset | `click` used to expose functionality | -| Linear Git workflow | Tags for rollback, no branches | - - diff --git a/docs/img/simple-example.png b/docs/img/simple-example.png new file mode 100644 index 0000000..1baabe6 Binary files /dev/null and b/docs/img/simple-example.png differ diff --git a/docs/index.rst b/docs/index.rst index ade6926..d9b2da2 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -4,7 +4,128 @@ contain the root `toctree` directive. Welcome to greater_tables's documentation! -========================================== +============================================ + +|image1| |image2| |image3| + +Greater Tables +-------------- + +Creating presentation quality tables is difficult. ``greater_tables`` +provides a flexible way to create consistent tables in HTML, LaTeX +(PDF), and terminal text outputs from Pandas dataframes. It has many +options but sensible defaults. It is designed for use in Jupyter Lab and +Quarto and will seamlessly return the correct format for each output +type. The basic usage is simply: + +.. code:: python + + from greater_tables import GT + # ...create dataframe df... + GT(df) + +or ``display(GT(df))`` if called within a Jupyter or Quarto code block. +Once created ``GT(df)`` is immutable; to change options re-create. +Presentation tables are small! They fit on one or two pages and, while +``GT`` does a lot of work to determine formating options, it still runs +very quickly. + +``greater_tables`` provides similar functionality to pandas ``to_html``, +``to_latex`` and ``to_markdown`` methods, without relying on them, and +improves them in various ways. LaTeX output uses Tikz tables for very +tight control over layout and grid lines. Arguments can be passed +directly or set via a YAML configuration file. Validation is handled by +``pydantic``. + +The package is tailored to more austere, black-and-white tables: no +sparklines, colors or background shading. Tables can include a simple +caption, but not more elaborate headers and footers. + +Installation +------------ + +.. code:: python + + pip install greater-tables + +Documentation +------------- + +`ReadtheDocs `__. + +Source +------ + +`GitHub `__. + +Licence +------- + +MIT. + +Examples +-------- + +The following example shows a tricky hard table. + +.. code:: python + + import pandas as pd + import numpy as np + from greater_tables import sGT + level_1 = ["Group A", "Group A", "Group B", "Group B", 'Group C'] + level_2 = ['Sub 1', 'Sub 2', 'Sub 2', 'Sub 3', 'Sub 3'] + + multi_index = pd.MultiIndex.from_arrays([level_1, level_2]) + start = pd.Timestamp.today().normalize() + end = pd.Timestamp(f"{start.year}-12-31") # End of the year + df = pd.DataFrame( + {'year': np.arange(2020, 2025, dtype=int), + 'a': np.array((100, 105, 2000, 2025, 100000), dtype=int), + 'b': 10. ** np.linspace(-9, 9, 5), + 'c': np.linspace(601, 4000, 5), + 'd': pd.date_range(start=start, end=end, periods=5), + 'e': 'once upon a time, risk is hard to define, not in Kansas anymore, neutrinos are hard to detect, $\\int_\\infty^\\infty e^{-x^2/2}dx$ is a hard integral'.split(',') + }).set_index('year') + df.columns = multi_index + gtc.GT(df, caption='A simple GT table.', + year_cols='year', + vrule_widths=(1,.5, 0)) + +.. figure:: img/simple-example.png + + +The output illustrates: + +- Quarto or Jupyter automatically calls the class’s ``_repr_html_`` + method (or ``_repr_latex_`` for pdf/TeX/Beamer output), providing + seamless integration across different output formats. ``print()`` + produces fixed-pitch text output. +- Text is left-aligned, numbers are right-aligned, and dates are + centered. +- The index is displayed, and formatted without a comma separator, + being specified in ``year_cols``. Columns specified in ``ratio_col`` + use % formatting. Explicit control provided over all columns; these + are just helpers. +- The first column of integers with a comma thousands separator and no + decimals. +- The second column of floats spans several orders of magnitude and is + formatted using Engineering format, n for nano through k for kilo. +- The third column of floats is formatted with a comma separator and + two decimals, based on the average absolute value. +- The fourth column of date times is formatted as ISO standard dates. +- Text, in the last column, is sensibly wrapped and can include TeX. +- The vertical lines separate the levels of the column multiindex. + +The Name +-------- + +Obviously, the name is a play on the ``great_tables`` package. I have +been maintaining a set of macros called +`GREATools `__ (generalized, +reusable, extensible actuarial tools) in VBA and Python since the late +1990s, and call all my macro packages *GREAT*. + .. toctree:: :maxdepth: 2 @@ -14,40 +135,17 @@ Welcome to greater_tables's documentation! greater_tables.data -.. include:: ../README.md - :parser: myst_parser.sphinx_ - -Other -======= - -Auto doc files generated with:: - - sphinx-apidoc -o . ..\greater_tables\ - -File layout: - - C:\S\TELOS\PYTHON\GREATER_TABLES_PROJECT\GREATER_TABLES - | cli.py - | gtconfig.py - | gtcore.py - | gtenums.py - | gtformats.py - | hasher.py - | testdf.py - | tex_svg.py - | __init__.py - | - +---data - | | tex_list.csv - | | tex_list.py - | | words-12.md - | | __init__.py - - - Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` + + + + +.. |image1| image:: https://img.shields.io/github/commit-activity/y/mynl/greater_tables_project +.. |image2| image:: https://img.shields.io/pypi/format/greater_tables +.. |image3| image:: https://img.shields.io/readthedocs/greater_tables_project +.. |image4| image:: docs/img/simple-example.png diff --git a/greater_tables/__init__.py b/greater_tables/__init__.py index 907ee4c..334d15e 100644 --- a/greater_tables/__init__.py +++ b/greater_tables/__init__.py @@ -1,4 +1,4 @@ -__version__ = '3.2.0' +__version__ = '3.3.0' __project__ = 'greater_tables' __author__ = 'Stephen J Mildenhall' diff --git a/greater_tables/gtconfig.py b/greater_tables/gtconfig.py index 034f1ff..ec4d704 100644 --- a/greater_tables/gtconfig.py +++ b/greater_tables/gtconfig.py @@ -35,8 +35,11 @@ class GTConfigModel(BaseModel): :see also: ``GTConfig`` for loading from YAML with overrides. ``gt write-template`` CLI command to generate a default config file. """ - # immutable - model_config = ConfigDict(frozen=True) + model_config = ConfigDict( + # make model immutable (no attribute reassignment) + frozen=True, + extra="forbid" # raise error on unexpected/extra fields + ) default_integer_str: str = Field( "{x:,d}", description="Format f-string for integers. Example: '{x:,d}'" ) @@ -125,10 +128,6 @@ class GTConfigModel(BaseModel): caption_align: str = Field( "center", description="Alignment of the caption text" ) - large_ok: bool = Field( - False, description="If True, allow full rendering of large tables without truncation" - ) - max_str_length: int = Field( -1, description="Maximum length for stringified objects (e.g. nested DataFrames); -1 = unlimited" ) @@ -152,7 +151,40 @@ class GTConfigModel(BaseModel): table_width_header_relax: float = Field( 10.0, description="Extra characters allowed per column heading to help header wrapping" ) - debug: bool = Field(False, description="Run in debug mode with more reporting, include internal ID in caption and use colored output lines") + + # tikz specific options + tikz_column_sep: float = Field( + 0.5, description="Separation between columns") + tikz_row_sep: float = Field( + 0.125, description="Separation between rows") + tikz_container_env: Literal["table", "figure", "sidewaysfigure"] = Field( + default="table", + description="Type of element: 'table', 'figure', or 'sidewaysfigure'" + ) + tikz_extra_defs: str = Field( + '', description="TeX defintions and commands put at top of table, eg \\centering.") + tikz_hrule: Optional[list[int]] = Field( + default=None, + description="Optional, list of (0-based) integers for horizontal rules below each value; None means no lines." + ) + tikz_vrule: Optional[list[int]] = Field( + default=None, + description="Optional, list of integers for vertical rules right of each value; None means no lines." + ) + tikz_post_process: str = Field( + '', description="non-line commands put at bottom of table") + tikz_latex: Optional[str] = Field( + None, description="arguments at top of table \\begin{table}[tikz_latex]") + + # meta + debug: bool = Field( + False, description="Run in debug mode with more reporting, include internal ID in caption and use colored output lines") + large_ok: bool = Field( + False, description="If True, allow full rendering of large tables without truncation" + ) + large_warning: int = Field( + 50, description="Warn for dataframes longer then large_warning unless large_ok==True" + ) def write_template(self, path: Path): """Generate a clean default config file at the given path.""" diff --git a/greater_tables/gtcore.py b/greater_tables/gtcore.py index 8404aca..ee08fa0 100644 --- a/greater_tables/gtcore.py +++ b/greater_tables/gtcore.py @@ -264,7 +264,8 @@ class GT(object): # access through config # update and validate; need to merge to avoid repeated args - merged = dict(base_config.model_dump(), **overrides) + # merged = dict(base_config.model_dump(), **overrides) + merged = base_config.model_dump() | overrides self.config = GTConfigModel(**merged) # no validation # self.config = base_config.model_copy(update=overrides) @@ -298,7 +299,7 @@ class GT(object): raise ValueError( 'df must be a DataFrame, a list of lists, or a markdown table string') - if len(df) > 50 and not config.large_ok: + if len(df) > self.config.large_warning and not config.large_ok: raise ValueError( 'Large dataframe (>50 rows) and config.large_ok not set to true...do you know what you are doing?') @@ -657,8 +658,21 @@ class GT(object): return self._clean_tex def cols_from_regex(self, regex): - """Return columns of self.df matching regex""" - return [col for col in self.df.columns if isinstance(col, str) and re.search(regex, col)] + """ + Return columns matching a regex. + + For Index and MultiIndex. Operates on ``self.df`` and includes + index (if ``show_index``) and columns of input dataframe. Search + applies to any level of the index. Case sensitive. + """ + pattern = re.compile(regex) + matching_cols = [ + col for col in self.df.columns + if any(pattern.search(str(level)) + for level in (col if isinstance(col, tuple) else (col,))) + ] + return matching_cols + # return [col for col in self.df.columns if isinstance(col, str) and re.search(regex, col)] def cache_get(self, key): """Retrieve item from cache.""" @@ -977,6 +991,7 @@ class GT(object): self._column_width_df = self.make_column_width_df() tikz_colw, tabs, scaled_tabs = self.estimate_column_widths() self._column_width_df['tikz_colw'] = tikz_colw + self._column_width_df['tikz_colw'] += 2 # for \I spacer! self._column_width_df['estimated_tabs'] = tabs self._column_width_df['estimated_scaled_tabs'] = scaled_tabs if self.tabs is not None: @@ -1535,6 +1550,7 @@ class GT(object): # with tex adjustment tex_colw = dict.fromkeys(df.columns, 0) headw = dict.fromkeys(df.columns, 0) + tikz_headw = dict.fromkeys(df.columns, 0) tabs = [] scaled_tabs = [] mxmn = {} @@ -1544,12 +1560,15 @@ class GT(object): for i, c in enumerate(df.columns): # figure width of the column labels; if index c= str, if MI then c = tuple # cw is the width of the column header/title + # tzcw is for tikz - no wrapping and no tex adjustment if type(c) == str: if i < nl: cw = GT.text_display_len(c) + tzcw = len(c) else: # for data columns look at words rather than whole phrase cw = max(map(GT.text_display_len, c.split(' '))) + tzcw = len(c) # logger.info(f'leng col = {len(c)}, longest word = {cw}') else: # column name could be float etc. or if multi index a tuple @@ -1559,14 +1578,18 @@ class GT(object): words = ' '.join(c).split(' ') cw = max( map(lambda x: GT.text_display_len(str(x)), words)) + tzcw = max(map(len, words)) else: cw = max(map(lambda x: GT.text_display_len(str(x)), c)) + tzcw = max(map(len, c)) # print(f'{c}: {cw=} no error') except TypeError: # not a MI, float or something cw = GT.text_display_len(str(c)) + tzcw = len(str(c)) # print(f'{c}: {cw=} WITH error') headw[c] = cw + tikz_headw[c] = tzcw # now figure the width of the elements in the column # mxmn is used to determine whether to center the column (if all the same size) if df.dtypes.iloc[i] == object: @@ -1591,6 +1614,9 @@ class GT(object): mxmn[c] = (lens.max(), lens.min()) raw_lens = df.iloc[:, i].map(len) tikz_colw[c] = raw_lens.max() + # pick up long headers too + for c in df.columns: + tikz_colw[c] = max(tikz_colw[c], tikz_headw[c]) # print(tikz_colw) # now know all column widths...decide what to do # are all the data columns about the same width? @@ -2008,16 +2034,15 @@ class GT(object): return self._clean_html def make_tikz(self, - column_sep=4 / 8, # was 3/8 - row_sep=1 / 8, - container_env='table', - extra_defs='', - hrule=None, - vrule=None, - post_process='', - label='', - latex=None, - sparsify=1): + # column_sep=4 / 8, # was 3/8 + # row_sep=1 / 8, + # container_env='table', + # extra_defs='', + # hrule=None, + # vrule=None, + # post_process='', + # latex=None, + ): """ Write DataFrame to custom tikz matrix. @@ -2050,8 +2075,6 @@ class GT(object): never be a rule to the far right...it looks plebby; remember you must include the index columns! - config.sparsify number of cols of multi index to config.sparsify - Issue: column with floats and spaces or missing causes problems (VaR, TVaR, EPD, mean and CV table) @@ -2071,7 +2094,7 @@ class GT(object): lines lines below these rows, -1 for next to last row etc.; list of ints post_process e.g., non-line commands put at bottom of table - latex arguments after \begin{table}[latex] + latex arguments after \\begin{table}[latex] caption text for caption Previous version see great.pres_maker @@ -2085,6 +2108,16 @@ class GT(object): :param label: :return: """ + # pull out arguments (convert to local vars - these used to be arguments) + column_sep = self.config.tikz_column_sep + row_sep = self.config.tikz_row_sep + container_env = self.config.tikz_container_env + extra_defs = self.config.tikz_extra_defs + hrule = self.config.tikz_hrule + vrule = self.config.tikz_vrule + post_process = self.config.tikz_post_process + latex = self.config.tikz_latex + # local variable - with all formatters already applied df = self.df.copy() # self.apply_formatters(self.raw_df.copy(), mode='raw') caption = self.caption @@ -2217,8 +2250,16 @@ class GT(object): f'\trow {i}/.style={{nodes={{text=black, anchor=north, inner ysep=0, text height=0, text depth=0}}}},\n') for i in range(2, nr_columns + 2): sio.write( - f'\trow {i}/.style={{nodes={{text=black, anchor=south, inner ysep=.2em, minimum height=1.3em, font=\\bfseries}}}},\n') + f'\trow {i}/.style={{nodes={{text=black, anchor=south, inner ysep=.2em, minimum height=1.3em, font=\\bfseries, align=center}}}},\n') + # override for index columns headers + # probably ony need for the bottom row with a multiindex? + for i in range(2, nr_columns + 2): + for j in range(1, 1+nc_index): + sio.write( + f'\trow {i} column {j}/.style=' + '{nodes={font=\\bfseries\\itshape, align=left}},\n' + ) # write column spec for i, w, al in zip(range(1, len(align) + 1), tabs, align): # average char is only 0.48 of M @@ -2235,7 +2276,7 @@ class GT(object): f'nodes={{align={ad[al]:<6s}}}, nosep, text width={max(2, 0.6 * w):.2f}em}},\n') # extra col to right which enforces row height sio.write( - f'\tcolumn {i+1:>2d}/.style={{text height=0.9em, text depth=0.2em, nosep, text width=0em}}') + f'\tcolumn {i+1:>2d}/.style={{text height=0.9em, text depth=0.2em, nosep, text width=0em}}\n') sio.write('\t}]\n') sio.write("\\matrix ({matrix_name}) [table, ampersand replacement=\\&]{{\n".format( @@ -2262,7 +2303,7 @@ class GT(object): # c = wfloat_format(c) s = f'{nl} {{cell:{ad2[al]}{colw[cn]}s}} ' nl = '\\&' - sio.write(s.format(cell=c + '\\grtspacer')) + sio.write(s.format(cell=c + '\\I')) # include the blank extra last column sio.write('\\& \\\\\n') else: @@ -2271,7 +2312,7 @@ class GT(object): # c = wfloat_format(c) s = f'{nl} {{cell:{ad2[al]}{colw[c]}s}} ' nl = '\\&' - sio.write(s.format(cell=c + '\\grtspacer')) + sio.write(s.format(cell=c + '\\I')) sio.write('\\& \\\\\n') # write table entries @@ -2380,389 +2421,6 @@ class GT(object): return sio.getvalue() -# def make_tikz_original(self, -# column_sep=4 / 8, # was 3/8 -# row_sep=1 / 8, -# container_env='table', -# extra_defs='', -# hrule=None, -# vrule=None, -# post_process='', -# label='', -# latex=None, -# sparsify=1): -# """ -# Write DataFrame to custom tikz matrix to allow greater control of -# formatting and insertion of horizontal and vertical divider lines - -# Estimates tabs from text width of fields (not so great if includes -# a lot of TeX macros) with a manual override available. Tabs gives -# the widths of each field in em (width of M) - -# Standard row height = 1.5em seems to work - set in meta. - -# first and last thick rules -# others below (Python, zero-based) row number, excluding title row - -# keyword arguments : value (no newlines in value) escape back slashes! -# ``#keyword...`` rows ignored -# passed in as a string to facilitate using them with %%pmt? - -# **Rules** - -# * hrule at i means below row i of the table. (1-based) Top, bottom and -# below index lines are inserted automatically. Top and bottom lines -# are thicker. -# * vrule at i means to the left of table column i (1-based); there will -# never be a rule to the far right...it looks plebby; remember you must -# include the index columns! - -# config.sparsify number of cols of multi index to config.sparsify - -# Issue: column with floats and spaces or missing causes problems (VaR, -# TVaR, EPD, mean and CV table) - -# From great.pres_maker.df_to_tikz - -# keyword args: - -# scale picks up self.config.tikz_scale; scale applied to whole -# table - default 0.717 -# height row height, rec. 1 (em) -# column_sep col sep in em -# row_sep row sep in em -# container_env table, figure or sidewaysfigure -# color color for text boxes (helps config.debugging) -# extra_defs TeX defintions and commands put at top of table, -# e.g., \\centering -# lines lines below these rows, -1 for next to last row -# etc.; list of ints -# post_process e.g., non-line commands put at bottom of table -# latex arguments after \begin{table}[latex] -# caption text for caption - -# Previous version see great.pres_maker -# Original version see: C:\\S\\TELOS\\CAS\\AR_Min_Bias\\cvs_to_md.py - -# :param column_sep: -# :param row_sep: -# :param figure: -# :param extra_defs: -# :param post_process: -# :param label: -# :return: -# """ -# # local variable - with all formatters already applied -# df = self.apply_formatters(self.raw_df.copy(), mode='raw') -# caption = self.caption -# label = self.label -# # prepare label and caption -# if label == '': -# lt = '' -# label = '' -# else: -# lt = label -# label = f'\\label{{{label}}}' -# if caption == '': -# if lt != '': -# logger.info( -# f'You have a label but no caption; the label {label} will be ignored.') -# caption = '% caption placeholder' -# else: -# caption = f'\\caption{{{self.caption}}}\n{label}' - -# if not df.columns.is_unique: -# # possible index/body column interaction -# raise ValueError('tikz routine requires unique column names') -# # {extra_defs} -# # centering handled by quarto -# header = """ -# \\begin{{{container_env}}}{latex} -# {caption} -# \\centering{{ -# \\begin{{tikzpicture}}[ -# auto, -# transform shape, -# nosep/.style={{inner sep=0}}, -# table/.style={{ -# matrix of nodes, -# row sep={row_sep}em, -# column sep={column_sep}em, -# nodes in empty cells, -# nodes={{rectangle, scale={scale}, text badly ragged {debug}}}, -# """ -# # put draw=blue!10 or so in nodes to see the node - -# footer = """ -# {post_process} - -# \\end{{tikzpicture}} -# }} % close centering -# \\end{{{container_env}}} -# """ - -# # always a good idea to do this...need to deal with underscores, % -# # and it handles index types that are not strings -# df = GT.clean_index(df) -# if not np.all([i == 'object' for i in df.dtypes]) and not df.empty: -# logger.warning('cols of df not all objects (expect all obs at this ' -# 'point): ', df.dtypes, sep='\n') -# # make sure percents are escaped, but not if already escaped -# df = df.replace(r"(?', 'c': '^'} -# # use df_aligners, at this point the index has been reset -# align = [] -# for n, i in zip(df.columns, self.df_aligners): -# if i == 'grt-left': -# align.append('l') -# elif i == 'grt-right': -# align.append('r') -# elif i == 'grt-center': -# align.append('c') -# else: -# align.append('l') - -# # start writing -# sio = StringIO() -# if latex is None: -# latex = '' -# else: -# latex = f'[{latex}]' -# debug = '' -# if self.config.debug: -# # color all boxes -# debug = ', draw=blue!10' -# else: -# debug = '' -# sio.write(header.format(container_env=container_env, -# caption=caption, -# extra_defs=extra_defs, -# scale=self.config.tikz_scale, -# column_sep=column_sep, -# row_sep=row_sep, -# latex=latex, -# debug=debug)) - -# # table header -# # title rows, start with the empty spacer row -# i = 1 -# sio.write( -# f'\trow {i}/.style={{nodes={{text=black, anchor=north, inner ysep=0, text height=0, text depth=0}}}},\n') -# for i in range(2, nr_columns + 2): -# sio.write( -# f'\trow {i}/.style={{nodes={{text=black, anchor=south, inner ysep=.2em, minimum height=1.3em, font=\\bfseries}}}},\n') - -# # write column spec -# for i, w, al in zip(range(1, len(align) + 1), tabs, align): -# # average char is only 0.48 of M -# # https://en.wikipedia.org/wiki/Em_(gtypography) -# if i == 1: -# # first column sets row height for entire row -# sio.write(f'\tcolumn {i:>2d}/.style={{' -# f'nodes={{align={ad[al]:<6s}}}, ' -# 'text height=0.9em, text depth=0.2em, ' -# f'inner xsep={column_sep}em, inner ysep=0, ' -# f'text width={max(2, 0.6 * w):.2f}em}},\n') -# else: -# sio.write(f'\tcolumn {i:>2d}/.style={{' -# f'nodes={{align={ad[al]:<6s}}}, nosep, text width={max(2, 0.6 * w):.2f}em}},\n') -# # extra col to right which enforces row height -# sio.write( -# f'\tcolumn {i+1:>2d}/.style={{text height=0.9em, text depth=0.2em, nosep, text width=0em}}') -# sio.write('\t}]\n') - -# sio.write("\\matrix ({matrix_name}) [table, ampersand replacement=\\&]{{\n".format( -# matrix_name=matrix_name)) - -# # body of table, starting with the column headers -# # spacer row -# nl = '' -# for cn, al in zip(df.columns, align): -# s = f'{nl} {{cell:{ad2[al]}{colw[cn]}s}} ' -# nl = '\\&' -# sio.write(s.format(cell=' ')) -# # include the blank extra last column -# sio.write('\\& \\\\\n') -# # write header rows (again, issues with multi index) -# mi_vrules = {} -# sparse_columns = {} -# if isinstance(df.columns, pd.MultiIndex): -# for lvl in range(len(df.columns.levels)): -# nl = '' -# sparse_columns[lvl], mi_vrules[lvl] = GT.sparsify_mi(df.columns.get_level_values(lvl), -# lvl == len(df.columns.levels) - 1) -# for cn, c, al in zip(df.columns, sparse_columns[lvl], align): -# # c = wfloat_format(c) -# s = f'{nl} {{cell:{ad2[al]}{colw[cn]}s}} ' -# nl = '\\&' -# sio.write(s.format(cell=c + '\\grtspacer')) -# # include the blank extra last column -# sio.write('\\& \\\\\n') -# else: -# nl = '' -# for c, al in zip(df.columns, align): -# # c = wfloat_format(c) -# s = f'{nl} {{cell:{ad2[al]}{colw[c]}s}} ' -# nl = '\\&' -# sio.write(s.format(cell=c + '\\grtspacer')) -# sio.write('\\& \\\\\n') - -# # write table entries -# for idx, row in df.iterrows(): -# nl = '' -# for c, cell, al in zip(df.columns, row, align): -# # cell = wfloat_format(cell) -# s = f'{nl} {{cell:{ad2[al]}{colw[c]}s}} ' -# nl = '\\&' -# sio.write(s.format(cell=cell)) -# # if c=='p': -# # print('COLp', cell, type(cell), s, s.format(cell=cell)) -# sio.write('\\& \\\\\n') -# sio.write(f'}};\n\n') - -# # decorations and post processing - horizontal and vertical lines -# nr, nc = df.shape -# # add for the index and the last row plus 1 for the added spacer row at the top -# nr += nr_columns + 1 -# # always include top and bottom -# # you input a table row number and get a line below it; it is implemented as a line ABOVE the next row -# # function to convert row numbers to TeX table format (edge case on last row -1 is nr and is caught, -2 -# # is below second to last row = above last row) -# # shift down extra 1 for the spacer row at the top -# def python_2_tex(x): return x + nr_columns + \ -# 2 if x >= 0 else nr + x + 3 -# tb_rules = [nr_columns + 1, python_2_tex(-1)] -# if hrule: -# hrule = set(map(python_2_tex, hrule)).union(tb_rules) -# else: -# hrule = list(tb_rules) -# logger.debug(f'hlines: {hrule}') - -# # why -# yshift = row_sep / 2 -# xshift = -column_sep / 2 -# descender_proportion = 0.25 - -# # top rule is special -# ls = 'thick' -# ln = 1 -# sio.write( -# f'\\path[draw, {ls}] ({matrix_name}-{ln}-1.south west) -- ({matrix_name}-{ln}-{nc+1}.south east);\n') - -# for ln in hrule: -# ls = 'thick' if ln == nr + nr_columns + \ -# 1 else ('semithick' if ln == 1 + nr_columns else 'very thin') -# if ln < nr: -# # line above TeX row ln+1 that exists -# sio.write(f'\\path[draw, {ls}] ([yshift={-yshift}em]{matrix_name}-{ln}-1.south west) -- ' -# f'([yshift={-yshift}em]{matrix_name}-{ln}-{nc+1}.south east);\n') -# else: -# # line above row below bottom = line below last row -# # descenders are 200 to 300 below baseline -# ln = nr -# sio.write(f'\\path[draw, thick] ([yshift={-descender_proportion-yshift}em]{matrix_name}-{ln}-1.base west) -- ' -# f'([yshift={-descender_proportion-yshift}em]{matrix_name}-{ln}-{nc+1}.base east);\n') - -# # if multi index put in lines within the index TODO make this better! -# if nr_columns > 1: -# for ln in range(2, nr_columns + 1): -# sio.write(f'\\path[draw, very thin] ([xshift={xshift}em, yshift={-yshift}em]' -# f'{matrix_name}-{ln}-{nc_index+1}.south west) -- ' -# f'([yshift={-yshift}em]{matrix_name}-{ln}-{nc+1}.south east);\n') - -# written = set(range(1, nc_index + 1)) -# if vrule and self.show_index: -# # to left of col, 1 based, includes index -# # write these first -# # TODO fix madness vrule is to the left, mi_vrules are to the right... -# ls = 'very thin' -# for cn in vrule: -# if cn not in written: -# sio.write(f'\\path[draw, {ls}] ([xshift={xshift}em]{matrix_name}-1-{cn}.south west) -- ' -# f'([yshift={-descender_proportion-yshift}em, xshift={xshift}em]{matrix_name}-{nr}-{cn}.base west);\n') -# written.add(cn - 1) - -# if len(mi_vrules) > 0: -# logger.debug( -# f'Generated vlines {mi_vrules}; already written {written}') -# # vertical rules for the multi index -# # these go to the RIGHT of the relevant column and reflect the index columns already -# # mi_vrules = {level of index: [list of vrule columns] -# # written keeps track of which vrules have been done already; start by cutting out the index columns -# ls = 'ultra thin' -# for k, cols in mi_vrules.items(): -# # don't write the lowest level -# if k == len(mi_vrules) - 1: -# break -# for cn in cols: -# if cn in written: -# pass -# else: -# written.add(cn) -# top = k + 1 -# if top == 0: -# sio.write(f'\\path[draw, {ls}] ([xshift={-xshift}em]{matrix_name}-{top}-{cn}.south east) -- ' -# f'([yshift={-descender_proportion-yshift}em, xshift={-xshift}em]{matrix_name}-{nr}-{cn}.base east);\n') -# else: -# sio.write(f'\\path[draw, {ls}] ([xshift={-xshift}em, yshift={-yshift}em]{matrix_name}-{top}-{cn}.south east) -- ' -# f'([yshift={-descender_proportion-yshift}em, xshift={-xshift}em]{matrix_name}-{nr}-{cn}.base east);\n') - -# sio.write(footer.format(container_env=container_env, -# post_process=post_process)) - -# return sio.getvalue() - @staticmethod def sparsify(df, cs): out = df.copy() diff --git a/greater_tables/testdf.py b/greater_tables/testdf.py index e4927d0..6782a7b 100644 --- a/greater_tables/testdf.py +++ b/greater_tables/testdf.py @@ -251,7 +251,8 @@ class TestDataFrameFactory: return pd.Series(self.rng.normal(loc=0.5, scale=0.35, size=n)) if dtype == 'l': # log float (greater range) - return pd.Series(np.exp(self.rng.normal(loc=-4 / 2 + 4, scale=4, size=n))) + scale = 10. + return pd.Series(np.exp(self.rng.normal(loc=-scale**2 / 2 + 15, scale=scale, size=n))) if dtype == 'v': # log float (greater range) sc = 5 diff --git a/greater_tables/tex_svg.py b/greater_tables/tex_svg.py index bb044e8..3b8bfdd 100644 --- a/greater_tables/tex_svg.py +++ b/greater_tables/tex_svg.py @@ -19,6 +19,8 @@ class TikzProcessor: # Full TeX preamble to generate a .fmt if needed _tex_template_full = r"""\documentclass[10pt, border=5mm]{standalone} \usepackage{amsfonts} +\usepackage{amsmath} +\usepackage{mathrsfs} \usepackage{url} \usepackage{tikz} \usepackage{color} @@ -33,7 +35,7 @@ class TikzProcessor: # Minimal template to embed user tikz _tex_template = r""" -\newcommand{{\grtspacer}}{{\vphantom{{lp}}}} +\newcommand{{\I}}{{\vphantom{{lp}}}} % fka grtspacer \def\dfrac{{\displaystyle\frac}} \def\dint{{\displaystyle\int}} \begin{{document}} @@ -65,13 +67,17 @@ class TikzProcessor: print('TikzProcessor: building TeX format fmt file...', end ='') tmp = self.out_path / 'tikz_format.tex' tmp.write_text(self._tex_template_full, encoding='utf-8') - self.run_command([ + cmd = [ 'pdflatex', f'-ini', f'-jobname={self.format_file.stem}', '&pdflatex', tmp.name, - ], raise_on_error=True, cwd=self.out_path) + ] + print(f'Running {" ".join(cmd)} to build format file...') + (self.file_path.parent / 'make_format.bat').write_text(" ".join(cmd), encoding='utf-8') + self.run_command(cmd, raise_on_error=True, cwd=self.out_path) + # tidy up ... to some extent # tmp.unlink() (self.out_path / f'{self.format_file.stem}.log').unlink() print('...success...format file built', self.format_file.resolve()) @@ -100,14 +106,12 @@ class TikzProcessor: ] if self.debug: print("Running:", " ".join(tex_cmd)) + (tex_path.parent / 'make_tikz.bat').write_text(" ".join(tex_cmd), encoding='utf-8') if self.run_command(tex_cmd): raise ValueError('TeX failed to compile, not pdf or svg output.') # no tidying up else: - # continue - - (tex_path.parent / 'make_tikz.bat').write_text(" ".join(tex_cmd), encoding='utf-8') - + # no error: continue svg_cmd = [ # 'C:\\temp\\pdf2svg-windows\\dist-64bits\\pdf2svg', 'pdf2svg', diff --git a/pyproject.toml b/pyproject.toml index 70fba1d..5757a28 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,5 +1,5 @@ [build-system] -requires = ["setuptools", "wheel"] # Tools needed to build the package +requires = ["setuptools", "build"] build-backend = "setuptools.build_meta" [project] @@ -8,7 +8,8 @@ dynamic = ["version"] description = "Perfect tables from pandas dataframes." authors = [{name = "Stephen J Mildehall", email = "mynl@me.com" }] readme = {file = "README.md", content-type = "text/markdown"} -license = { text = "MIT" } +license = "MIT" +license-files = ["LICENSE"] requires-python = ">=3.10" dependencies = [ "bs4", @@ -21,14 +22,13 @@ dependencies = [ ] classifiers = [ "Development Status :: 4 - Beta", - "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Topic :: Office/Business" ] [tool.setuptools.packages.find] -include = ["greater_tables"] +include = ["greater_tables", "greater_tables.data"] exclude = ["img", "tests", "docs"] [tool.setuptools.package-data]