fix(html): keep the rows under a rowspan in their own columns - #2522
Lukas (L4XB) wants to merge 3 commits into
Conversation
A Markdown table has no way to merge cells down, so a cell carrying a rowspan
is written once and every row it reaches into comes out one cell short:
| Region | Product | Units |
| --- | --- | --- |
| EU | Cable | 12 |
| Hub | 7 |
Read back, `Hub` sits under Region and `7` under Product, so the value is
attributed to the wrong column and Units is empty. A Word table with a
vertically merged cell produces exactly this: mammoth writes the merge as
rowspan="2".
Fill the rows a span reaches into with the empty cells it stands for, before
markdownify lays the table out.
Two costs in the rowspan padding, both reachable from any HTML input: - A span asked for its placeholders unbounded. One cell with rowspan="1000" colspan="1000" over 999 more rows requested a million of them: 19 KB of HTML became 3 MB of Markdown in 20 s. A rowspan now reaches no further than the table's last row, and a table gets placeholders only while they stay within 64 + 8 per real cell; past that it is converted as markdownify converts it, without the padding. The same input now converts in 0.05 s. - Each placeholder was inserted with insert_before, which finds its anchor by scanning the siblings, so many placeholders in one row cost O(n^2): 30,000 in front of one cell took 12 s. The row is now rebuilt in one pass, taking its children out front to back (every extract() finds its node at index 0) and putting them back with the placeholders in place: 1 s. Tables within the budget, including every existing case, come out the same.
|
Pushed 1538d62 after reviewers of the same padding logic elsewhere (QwenLM/qwen-code#12039, LearningCircuit/local-deep-research#6520) measured two costs that apply here too. Both are reachable from any HTML input:
Two tests added: |
`rowspan="0"` spans a cell to the last row of its row group, but the
placeholder pass read every span through markdownify's colspan clamp, which
turns 0 into 1. Only the cell's own row was laid out, so the rows below it
came out one cell short, the same defect this branch fixes for a numbered
rowspan:
| Region | Product | Units |
| --- | --- | --- |
| EU | Cable | 12 |
| Hub | 7 |
Lay each span out inside its own row group, the way a browser does: a thead,
tbody or tfoot, or the table itself for the rows written directly under it. A
rowspan of 0 then reaches to the end of that group, and a rowspan too long for
its group no longer leaks a placeholder into the next one.
A span is still counted before any placeholder is made, so the budget holds:
200 cells with rowspan="0" over 2000 rows ask for 399,800 placeholders, and
the table is left as markdownify writes it, in 0.15 s.
colspan="0" is unchanged. HTML5 dropped it and a browser reads it as a single
column, which is what markdownify's clamp already gives.
|
One more commit on this branch:
Chrome reports
Tests: 4 new cases plus a budget guard in |
The bug
A Markdown table has no way to merge cells downward, so a cell carrying a
rowspanis written once and every row it reaches into comes out one cell short. The rows below it then read one column out of step.Measured on
main(eb31b5c):Read back,
Hubsits under Region and7under Product.Unitsis empty for that row, and the count7is silently attributed to the wrong column — which is worse than losing it, because nothing about the output looks wrong.A Word table with a vertically merged cell produces exactly this. Measured end to end on a
.docxbuilt withpython-docx(table.cell(1, 0).merge(table.cell(2, 0))), mammoth writes the merge asrowspan="2":markdownify pads a
colspanbut has nothing forrowspan, so this reaches.docx,.html,.epuband RSS alike. A region spanning several product rows, a category spanning its items, a date spanning a day's entries — merged first columns are ordinary in real documents.The fix
convert_soupfills the rows a span reaches into with the empty cells it stands for, before markdownify lays the table out:_fill_row_spanswalks each table once and builds the grid the way HTML defines it — a cell takes the next column not already taken by a span from above, then claimscolspancolumns across androwspanrows down. Each row that ends up with claimed columns gets an empty<td>at each of them.Two details worth calling out:
_spanclamps to1..1000, the same bounds markdownify already applies tocolspan, so a hostilerowspan="999999"cannot blow up the row count.A table with no
rowspanis untouched, andcolspankeeps being handled by markdownify exactly as before.Tests
packages/markitdown/tests/test_table_rowspan.py, 7 cases, all reading the produced table back the way a reader does and asserting on the cells:rowspan="3"filling both rows below;colspan-only table;.docxwith a real vertically merged Word cell.Measured:
Full suite:
849 passed, 14 skipped, no failures.black --checkclean.