summaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Readers/HTML.hs
AgeCommit message (Collapse)Author
2023-12-06Revert "Use base64 instead of base64-bytestring."John MacFarlane
This reverts commit 6625e9655ed2bb0c4bd4dd91b5959a103deab1cb. base64 is currently buggy on 32-bit systems. Closes #9233.
2023-11-15HTML reader: Fix handling of invalidly nested sublists.John MacFarlane
This revises the fix to #8150 (and the test case) and closes #9187. HTML in the (invalid) form: <ul> <li>L1</li> <ul> <li>L1.1</li> </ul> </ul> is treated by browsers like <ul> <li>L1 <ul> <li>L1.1</li> </ul> </li> </ul> not <ul> <li>L1 <li><ul> <li>L1.1</li> </ul> </li> </ul> as pandoc previously assumed. This change will give a similar treatment to <ul> <li>L1</li> <p>foobar</p> </ul> which also seems to match browser behavior.
2023-10-27Fix spacing.John MacFarlane
2023-09-16HTML reader: parse task lists using input elements (#9066)Seth Speaks
Allow the HTML reader to parse task lists of the sort produced by pandoc. Closes #9047
2023-08-09HTML reader: avoid duplicate id on header and div.John MacFarlane
Closes #8991.
2023-05-17HTML reader: fix iframe with data URI of an image.John MacFarlane
Closes #8856. In this case we don't want to try to parse the data at the URL. Instead, create an image inside a div.
2023-03-20HTML reader: fix behavior with `-native_spans-raw_html`.John MacFarlane
Previously with this configuration, `<span>`s were not treated as inline elements at all. Closes #8711.
2023-01-13Support complex figures. [API change]Albert Krewinkel
Thanks and credit go to Aner Lucero, who laid the groundwork for this feature in the 2021 GSoC project. He contributed many changes, including modifications to the readers for HTML, JATS, and LaTeX, and to the HTML and JATS writers. Shared (Albert Krewinkel): - The new function `figureDiv`, exported from `Text.Pandoc.Shared`, offers a standardized way to convert a figure into a Div element. Readers (Aner Lucero): - HTML reader: `<figure>` elements are parsed as figures, with the caption taken from the respective `<figcaption>` elements. - JATS reader: The `<fig>` and `<caption>` elements are parsed into figure elements, even if the contents is more complex. - LaTeX reader: support for figures with non-image contents and for subfigures. - Markdown reader: paragraphs containing just an image are treated as figures if the `implicit_figures` extension is enabled. The identifier is used as the figure's identifier and the image description is also used as figure caption; all other attributes are treated as belonging to the image. Writers (Aner Lucero, Albert Krewinkel): - DokuWiki, Haddock, Jira, Man, MediaWiki, Ms, Muse, PPTX, RTF, TEI, ZimWiki writers: Figures are rendered like Div elements. - Asciidoc writer: The figure contents is unwrapped; each image in the the figure becomes a separate figure. - Classic custom writers: Figures are passed to the global function `Figure(caption, contents, attr)`, where `caption` and `contents` are strings and `attr` is a table of key-value pairs. - ConTeXt writer: Figures are wrapped in a "placefigure" environment with `\startplacefigure`/`\endplacefigure`, adding the features caption and listing title as properties. Subfigures are place in a single row with the `\startfloatcombination` environment. - DocBook writer: Uses `mediaobject` elements, unless the figure contains subfigures or tables, in which case the figure content is unwrapped. - Docx writer: figures with multiple content blocks are rendered as tables with style `FigureTable`; like before, single-image figures are still output as paragraphs with style `Figure` or `Captioned Figure`, depending on whether a caption is attached. - DokuWiki writer: Caption and "alt-text" are no longer combined. The alt text of a figure will now be lost in the conversion. - FB2 writer: The figure caption is added as alt text to the images in the figure; pre-existing alt texts are kept. - ICML writer: Only single-image figures are supported. The contents of figures with additional elements gets unwrapped. - HTML writer: the alt text is no longer constructed from the caption, as was the case with implicit figures. This reduces duplication, but comes at the risk of images that are missing alt texts. Authors should take care to provide alt texts for all images. Some readers, most notably the Markdown reader with the `implicit_figures` extension, add a caption that's identical to the image description. The writer checks for this and adds an `aria-hidden` attribute to the `<figcaption>` element in that case. - JATS writer: The `<fig>` and `<caption>` elements are used write figures. - LaTeX writer: complex figures, e.g. with non-image contents and subfigures, are supported. The `subfigure` template variable is set if the document contains subfigures, triggering the conditional loading of the *subcaption* package. Contants of figures that contain tables are become unwrapped, as longtable environments are not allowed within figures. - Markdown writer: figures are output as implicit figures if possible, via HTML if the `raw_html` extension is enabled, and as Div elements otherwise. - OpenDocument writer: A separate paragraph is generated for each block element in a figure, each with style `FigureWithCaption`. Behavior for single-image figures therefore remains unchanged. - Org writer: Only the first element in a figure is given a caption; additional block elements in the figure are appended without any caption being added. - RST writer: Single-image figures are supported as before; the contents of more complex images become nested in a container of type `float`. - Texinfo writer: Figures are rendered as float with type `figure`. - Textile writer: Figures are rendered with the help of HTML elements. - XWiki: Figures are placed in a group. Co-authored-by: Aner Lucero <4rgento@gmail.com>
2023-01-10Update copyright years, it's 2023!Albert Krewinkel
2022-10-18T.P.Shared: remove `escapeURI`, `isURI`.John MacFarlane
These are now exported by Text.Pandoc.URI, and removing them from Shared helps make module structure more straightforward.
2022-10-18Revert "HTML reader: avoid duplicating any existing identifier..."John MacFarlane
This reverts commit e5fbddd3b6c0c7a3b76b313edbe55242e3b138fc.
2022-10-18Revert "T.P.Parsing: export `registerIdentifier`."John MacFarlane
This reverts commit 20492d523c8324e36781cfbbc8092c796f94b151.
2022-10-18T.P.Parsing: export `registerIdentifier`.John MacFarlane
[API change] Use this in the HTML reader to register identifiers to avoid duplicates created by `auto_identifiers`.
2022-10-18HTML reader: avoid duplicating any existing identifier...John MacFarlane
with `auto_identifiers`. Closes #8383.
2022-10-16T.P.Parsing: export errorMessages, messageString.John MacFarlane
[API change]
2022-10-16T.P.Parsing: Remove gratuitious renaming of Parsec types.John MacFarlane
We were exporting Parser, ParserT as synonyms of Parsec, ParsecT. There is no good reason for this and it can cause confusion. Also, when possible, we replace imports of Text.Parsec with T.P.Parsing. The idea is to make it easier, at some point, to switch to megaparsec or another parsing engine if we want to. T.P.Parsing new exports: Stream(..), updatePosString, SourceName, Parsec, ParsecT [API change]. Removed exports: Parser, ParserT [API change].
2022-09-28HTML Reader: Tests for 'tt' and 'code' elements (#8330).Justin Wood
tt is a deprecated element in HTML, but is still used in the wild in some places, support reading it as just another 'code' element. Commit 4abb9d0ad8dbb88fbc443a78d5a1b116cb7a5816 was originally part of this PR as well.
2022-09-28HTML reader: fix regression for `<tt>`.John MacFarlane
It was no longer being parsed as Code. See #8330.
2022-08-30Mediawiki reader: Parse table cell with attribs, to support rowspan, colspan ↵Ruqi
(#8231)
2022-08-21Fix regression with data uris in 2.19.1.John MacFarlane
In 2.19.1 we used the base64URL encoding rather than base64. This works in Safari, apparently, but not in other browsers. Closes #8239.
2022-08-14Use base64 instead of base64-bytestring.John MacFarlane
It is supposed to be faster and more standards-compliant.
2022-08-03HTML reader: allow sublists that are not marked as items.Albert Krewinkel
The HTML standard requires all list items to be marked with a `<li>` tag, but some tools fail to do so for sublists. The reader now accepts these unwrapped lists as sublists. Closes: #8150
2022-07-06Use `formatCode` from #7525 in HTML and MediaWiki (#8162)Elliot Bobrow
Move formatting from inside inline code elements to the outside in order to retain formatting.
2022-02-09Fix parsing of epub footnotes.John MacFarlane
Closes #7884.
2022-02-07HTML reader: give warnings and emit empty note...John MacFarlane
when parsing `<a epub:type="noteref">` and the identifier doesn't correspond to anything in the note table. Previously we just silently skipped these cases. See #7884.
2022-01-02Copyright notices: update for 2022Albert Krewinkel
2021-11-24HTML reader: parse attributes on links and images.John MacFarlane
Closes #6970.
2021-10-22Use simpleFigure in Readers.Aner Lucero
2021-07-06Recognize data-external when reading HTML img tags (#7429)Michael Hoffmann
Preserve all attributes in img tags. If attributes have a `data-` prefix, it will be stripped. In particular, this preserves a `data-external` attribute as an `external` attribute in the pandoc AST.
2021-05-17HTML writer: keep attributes from code nested below pre tag.Albert Krewinkel
If a code block is defined with `<pre><code class="language-x">…</code></pre>`, where the `<pre>` element has no attributes, then the attributes from the `<code>` element are used instead. Any leading `language-` prefix is dropped in the code's *class* attribute are dropped to improve syntax highlighting. Closes: #7221
2021-05-15HTML writer: parse `<header>` as a DivAlbert Krewinkel
HTML5 `<header>` elements are treated like `<div>` elements.
2021-05-14HTML reader: keep h1 tags as normal headers (#7274)Albert Krewinkel
The tags `<title>` and `<h1 class="title">` often contain the same information, so the latter was dropped from the document. However, as this can lead to loss of information, the heading is now always retained. Use `--shift-heading-level-by=-1` to turn the `<h1>` into the document title, or a filter to restore the previous behavior. Closes: #2293
2021-05-14HTML reader: don't fail on unmatched closing "script" tag.Albert Krewinkel
Prevent the reader from crashing if the HTML input contains an unmatched closing `</script>` tag. Fixes: #7282
2021-05-09Change reader types, allowing better tracking of source positions.John MacFarlane
Previously, when multiple file arguments were provided, pandoc simply concatenated them and passed the contents to the readers, which took a Text argument. As a result, the readers had no way of knowing which file was the source of any particular bit of text. This meant that we couldn't report accurate source positions on errors or include accurate source positions as attributes in the AST. More seriously, it meant that we couldn't resolve resource paths relative to the files containing them (see e.g. #5501, #6632, #6384, #3752). Add Text.Pandoc.Sources (exported module), with a `Sources` type and a `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs. [API change] A parsec `Stream` instance is provided for `Sources`. The module also exports versions of parsec's `satisfy` and other Char parsers that track source positions accurately from a `Sources` stream (or any instance of the new `UpdateSourcePos` class). Text.Pandoc.Parsing now exports these modified Char parsers instead of the ones parsec provides. Modified parsers to use a `Sources` as stream [API change]. The readers that previously took a `Text` argument have been modified to take any instance of `ToSources`. So, they may still be used with a `Text`, but they can also be used with a `Sources` object. In Text.Pandoc.Error, modified the constructor PandocParsecError to take a `Sources` rather than a `Text` as first argument, so parse error locations can be accurately reported. T.P.Error: showPos, do not print "-" as source name.
2021-03-18Rewrite a foldl1 as a foldl'.John MacFarlane
2021-02-20HTML reader: small performance tweak.John MacFarlane
2021-02-20HTML reader: small efficiency improvements.John MacFarlane
Also, remove exported class NamedTag(..) [API change]. This was just intended to smooth over the transition from String to Text and is no longer needed. The functions isInlineTag and isBlockTag are no longer polymorphic.
2021-02-20HTML reader: efficiency improvements.John MacFarlane
Do a lookahead to find the right parser to use. Benchmarks from 34ms to 23ms, with less allocation. Also speeds up the epub reader.
2021-02-13HTML reader: fix bad handling of empty src attribute in iframe.John MacFarlane
- If src is empty, we simply skip the iframe. - If src is invalid or cannot be fetched, we issue a warning and skip instead of failing with an error. - Closes #7099.
2021-01-08Update copyright notices for 2021 (#7012)Albert Krewinkel
2020-12-28HTML reader: use renderTags' from Text.Pandoc.Shared.Albert Krewinkel
The `renderTags'` function was duplicated when the reader used `Text` as its string type. The duplication is no longer necessary. A side effect of this change is that empty `<col>` elements are written as self-closing tags in raw HTML blocks.
2020-12-10HTML reader: pay attention to lang attributes on body.John MacFarlane
These (as well as lang attributes on html) should update lang in metadata. See #6938.
2020-12-10HTML reader: retain attribute prefixes and avoid duplicates.John MacFarlane
Previously we stripped attribute prefixes, reading `xml:lang` as `lang` for example. This resulted in two duplicate `lang` attributes when `xml:lang` and `lang` were both used. This commit causes the prefixes to be retained, and also avoids invald duplicate attributes. Closes #6938.
2020-11-26HTML reader: improve support for table headers, footer, attributesAlbert Krewinkel
- `<tfoot>` elements are no longer added to the table body but used as table footer. - Separate `<tbody>` elements are no longer combined into one. - Attributes on `<thead>`, `<tbody>`, `<th>`/`<td>`, and `<tfoot>` elements are preserved.
2020-11-26HTML reader: allow finer grained options for tag omissionAlbert Krewinkel
2020-11-25HTML reader: simplify list attribute handlingAlbert Krewinkel
This removes the `foldOrElse` function from the internal Text.Pandoc.CSS module.
2020-11-24HTML reader: support row or column-spanning table cellsAlbert Krewinkel
2020-11-24HTML reader: support blocks in captionAlbert Krewinkel
2020-11-24HTML reader: extract table parsing into separate moduleAlbert Krewinkel
2020-11-23HTML reader: extract submodulesAlbert Krewinkel
Reducing module size should reduce memory use during compilation. This is preparatory work to tackle support for more table features.