summaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Readers/Markdown.hs
AgeCommit message (Collapse)Author
2024-03-14Markdown reader: Fix bug with footnotes at end of fenced div.John MacFarlane
Cloess #9576.
2024-02-19Markdown reader: fix regression in link parsing...John MacFarlane
with wikilinks extensions. This fixes a regression introduced in 3.1.12. Closes #9481.
2024-02-13Markdown reader: fix wikilinks extension to allow newlines in titles.John MacFarlane
Closes #9454.
2023-11-05Markdown reader: make attributes work with reference links.John MacFarlane
Closes #9171.
2023-10-20Markdown reader: fix blindspot with superscript in links.John MacFarlane
Previously `[^super^](#ref)` wasn't parsed as a link, due to code that was meant to prevent footnote markers from being recognized as reference links. This commit tightens up that code to avoid this bad effect. We have also added a new restriction on footnote labels: they cannot contain the characters `^`, `[`, or `]`. Though this is technically a breaking change, we suspect that the impact will be minimal, as it's very unlikely people would be using these characters in their note labels. Closes #8981.
2023-10-03Markdown reader: Don't apply `--default-image-extension` to data URIs.John MacFarlane
Closes #9118.
2023-09-14Markdown reader: More accurate check that a normalCite...John MacFarlane
...is not a link, bracketed span, or reference. See #9080.
2023-08-29Support images with wikilink syntax.John MacFarlane
E.g. `![[foo|bar]]`. (This requires enabling one of the `wikilinks` extensions.) Closes #8853.
2023-08-29Markdown: Allow a citation or reference link to be parsed after a `!`.John MacFarlane
Closes #8254.
2023-08-29Markdown reader: fix dropped `!` before nonexistent reference.John MacFarlane
In e.g. `![foo]` the `!` would be silently dropped if `[foo]` wasn't a reference link label. Closes #9038.
2023-06-28Markdown reader: Further strictness annotations.John MacFarlane
2023-06-28Markdown reader: add strictness annotations.John MacFarlane
This fixes the memory leak noted in #8762. Closes #8762.
2023-05-04Markdown reader: disallow escaping of `~` and `"` in markdown_strictAlbert Krewinkel
This matches the behavior of the legacy `markdown.pl` as well as what is described in the manual. Fixes: #8777
2023-01-18Fix line lengths in Markdown reader, DocBook writerAlbert Krewinkel
Trying to limit lines to 80 chars.
2023-01-15Markdown, CommonMark: add support for wiki links. [API change]Albert Krewinkel
Adds the Markdown/CommonMark extensions `wikilinks_title_after_pipe` and `wikilinks_title_before_pipe`. The former enables links of style `[[Name of page|Title]]` and the latter `[[Title|Name of page]]`. Titles are optional in both variants, so this works for both: `[[https://example.org]]`, `[[Name of page]]`. The writer is modified to render links with title `wikilink` as a wikilink if a respective extension is enabled. Pandoc will use `wikilinks_title_after_pipe` if both extensions are enabled. Closes: #2923
2023-01-13Support complex figures. [API change]Albert Krewinkel
Thanks and credit go to Aner Lucero, who laid the groundwork for this feature in the 2021 GSoC project. He contributed many changes, including modifications to the readers for HTML, JATS, and LaTeX, and to the HTML and JATS writers. Shared (Albert Krewinkel): - The new function `figureDiv`, exported from `Text.Pandoc.Shared`, offers a standardized way to convert a figure into a Div element. Readers (Aner Lucero): - HTML reader: `<figure>` elements are parsed as figures, with the caption taken from the respective `<figcaption>` elements. - JATS reader: The `<fig>` and `<caption>` elements are parsed into figure elements, even if the contents is more complex. - LaTeX reader: support for figures with non-image contents and for subfigures. - Markdown reader: paragraphs containing just an image are treated as figures if the `implicit_figures` extension is enabled. The identifier is used as the figure's identifier and the image description is also used as figure caption; all other attributes are treated as belonging to the image. Writers (Aner Lucero, Albert Krewinkel): - DokuWiki, Haddock, Jira, Man, MediaWiki, Ms, Muse, PPTX, RTF, TEI, ZimWiki writers: Figures are rendered like Div elements. - Asciidoc writer: The figure contents is unwrapped; each image in the the figure becomes a separate figure. - Classic custom writers: Figures are passed to the global function `Figure(caption, contents, attr)`, where `caption` and `contents` are strings and `attr` is a table of key-value pairs. - ConTeXt writer: Figures are wrapped in a "placefigure" environment with `\startplacefigure`/`\endplacefigure`, adding the features caption and listing title as properties. Subfigures are place in a single row with the `\startfloatcombination` environment. - DocBook writer: Uses `mediaobject` elements, unless the figure contains subfigures or tables, in which case the figure content is unwrapped. - Docx writer: figures with multiple content blocks are rendered as tables with style `FigureTable`; like before, single-image figures are still output as paragraphs with style `Figure` or `Captioned Figure`, depending on whether a caption is attached. - DokuWiki writer: Caption and "alt-text" are no longer combined. The alt text of a figure will now be lost in the conversion. - FB2 writer: The figure caption is added as alt text to the images in the figure; pre-existing alt texts are kept. - ICML writer: Only single-image figures are supported. The contents of figures with additional elements gets unwrapped. - HTML writer: the alt text is no longer constructed from the caption, as was the case with implicit figures. This reduces duplication, but comes at the risk of images that are missing alt texts. Authors should take care to provide alt texts for all images. Some readers, most notably the Markdown reader with the `implicit_figures` extension, add a caption that's identical to the image description. The writer checks for this and adds an `aria-hidden` attribute to the `<figcaption>` element in that case. - JATS writer: The `<fig>` and `<caption>` elements are used write figures. - LaTeX writer: complex figures, e.g. with non-image contents and subfigures, are supported. The `subfigure` template variable is set if the document contains subfigures, triggering the conditional loading of the *subcaption* package. Contants of figures that contain tables are become unwrapped, as longtable environments are not allowed within figures. - Markdown writer: figures are output as implicit figures if possible, via HTML if the `raw_html` extension is enabled, and as Div elements otherwise. - OpenDocument writer: A separate paragraph is generated for each block element in a figure, each with style `FigureWithCaption`. Behavior for single-image figures therefore remains unchanged. - Org writer: Only the first element in a figure is given a caption; additional block elements in the figure are appended without any caption being added. - RST writer: Single-image figures are supported as before; the contents of more complex images become nested in a container of type `float`. - Texinfo writer: Figures are rendered as float with type `figure`. - Textile writer: Figures are rendered with the help of HTML elements. - XWiki: Figures are placed in a group. Co-authored-by: Aner Lucero <4rgento@gmail.com>
2023-01-10Update copyright years, it's 2023!Albert Krewinkel
2022-11-19Add support for `mark` extension for highlighted text.John MacFarlane
+ Extensions: Add `Ext_mark` extension. This is not part of the pandoc extensions by default. + Markdown reader: parse `==..==` if `mark` extension enabled. + Markdown writer: support `mark` extension. + Docx writer: render Span with class `mark` as highlighted. Currently yellow is hardcoded. + LaTeX writer: support highlighted text for Span with class `mark`. + RST writer: use special `mark` role for Span with class `mark`. + Update MANUAL.txt. Closes #7743.
2022-10-29T.P.Parsing.General: change `characterReference`, `charsInBalanced`.John MacFarlane
`characterReference` now returns a Text (as it should, because some named references don't correspond to a single Char), and uses the `lookupEntity` function from commonmark-hs instead of the slow one from tagsoup. `charsInBalanced` now takes a Text parser rather than a Char parser as argument. [API change]
2022-10-18T.P.Shared: remove `escapeURI`, `isURI`.John MacFarlane
These are now exported by Text.Pandoc.URI, and removing them from Shared helps make module structure more straightforward.
2022-10-18Revert "Markdown reader: avoid duplicate ids with auto_identifiers."John MacFarlane
This reverts commit eff82cfe4de44a111250ce9ce3ecee2fd4d99924.
2022-10-18Markdown reader: avoid duplicate ids with auto_identifiers.John MacFarlane
We previously avoided generating a duplicate with another automatically generated identifier; now we also avoid duplicates with explicit identifiers that occur before the header for which an identifier is being generated. (Collisions are still possible for identifiers that occur after the header.) T.P.Shared: `makeSections` is also modified so it doesn't give bad results when the enclosing Div has a different identifier from the header, as may now happen.
2022-10-16T.P.Parsing: Remove gratuitious renaming of Parsec types.John MacFarlane
We were exporting Parser, ParserT as synonyms of Parsec, ParsecT. There is no good reason for this and it can cause confusion. Also, when possible, we replace imports of Text.Parsec with T.P.Parsing. The idea is to make it easier, at some point, to switch to megaparsec or another parsing engine if we want to. T.P.Parsing new exports: Stream(..), updatePosString, SourceName, Parsec, ParsecT [API change]. Removed exports: Parser, ParserT [API change].
2022-09-14Fix implicit_header_references with duplicate headings.John MacFarlane
Documentation says that when more than one heading has the same text, an implicit reference `[Heading text][]` refers to the first one. Previously pandoc linked to the last one instead. This patch makes pandoc conform to the documented behavior. See #8300.
2022-08-30Markdown: Allow table caption labels to start with lowercase t.John MacFarlane
Closes #8259.
2022-08-27Avoid thunk in markdown reader.John MacFarlane
2022-08-25Markdown reader: fenced code block shortcuts with attributes (#8174)Siphalor
This allows the combination of the fenced code block shortcut form with attributes: ```` ```haskell {.class #id} ``` ```` The code syntax class will be combined with the attribute classes. This syntax allows for more intuitive writing and for better compatibility with other Markdown parsers such as GitHub or Codeberg. Closes #8174.
2022-08-01Markdown reader: allow special span classes in any positionAlbert Krewinkel
2022-08-01Markdown reader: allow more attributes in special spansAlbert Krewinkel
Spans with "smallcaps" as the first class are converted to *SmallCaps* elements. While previously no other classes or attributes were allowed, additional classes, attributes, and an identifier are not permitted and kept in a *SmallCaps* wrapping *Span* element. The same change is applied to underline spans, where the first class must be either "ul" or "underline". Closes: #4102
2022-07-30Support rowspans and colspans in grid tables (#8202)Albert Krewinkel
* Add tests for zero-width and fullwidth chars in grid tables * T.P.Parsing: simplify `gridTableWith'`, `gridTableWith` [API Change] The functions `gridTableWith` and `gridTableWith'` no longer takes a boolean argument that toggles whether a table head should be parsed: both, tables with heads and without heads, are always accepted now. * Support colspans, rowspans, and multirow headers in grid tables. Grid tables in Markdown, reStructuredText, and Org can now contain cells spanning over multiple columns and/or multiple rows; table headers containing multiple rows are supported as well. Note: the markdown writer does not yet support these more complex grid table features.
2022-04-20Don't parse inline notes with blank lines inside.John MacFarlane
Previously we parsed them but discarded part of the content. Closes #8028.
2022-03-31Markdown reader: add some strictness.John MacFarlane
This improves some benchmarks significantly.
2022-03-24[API change] Unify grid table parsing (#7971)Albert Krewinkel
Grid table parsing in Markdown and rst are updated use the same functions. Functions are generalized to meet requirements for both formats. This change also lays the ground for further generalizations in table parsers, including support for advanced table features. API changes in Text.Pandoc.Parsing: - Parse results of functions `tableWith'` and `gridTableWith'` are now a `mf TableComponents` instead of a quadruple of alignments, column widths, header rows and body rows. Additional exports from Text.Pandoc.Parsing: - `tableWith'` - `TableComponents` - `TableNormalization` - `toTableComponents` - `toTableComponents'`
2022-02-23Markdown reader: remove restriction on identifiers...John MacFarlane
so they no longer need to begin with a letter. Closes #7920.
2022-02-13Markdown reader: allow one-column pipe tables with pipe on right.John MacFarlane
See #7919. We still need to implement this for gfm (commonmark). This must be done via changes in commonmark-hs.
2022-01-07Fix parsing of footnotes in `--metadata-file`.John MacFarlane
Closes #7813.
2022-01-02Copyright notices: update for 2022Albert Krewinkel
2021-12-13Markdown reader: fix parsing of "bare locators"...John MacFarlane
...after author-in-text citations. Previously `@item [p. 12; @item2]` was incorrectly parsed as three citations rather than two. This is now fixed by ensuring that `prefix` doesn't gobble any semicolons.
2021-12-07Revert "Markdown reader: Improve inlinesInBalancedBrackets."John MacFarlane
This reverts commit fa83246d7de8527bbf59dfac9636a42ede185194.
2021-11-30Markdown reader: don't allow `^` at beginning of link or image label.John MacFarlane
This is reserved for footnotes. Fixes a regression introduced by 0a93acf. Closes #7723.
2021-11-23Improve detection of pipe table line widths.John MacFarlane
Fixed calculation of maximum column widths in pipe tables. It is now based on the length of the markdown line, rather than a "stringified" version of the parsed line. This should be more predictable for users. In addition, we take into account double-wide characters such as emojis. Closes #7713.
2021-11-02Markdown reader: Improve inlinesInBalancedBrackets.John MacFarlane
This is just a small improvement in terms of performance, but it's simpler and more direct code. Also, we avoid parsing interparagraph spaces in balanced brackets, as the original did.
2021-10-27Switch back from HsYAML to yaml.John MacFarlane
Reasons: - Performance: HsYAML is around 20 times slower in parsing large YAML bibliographies (#6084). - An issue was submitted to HsYAML, but it hasn't gotten any attention. HsYAML seems borderline unmaintained; it hasn't had a commit in over a year. - Unfortunately this goes back on our attempts to free ourselves from C dependencies (#4535). But I don't see a better alternative until a better pure Haskell parser is available. Closes #6084. Notes: - We've removed the FromYAML instances for all types that had them, since this is a HsYAML-specific typeclass [API change]. (The yaml package just uses From/ToJSON.) - Unlike HsYAML (in the configuration we were using), yaml parses 'Y', 'N', 'Yes', 'No', 'On', 'Off' as boolean values. Users may need to quote these when they are meant to be interpreted as strings. Similarly, 'null' is parsed as a YAML null value (and will be treated as an empty string by pandoc rather than the string 'null'). Quoting it will force it to be interpreted as a string. - Some tests had to be adjusted accordingly. - Pandoc now behaves better when the YAML metadata contains escaping errors: instead of just falling back on treating the section as a table, it raises a YAML parsing error.
2021-10-22Use simpleFigure in Readers.Aner Lucero
2021-10-20Markdown reader: don't parse links or bracketed spans as citations.John MacFarlane
Previously pandoc would parse [link to (@a)](url) as a citation; similarly [(@a)]{#ident} This is undesirable. One should be able to use example references in citations, and even if `@a` is not defined as an example reference, `[@a](url)` should be a link containing an author-in-text citation rather than a normal citation followed by literal `(url)`. Closes #7632.
2021-10-13Fix markdown parsing bug for math in bracketed spans and links.John MacFarlane
This affects math with unbalanced brackets (e.g. `$(0,1]$`) inside links, images, bracketed spans. Closes #7623.
2021-09-17Fix linter warning.John MacFarlane
2021-09-16Fix code blocks using `--preserve-tabs`.John MacFarlane
Previously they did not behave as the equivalent input with spaces would. Closes #7573.
2021-08-23Markdown reader: fix interaction of --strip-comments and listJohn MacFarlane
parsing. Use of `--strip-comments` was causing tight lists to be rendered as loose (as if the comment were a blank line). Closes #7521.
2021-08-16Fix bug in last commit due to removal of take1WhileP.John MacFarlane