path: root/WORKLOG.adoc

= WORKLOG

* use Groovy language specification as case study to find corner cases
  - perhaps progit2 as well; others?

== TODO

* use <a id=""></a> instead of <a name=""></a> for anchor point in formatted text
  - benchmark to see if it's faster to use empty or non-empty element in parser
* rename "convert_context_for_" since it can collide with existing blocks
* rework resolve_image_path API so it's more logical; override based on type of first argument; document as option
* implicit width of inline image calculated incorrectly (see sandbox/issue-494)
* QUESTION should we report full image path of gif in warning message?
* QUESTION should we add destination to top of imported PDF page?
  - import page should accept id as section, optional argument
* leading (line height) isn't applied when content is split across pages
* generate fonts without PS Glyph Names to reduce file size (waiting PR #550 to be marged)
  - create script that can generate fonts entirely from original font source
* if start_new_page is called at end of layout_chapter_title, and media=prepress, ghostscript reports an error
  - problem is no color space is set; can fix by calling update_colors before advancing to recto page in start_new_chapter
  - maybe introduce a skip_page helper to combine these operations?
* SIMPLE: document vw/vh measurement units in theming guide (make it clear it only applies in certain cases)
* SIMPLE: in theming guide, emphasize not using the fallback font when working with very large documents; explain that default theme uses a fallback font
* document how to test / use a PR
  - see https://github.com/asciidoctor/asciidoctor-pdf/issues/462#issuecomment-246200953
* FILE ISSUE break printed URIs at each forward slash (and perhaps at ? and & too)
* add empty? method to Page (instead of page_empty? on document)
* SIMPLE: add warning that AsciiDoc table cell content is not supported; print as raw source
* FILE ISSUE in prawn-svg not to move cursor (maybe an option?)
* FILE ISSUE: prevent import_page from adding an extra page; leaves blank page when placing two inserts in a row
  - have import_page first delete current page if page is empty
* FILE ISSUE in prawn: go_to_page should accept second argument that is cursor position
* autowidth causes cells in header row to break word (wraps by char to fit; does not expand table)
* UNRESOLVED: dry_run should start at cursor of main document (or as option); total height calculation would need to be revised
  - box_height isn't currently accurate when it spans more than one page
  - this should fix height calculation when content is split over a page break (leaving small amount of excess)
  - however, if cursor is advanced to fit content on page, then that excess will cause box_height to be too large
  - life would be simpler if Prawn allowed us to draw graphics at bottom layer
* allow height of inline image to be set to line height (perhaps 1em?)
* space around inline anchors/index entries doesn't get collapsed by text formatter
* add support for format attribute on image macro to image-related attributes such as title-page-background-image
  - support explicit image format for cover page image and page background image
* need some sort of post_construct method for converter that receives document
  - monkeypatch?
* backport resize method from prawn-svg and use it in converter
* create document that explains how built-in fonts are generated and what subsets are selected
  - I need instructions for myself so I know how to update/modify the fonts
  - document in theming guide what must be done to prepare fonts (old-style 'kern' table, optionally subset) (file issue!)
* consider supporting icon tag in parser to simplify how inline icons are stored; simpler use of passthrough content
* cache stateless cell data resolved from theme (don't need to recalc each time; at least per table)
* autowidth on table doesn't work for multi-line content (prawn-table bases width calculation on normalized value)
* table logic: does the layout_table_caption have to be inside the table block? can we pre-calculate the actual width for the caption? does the table offer a callback we can use to keep the caption on the same page as the table?
* introduce object to store/organize running content data and specs
* QUESTION: should we set pdf-anchor attribute on every node that has an id?
  - isn't it required for cross references to work?
* QUESTION: should theme font handle hierarchical keys (either explicitly or implicitly)
* need to support .canvas role on image so it isn't shrunk to fit inside top/bottom margins
  - perhaps .canvas, .canvas-x, .canvas-y
* SIMPLE: missing text_transform in some places in theming guide
* FILE ISSUE: when split source listing, add top padding to bounding box (or is it the line metrics top?)
  - perhaps this has to do with the a miscalculation in dry run when not starting from same y position?
* test prawn-gmagick (fixes PNG transparency issue, adds gif support); don't fail-fast on gif if gem is detected
  - https://github.com/packetmonkey/prawn-gmagick
* stroke and fill multi-page sidebar block (#259), quote/verse block (#270), and example block (#362)
* support URL images in running content
* support pdfwidth for inline image
* SIMPLE: add defaults for table_header_cell in theming guide, including when a key inherits by default
* rewrite optimize-pdf using rghost (#535) (also see #21 and #22)
  - add Optimizer class; wire to cli (separate issue?)
* add feature to number bullets according to section number (needed for OpenDevise agreements)
* outline should link to title page if there's a cover page (skip cover page and ensuing blank page)
* might be better to organize fragments of source chunks by lines (and pass that around) to simplify post-processing
* support negative start value for list (#498)
  - need to count negative numbers in correct direction
* SIMPLE: submit pastie theme for Rouge upstream
* the nested? method on list isn't checking if nested inside a block inside a list
* wrapped lines in source listing should be indented to account for line number gutter (#504)
* add sample SVG to content of chronicles-example.adoc (we do already use one for title page)
* inline images: allow built-in font family names for SVG to be remapped
* inline images: should we be passing absolute image path in tag or something relative (or even a lookup id?)?
* support zero-leading integers (use dedicated type like w/ roman numerals) in reversed order lists
* large image runs into running footer (doesn't bottom margin need to be aligned with running footer height?)
* should str_to_pt helper handle % and vw units?
* allow style of visible link to be controlled by theme (similar to what we do in the manpage converter)
* check that color of table edges set to table_grid_color??
* shouldn't the default align for headings be $base_align??
* should not wrap at formatting mark unless it is at a break opportunity
  - the problem here is that Prawn is allowing breaks at the boundaries of text fragments; it should only look at the contents
  - FILE ISSUE in Prawn!
* be more consistent with how examples are shown in theming-guide.adoc (do we show last segment only?)
* allow top as alternative to margin_top for all elements on title page (#431)
* swallowing exceptions! (any use of e.message in a string is dangerous)
* allow ordered list marker to be prefixed by section number (a global setting?)
* conum not aligned vertically with callout text (perhaps too small?)
* conum should never wrap (push it into the text if necessary)
* decouple theme settings for section titles and discrete headings
* table should expand to 100% if "spread" role is set
* replace explicit char ranges with classes in regexp (e.g., [[:word:]] or \w)
* test Rouge upgrade (should address workaround we have in encoder)
* devise a way to specify a value as a string literal (variable replacement only) in theme
* document pdfwidth in themeing guide (currently no mention of it)
* convert "-" in variable name in theme to underscore (e.g., $base-font-size)
* apply calculated theme values after loading?
* allow "content" in place of recto_content & verso_content for running header/footer
* allow image to span width of page (role=canvas, role=canvas-x or role=canvas-y); if role is canvas or canvas-y, then it does not consume height
  - partially addressed by vw units
* allow valign value to be a number (requires change to Prawn)
* layout SVG without using keep_together (since we know all the dimensions)
  - fix SVG to a single page (check height + caption height)
* allow background color to be set for heading
* allow border to be set around block image
* switch wolpertinger to howling wolf mouse
* file issue in prawn to dispatch to image handler for images it doesn't know about
* support arbitrary numeric entities (convert to unicode)
* add brief mention in theming guide that deeper customizations can be achieved by extending the converter
* allow alignment of list to be set separately from base align (and perhaps a hint in document)
* due to order of loading, getting warnings when redefining methods in Asciidoctor core
* prawn-pdfimage https://github.com/packetmonkey/prawn-pdfimage
* rename ThemeLoader to ThemeReader (or ThemeManager)?
* document autofit option on verbatim blocks
* document how to extend the converter, use Prawn
* using tabs instead of spaces for source code indentation allows leading whitespace to be preserved when copying
  - is this always the case?
* normalize step leaves space after endline (i.e., `\n `) at a hard line break (doesn't seem to affect flow)

* *margin per heading level* (see https://github.com/asciidoctor/asciidoctor-pdf/issues/176)
* document how to override the Ruby code to get custom styling in the theming guide
* convenience method to check if there's enough room for another line on page
* allow font size in theme to be specified in em or %
* allow dynamic background image with page number in path
* running header/footer covers content (perhaps just a limitation that needs to be documented)
* document that palette-based transparent in PNGs is not supported
* add broader character range to monospace font so we can drop fallback font by default (#282)
  - consider having a fallback for prose and fallback for literal
* file issue for prawn to preserve space (even w/ guards, spaces don't preserve over wrapped lines)
  - if this is fixed, we can remove all the guard indent code
* file issue for prawn to support spacer fragments with fixed width / height and no text (or text is ignored in calculations)
* file issue that prawn-svg messes with the cursor (need to explain how)
* document limitations in README (such as no linear gradients in SVG, embedded images in SVG must be URL or inlined, PNGs must be flattened, etc)
* document all permutations of image sizing
* set vposition on title page logo image explicitly to avoid page overrun?

* allow pdf-page-layout (portrait || landscape) to be set in document
* allow pdf-page-margin to be set in document
* verse has problems with wrapping if line is long
* allow links to be underlined (theme setting)
* allow default kerning to be set using theme
* keep line comment in front of callout number to aid copying?
* table column header is wrapping by char (verify?)
* implement margin collapsing (between blocks)
* rework pull request for source line numbers (combine with restore conum logic if conums are enabled)
  - also combine with the preserve_space logic
* allow front cover and back cover image to be defined in theme; document in theming-guide
* should we shorten the keys to front-cover and back-cover (since image is implied?)
* keep caption with table (check for sufficient space); only for top placement since bottom placement is much harder
* allow valign to be set on image block (vertical center in page for things like slides)
* separate theme control for listing vs literal block (and maybe source too)
* allow title page image "bottom" to be set instead of "top" (mutually exclusive)
* rtl (see ./sandbox/rtl/ folder)

* pass macro doesn't work in source block when macro subs and highlighting are both enabled (#180)
* enable cache_images option for prawn-svg (#223)
* bind image_registry between scratch and main document so we don't process the same image more than once
  - need to do some testing
* show SVG warnings if debug (or trace) is on
* clean temporary files once per conversion? (file issue)
* clear font paths in SVG interface so it doesn't scan system? (since it's not portable anyway)
* title is being rendered 3 times (maybe one for scratch?); explain why in comments if normal
* finish docs/theming-guide.adoc
  - continue working on json schema for theme; try to generate keys section from it
* rethink how we're handling line heights for fonts, then document carefully
  - look closer at line_height and line_height_length and see if we need to document other details
* implement first-line indent for paragraphs (seems like conflict w/ our text formatter)
  - option to not indent first paragraph in section
  - if you indent, perhaps drop the margin between paragraphs?
* add index support
* add entry to TOC for preamble/preface
* can we create fragments in converter instead of using the formatted text parser?
* allow text alignment of prose to be set in document
* don't issue warnings on scratch document
* rake release seems messed up (tagging the wrong commit)
* getting a line wrap break before comma if preceding word is emphasized (problem in Prawn wrapping)
* toc
  - make dot leader style separate from title / number
* running content
  - side margins (allow override, default to content margins)
  - numbered and unnumbered chapter and section titles (file issue)
  - chapter and section number (easily solved by previous)
  - separate running content for chapter page (by default uses normal content)
* should we rename base_ to body_ to make it more familiar to CSS developers?
* support !include in theme file (file issue)
* add cover page example to chronicles so people see how to use it
* don't orphan a single line of paragraph (send it with a buddy line)
* implement stem support
* fail gracefully if theme file cannot be found
* expose theme variable on document (attr_reader?)
* dedicated style for top/bottom margin of outline list
  - allow margin top and bottom to be set for lists (applies to outer-most list)
  - allow spacing between nested lists levels be configured in theme
* need dedicated theme styles for paragraph spacings, etc
* can't put margin top on chapter (chapter_top?)
  - chapter / heading background color
* recto/verso indentation (on body?)
* don't indent and draw line next to quote block unless width > 0 or color != transparent
* subtitles for chapters
* part titles need their own page and styling
* add color calculation functions in theme file (like in SASS)
* create utility method to get % offset of page as y value (option to constrain to bounds)
* document converter assignment in convert_content_for_block method
* support transparency for colors (this is now supported by resolve_theme_color)
* support generic color (or value) attribute in formatted text parser instead of specific color systems (rgb, cmyk)
* **allow theme_font to set line_height** (honor this setting from document)
  - theme setting for code line height (currently using base_line_height)
* should we put an entry for doctitle in the outline if notitle is set? (need to test these edge cases)
* use docdate attribute to set modification date on document (file issue)
* add more theme control over toc (font size, style, color per level)
* don't allow formatted text (e.g., monospace) in toc entries
* prevent title-logo-image from spilling to next page (same with title content)
* document what each keep_together is doing / expects
  - keep_together really needs to pick up the inherited horizontal bounds or else measurement is inaccurate
* code cleanups (regexps to constants, nil? checks and such)
  - split prawn_ext/extensions into individual files based on function
* enable line above (or below?) title on title page (file issue)
* enable text_transform for table foot row
* file upstream issue for Prawn to warn if it can't resolve a glpyh (or monkeypatch it)
* support web fonts; use uri-cache to avoid redundant fetching
* align caption to match alignment of block image
* attribute or role to control table shading (all, even, odd) (or call it striped like bootstrap?)
* make conum glyphs configurable in theme (use reference table to resolve)
* do we still need the converter hack in convert_content_for_block? (seems to be needed for admonitions)
* utility to coerce the color value transparent to nil (better handling in general)
* CJK and/or multilingual support (see https://github.com/chloerei/asciidoctor-pdf-cjk)
* description list term should keep together with content (file issue)
* hardbreak in table cell results in extra endline (likely not normalizing cell content)
* remove pdfmarks file after optimizing
* add note to README that Prawn will subset any fonts provided
* look into single_line + shrink_to_fit in listings, perhaps other places
* refactor as Prawn view to avoid method name conflicts (also see https://github.com/prawnpdf/prawn/issues/802)
* make CodeRay theme colors configurable (in theme?) (now that we have Rouge, this may be obsolete)
* create proper default (Asciidoctor) theme
* document how the treetop parser is rebuilt
* use ImageMagick to uncompress PNG images before reading them (could also just document this)
* rework font so we can set actual height, calculate x_height internally (use 1em for spacings)
* padding top and bottom on content affects height_of calculations (need to review)
* code font needs to support more than just ascii (Golo license block is an example)
* don't cutoff content in partintro
* admonition styles are one big hack; need to be organized and based on theme
* add admonition_label_font_color to theme
* autofit logic not working with Courier (still overrunning line)
* honor safe mode rules
* allow cover images to be specified by theme as a fallback
* stop using fallback fonts in default theme (instead, bundle a fuller font)
  - using fallback fonts significantly slows down Prawn because it checks every letter every time (see https://github.com/prawnpdf/prawn/blob/master/lib/prawn/text/formatted/box.rb#L427-L434)
* print scratch.pdf file if verbose / trace mode is on in Asciidoctor
* introduce setting to indent section content
* rename default theme to docbook theme, make default the Asciidoctor theme (should we have a base theme?)
* allow relative font size for inline code to be set (perhaps a percentage or em value? there are problems with this in arranger)
* set defaults in ThemeLoader for required theme settings like prose_margin_top/bottom so we don't need fallbacks in code
* implement orphan sentences for paragraph
* apply line height metrics for table content
  - figure out how to adjust line height for monospaced cell content
  - figure out how to layout regular cell content to adjust for line height
* document the typeset_text methods very clearly
* move check for node.title? inside layout_caption
* theme idea / tester: see sandbox/ebook-learn_version_control_with_git-SAMPLE.pdf
* make alternating page title position optional (via theme?)
* fix passthrough placeholders that get caught up in syntax highlighting (see https://github.com/asciidoctor/asciidoctor/blob/master/test/blocks_test.rb#L2258-L2277)
* add Preamble to TOC
* NOTE prawn-svg supports loading from a URI (only applies to embedded images); does it work in Asciidoctor PDF?
* honor font defs in SVG (to get M+ 1p); prawn-svg supports loading fonts; need to pass fonts to prawn-svg
* should we support % as a unit in theme (divides by 100 and sets float value)?
* disable monospace color in headings
* add source language to upper-left corner of listing block

* implement quote style from default Asciidoctor stylesheet
* reorganize Prawn extensions (see prawn-table for example)
* rename "theme" to "style"?
* restrict custom theme path to jail (or load from load_path)
* implement convert_toc
* only create title page if doctype=book
* italic text in a line of text styled as bold in the theme loses its bold style

* introduce method for start_initial_page?
* make outline a document option (perhaps "outline" like "toc")
* shrink / squeeze source code to avoid wrapping (see original impl in nfjsmag, also shrink_to_fit)
* add bench/ directory for the script to test the speed of the formatted text parser
* start page numbering on page 1 (use /PageLabels reference to make i the title page number)
  - add this feature upstream to Prawn
* *report image only page w/ stamps corruption issue to Prawn*
* add /PageMode /UseOutlines
* what does fopub do to calculate scaling images? reduces width more?
* replace tabs with spaces in source code (Asciidoctor core change?)
* preamble on separate page?
* part on separate page for book doctype? (which other sections?)
* make default image scale width a theme setting
* cli arguments
  - theme (pdf-style, pdf-stylesdir)
  - enable/disable writing pdfmarks file
  - optimize-pdf
* section numbering
* implement footnotes correctly
* table footer
* flesh out outline more
* flesh out title page more
  - document subtitle (partially solved)
* don't create title page for article doctype
* implement toc and activate if toc is set on document (need to reorder pages)
* inline image
* callbacks for title page, new part, new chapter, etc
* split out render methods for chapter, part, section, etc
* custom subs in verbatim blocks
* captions/titles on all blocks that support them
* make font size and character spacing scaling of inline code part of theme
* customizable character spacing
* might be able to avoid dry run for listing/literal in obvious cases
* implement index of index terms
* bw theme for CodeRay to match output of Pygments bw
* inline tabs should be replaced in layout_prose (etc) when normalize is enabled

* use treetop to parse and evaluate theme file
* make source code highlighting theme configurable (should be now, but has problems with conums); still?
* use or don't use pad method? check performance

== Major Milestones

* add a test suite
* refactor as Prawn View
* add support for AsciiDoc table cell content by writing custom Table::Cell implementation
* add support for nested tables by writing custom Table::Cell implementation
* add support for footnotes (as article or chapter endnotes)
* rework text handling in Prawn to support line height natively

== Documentation

* control page numbering using pagenums attribute
* "Incorrect number of arguments in 'SCN' command" happens when you add a stamp to an imported page
* be mindful that layout_prose adds margin to bottom of content by default (important when working in a bounding box)
* ttfunk does not support ligatures (e.g., fi -> fi); we could do this manually in post_replacements

== Notes

* when using `single_line: true` on formatted_text, it's necessary to reapply our padding top/bottom from line metrics
* we always leave cursor on start of page we're about to write on; certain checks rely on this fact
* "section title" is the semantic element; "heading" is the structural element
* /PageLabels/Nums must have entry for every page in front matter, even if a blank page
  - in fact, must account for every page or else numbering lags behind when scrolling document
* if we set the vposition on image to a numeric value, it skips the overrun check that happens internally
* any instance variables referenced by converter methods for inline nodes could get accessed before the converter for document is called
* Evince throws warning when printing PDF if & is used in document title; but this is valid according to the PDF specification
* Prawn drops fragments with empty text (hence the need to use zero-width space)
  - analyze_glyphs_for_fallback_font_support drops fragments with empty text
  - later on, initialize_wrap drops fragment with empty text
* use term "page number label" to refer to the visible, printed page number (not the implicit page number)

== Potential Optimizations

* if autofit is set on a listing/literal block that has conums, we are splitting fragments by line twice
* comparing > 0 is slightly faster than == 0 (for cases when we can swap the logic)

== Usage Optimizations

* uncompress PNG files to avoid slow zlib inflating step in Prawn
* flatten PNGs (remove alpha channel) since it messes up font rendering on the page in Adobe Acrobat Reader (need to verify)
* avoid the fallback font if possible (use full fonts in your theme) because it checks for *every* glyph
* font families used in SVGs must match keys in the font catalog

== Open Questions

== Implementation

* should we read SVG file using UTF-8 encoding; or does REXML handle encoding?
* can we leverage before_rendering_page callback on table?
* should we use move_past_bottom in some places instead of start_new_page?

=== Design

* remove/reduce padding above heading when it appears at the start of a page?
* Default line height?
* Should the heading sizes be calculated according to the default font size?
* Page margins
* Body indentation?
  - recto / verso indentation?
* Size of masthead / footer
* Line separating masthead / footer?
* Separate title page
* Start chapter on new page?
* Special layout for chapter page?

=== Theme

* keep or drop base_ prefix in theme? I think we should keep it because it provides context elsewhere in the document (e.g. $base_font_size vs $font_size)

== Resources

* https://code.google.com/p/origami-pdf/[Origami PDF: A PDF inspection library]
* https://github.com/a1ee9b/PrintPretty[A theme for PDF designed for printing]
* http://randomtextgenerator.com[Random Text Generator, supports multiple languages]
* http://clagnut.com/blog/2380[List of pangrams]
  - http://www.camcc.org/_media/reading-group/qianziwen-en.pdf[1,000 character classic (Chinese)]
* pdf2svg can convert the PDF file into an SVG (one SVG per page)
* https://blog.codeship.com/build-math-evaluation-engine[How to Build a Simple Math Evaluation Engine]
* http://blog.typekit.com/2011/11/03/optimizing-fonts-for-the-web-unicode-values-glyph-set-underlines-and-strike-through/[Optimizing Fonts for the Web]