Filters: spellcheck.lua example fails (bug?)

[HenrikBengtsson]

Description

TL;DR

When following the filter example on https://quarto.org/docs/extensions/filters.html#activating-filters, quarto render fails to generate HTML with an error.

Reproducible example

$  quarto create project website "quarto-filter-test" "test" --no-prompt --no-open
Creating project at /home/hb/quarto-tests/quarto-filter-test:
  - Created _quarto.yml
  - Created index.qmd
  - Created about.qmd
  - Created styles.css

Then add "filters" to header per https://quarto.org/docs/extensions/filters.html#activating-filters;

$ cd /home/hb/quarto-tests/quarto-filter-test
$ cat index.qmd 
---
title: "test"
filters:
  - spellcheck.lua
---

This is a Quarto website.

To learn more about Quarto websites visit <https://quarto.org/docs/websites>.

Download spellcheck.lua;

$ curl --silent -L -O https://raw.githubusercontent.com/pandoc/lua-filters/refs/heads/master/spellcheck/spellcheck.lua
$ head -n 2 spellcheck.lua 
-- lua filter for spell checking: requires 'aspell'.
-- Copyright (C) 2017-2020 John MacFarlane, released under MIT license
$  md5sum spellcheck.lua 
fb35848ec57f789795f7021d109ab25e  spellcheck.lua

Then, trying to render gives an error:

$ quarto render
[1/2] about.qmd
[2/2] index.qmd
ERROR: NotFound: No such file or directory (os error 2): readfile '/home/hb/quarto-tests/quarto-filter-test/index.html'
Path: /home/hb/quarto-tests/quarto-filter-test/index.html

Stack trace:
    at readTextFileSync (ext:deno_fs/30_fs.js:864:10)
    at Object.Deno.readTextFileSync (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:5291:25)
    at file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:79666:36
    at withTimingAsync (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:17114:31)
    at runHtmlPostprocessors (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:79664:15)
    at Object.complete (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:79502:49)
    at Object.onPostProcess (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:87114:53)
    at renderFileInternal (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:87098:26)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async renderFiles (file:///home/hb/shared/software/CBI/quarto-1.7.33/bin/quarto.js:86910:17)

It renders just fine if I remove the filters declaration;

$ quarto render
[1/2] about.qmd
[2/2] index.qmd

Output created: _site/index.html

Session info

$ quarto --version
1.7.33

$ pandoc --version
pandoc 3.7.0.2
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: /home/hb/.local/share/pandoc
Copyright (C) 2006-2024 John MacFarlane. Web: https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

$ aspell --version
@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.8.1)

[HenrikBengtsson]

I get the same if I try with:

filters:
  - spellcheck.lua
  - quarto

and

filters:
  - quarto
  - spellcheck.lua

[mcanouil]

I'm not sure this filter is quite up to date with Pandoc API
Could you try to run it via Pandoc:

quarto pandoc --lua-filter spellcheck.lua --from markdown index.qmd

[HenrikBengtsson]

It's nice to use spelling as an introduction to filters.

I agree having aspell as a requirement adds a hurdle. I've also tried to figure out how to use a project/folder specific .aspell.en.pws dictionary, and it turns out to be more tricky than I thought. Having a quarto project rely on ~/.aspell.en.pws complicates things, e.g. you have to instruct collaborators to update their personal dictionary.

Not sure how tricky it is to call Rscript from a Pandoc filter, but using something like spelling::spell_check_text() would be natural to R users. OTH, it introduces R as a dependencies.

Maybe something completely different could be used for a first filter example. Maybe a list-toc.lua filter that outputs all H1, H2, ..., headers and possibly also any HTML attributes associated with them. That might help share the gist of how filters work. Another one along this line would be a title-case-headers.lua filter that converts headers into Title Case (but that would requires something like tools::toTitleCase()).

[mcanouil]

Not sure how tricky it is to call Rscript from a Pandoc filter, but using something like spelling::spell_check_text() would be natural to R users. OTH, it introduces R as a dependencies.

Not too hard:

local output = pandoc.pipe("program_name", {"arg1", "arg2"}, "input text")

There is even a Quarto Lua API helper function: quarto.paths.rscript() (https://quarto.org/docs/extensions/lua-api.html#paths)

[mcanouil]

There is quite a wide variety of filters already out-there.
You can take a look (self-promotion) on my extension listing: https://m.canouil.dev/quarto-extensions/

[tarleb]

That's super weird. @tarleb, What's the expected behavior for os.exit(0) in Lua filters?

To terminate the whole program, i.e., pandoc.

The filter was one of the first Lua filters ever written, and I agree that's it's not a good example. The filter should either be a custom writer so that os.exit becomes unnecessary, or should just not call os.exit.

[cscheid]

I apologize. @tarleb, when I wrote that message to you, I believed to have a setup where pandoc was not actually terminating the process on os.exit(0), which was very weird. But I can no longer reproduce that behavior.

In that case, I stand by my original claim that spellcheck.lua is either buggy or not a filter we should use in our demos. (and I bugged Albert for no good reason 🤦)