Rendering R scripts using Jupyter `ir` kernels

[tzakharko]

Bug description

I am attempting to use quarto render to render an R script using the Jypyter engine as described in the documentation and encountering some issues. See also related discussion.

1. Quarto does not correctly auto-detect the Jupyter kernel for R. If no jupyter: kernel option is specified, it launches the python kernel but does not evaluate any code (cells are just passed through as markdown chunks). One has to specify jupyter: ir or jupyter: ark explicitly for the R code cells to be evaluated.
2. output: asis is not always correctly handled. For example, when used with kable(), the engine marks the cell output as "text/plain" rather than "asis", forcing it to be displayed verbatim and not as markdown. See this comment: Rendering R scripts using Jupyter kernels #12505 (comment)

Steps to reproduce

Render the following script using quarto render script.r

# %% [markdown]
# ---
# title: A test
# author: A reporter
# jupyter: ir
# ---

# %% [markdown]
# Sample markdown text

# %%
#| echo: false
1 + 1

# %%
#| echo: false
#| output: asis
knitr::kable(head(iris))

# %% [markdown]
# Some markdown

# %%
#| echo: false
#| fig-width: 4
#| fig-cap: A plot
ggplot2::ggplot(iris) + ggplot2::geom_point(ggplot2::aes(x = Sepal.Length, y = Sepal.Width, color = Species))

Actual behavior

Following html output is rendered (notice that the table has been rendered verbatim)

Notes:

• In the intermediate rendered markdown file, the second code cell output is wrapped in a backticks block, so it is passed verbatim instead of being interpreted as markdown. This is not the same behavior as when using a Quatro Notebook. Similarly, Quatro Notebook can detect and style kable() output directly, which does not happen when using the Jupyter kernel

• When using jupyter: ark, the figure in the code cell 3 is not sized correctly

Expected behavior

The same output as an equivalent qmd notebook, i.e.

Your environment

• OS: macOS 15.4
• Quarto CLI version 1.7.22

Quarto check output

Quarto check

Quarto 1.7.22
[✓] Checking environment information...
      Quarto cache location: ~/Library/Caches/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.6.3: OK
      Dart Sass version 1.85.1: OK
      Deno version 1.46.3: OK
      Typst version 0.13.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.7.22
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /Library/TeX/texbin
      Version: 2024

[✓] Checking Chrome Headless....................OK
      Chrome:  (not detected)

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.13.2
      Path: /opt/homebrew/opt/python@3.13/bin/python3.13
      Jupyter: 5.7.2
      Kernels: ir, ark, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.4.2
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
      knitr: 1.50
      rmarkdown: 2.29

[✓] Checking Knitr engine render......OK

Installed Jupyter kernels (all kernels verified to work )

Available kernels:
  ark              ~/Library/Jupyter/kernels/ark
  ir                 ~/Library/Jupyter/kernels/ir
  python3     /usr/local/share/jupyter/kernels/python3

[tzakharko]

I would be happy to assist with debugging, it would be great if a contributor could point me to a relevant entry point. I see a markdownFromJupyterPercentScript(), but it not quite clear to me where it gets invoked and what happens to the markdown representation next.

[mcanouil]

For reference, my quick look at this from the discussion:

• Rendering R scripts using Jupyter kernels #12505 (comment)

Note that I get the exact same output using Quarto document or R script.

@tzakharko Are you sure the ark kernel was used for your second screenshot?

I've just tried a plain Quarto document. Quarto does detect the kernel (quarto check) but don't use it, so it has to be set explicitly set in the document.

---
title: "Reproducible Quarto Document"
format: html
engine: jupyter
jupyter: ark
---

```{r}
x <- c(1, 2, 3, 4, 5)
y <- c(1, 4, 9, 16, 25)

plot(x, y)

#| output: asis
cat("This is a **markdown** document.")

```shell
Quarto 1.7.21
[✓] Checking Python 3 installation....OK
      Version: 3.13.2
      Path: /usr/local/python/current/bin/python3
      Jupyter: 5.7.2
      Kernels: julia-1.11, ark, python3

[✓] Checking Jupyter engine render....OK

output: asis seems to work properly.

Same behaviour using an R script.

# %% [markdown]
# ---
# title: "Reproducible Quarto Document"
# format: html
# engine: jupyter
# jupyter: ark
# keep-ipynb: true
# ---

# %%
x <- c(1, 2, 3, 4, 5)
y <- c(1, 4, 9, 16, 25)

plot(x, y)

# %%
#| output: asis
cat("This is a **markdown** document.")

Note that I can reproduce the markdown table output from knitr::kable() not being rendered as markdown. To me the issue, is not the output: asis, but the information in the Jupyter Notebook.

I've simplified the code to avoid the base64 encoded image.

# %% [markdown]
# ---
# title: A test
# engine: jupyter
# jupyter: ark
# keep-ipynb: true
# ---

# %%
#| echo: false
#| output: false
library(knitr)

# %%
#| echo: false
#| output: asis
knitr::kable(head(iris))

# %%
#| echo: false
#| output: asis
cat("Some **markdown** text _formatted_ with `rmarkdown` and **knitr**.")

The notebook:

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "48300306",
   "metadata": {},
   "source": [
    "---\n",
    "title: A test\n",
    "engine: jupyter\n",
    "jupyter: ark\n",
    "keep-ipynb: true\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "207f446e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- Project '/workspaces/quarto-codespaces' loaded. [renv 1.1.4]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- The project is out-of-sync -- use `renv::status()` for details.\n"
     ]
    }
   ],
   "source": [
    "#| echo: false\n",
    "#| output: false\n",
    "library(knitr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "26669104",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "\n",
       "| Sepal.Length| Sepal.Width| Petal.Length| Petal.Width|Species |\n",
       "|------------:|-----------:|------------:|-----------:|:-------|\n",
       "|          5.1|         3.5|          1.4|         0.2|setosa  |\n",
       "|          4.9|         3.0|          1.4|         0.2|setosa  |\n",
       "|          4.7|         3.2|          1.3|         0.2|setosa  |\n",
       "|          4.6|         3.1|          1.5|         0.2|setosa  |\n",
       "|          5.0|         3.6|          1.4|         0.2|setosa  |\n",
       "|          5.4|         3.9|          1.7|         0.4|setosa  |"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#| echo: false\n",
    "#| output: asis\n",
    "knitr::kable(head(iris))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9b0cc8e0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Some **markdown** text _formatted_ with `rmarkdown` and **knitr**."
     ]
    }
   ],
   "source": [
    "#| echo: false\n",
    "#| output: asis\n",
    "cat(\"Some **markdown** text _formatted_ with `rmarkdown` and **knitr**.\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Ark R Kernel",
   "language": "R",
   "name": "ark",
   "path": "/home/vscode/.local/share/jupyter/kernels/ark"
  },
  "language_info": {
   "file_extension": ".R",
   "mimetype": "text/r",
   "name": "R",
   "positron": {
    "continuation_prompt": "+ ",
    "input_prompt": "> "
   },
   "version": "R version 4.4.3 (2025-02-28)"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

Environnement:

• ark-0.1.177-linux-x64.zip
• Codespace https://github.com/mcanouil/quarto-codespaces

[tzakharko]

@tzakharko Are you sure the ark kernel was used for your second screenshot?

If by the second screenshot you mean the one with correctly rendered table, that was done in RStudio using an equivalent RMarkdown notebook.

I can replicate your observed behavior with output: asis (forwarded with simple cells, replaced by "text/plain" for kable()). So this seems to be the main issue.

There is still a difference between ir and ark in how they detect figure dimensions, but since you already clarified that these should be specified at the document level, my usage was invalid anyway.

[mcanouil]

If by the second screenshot you mean the one with correctly rendered table, that was done in RStudio using an equivalent RMarkdown notebook.

What do you call "RMarkdown notebook"?

See below what I have regardless of the IDE being used as it's from the terminal using quarto render file.r/qmd --execute

Using Quarto 1.7.21 and ark 0.1.177

Input	Output
# %% [markdown] # --- # title: A test # format: html # engine: jupyter # jupyter: ark # keep-ipynb: true # --- # %% #| echo: false #| output: false library(knitr) # %% #| echo: true #| output: asis knitr::kable(head(iris)) # %% #| echo: false #| output: asis cat("Some **markdown** text _formatted_ with `rmarkdown` and **knitr**.")
--- title: "Reproducible Quarto Document" format: html engine: jupyter jupyter: ark keep-ipynb: true --- ```{r} #| echo: false #| output: false library(knitr) ``` ```{r} #| echo: true #| output: asis knitr::kable(head(iris)) ``` ```{r} #| echo: false #| output: asis cat("Some **markdown** text _formatted_ with `rmarkdown` and **knitr**.") ```

If you are not convinced, below is the diff between the two Jupyter notebooks, as you can see they are the same.

There is still a difference between ir and ark in how they detect figure dimensions, but since you already clarified that these should be specified at the document level, my usage was invalid anyway.

Quarto does not control what the kernels do. It uses what they produce.

[tzakharko]

This is rendered using default QMD R engine, which I assume is knitr.

[mcanouil]

Indeed, so of course you don't get the same thing if the engine is knitr.
Let's keep this issue on the engine: jupyter side. (Jupyter does not behave like knitr for many reasons beyond Quarto)

[tzakharko]

I had a quick look at the code. The issue appears to be the design of mdOutputDisplayData(). It seems that any text output is wrapped in a markdown code block, regardless of the display option. Furthermore, it does not seem that the asis flag is forwarded to mdOutputDisplayData() at all.

Producing markdown with cat() and friends works because this output is marked as "stream" output and emitted directly.

Proposed fix: at the very least, mdOutputDisplayData() should be able to detect if the output is asis so that it can emit plain text output as markdown and not as verbatim content. This can be also done at a higher level by overriding "text/plain" as markdown (e.g. in mdFromCodeCell())

[cderv]

Regarding output: asis, its behavior in Jupyter context is described at
https://quarto.org/docs/computations/execution-options.html#raw-output

• output: asis works with explicit print() or cat() in Jupyter
• Or cell output need to be marked as Markdown (e.g. using IPython.display.Markdown)

So it is expected that mdOutputDisplayData() does not handle asis currently.

The expected behavior when using the R kernel is the same as that of a Python kernel.

• Use cat() (as you would do in knitr engine as described in same link as above)

• Or use the IRdisplay package (https://cran.r-project.org/package=IRdisplay), equivalent to IPython.display (see https://github.com/IRkernel/IRdisplay/blob/master/README.md)

   ---
   jupyter: ir
   keep-ipynb: true
   ---
   
   ```{r}
   library(IRdisplay)
   
   knitr::kable(head(iris)) |> paste0(collapse = "\n") |>  display_markdown()
   ```

  

As I expected, knitr is not be fully compatible to be used in Jupyter context. knitr was designed to be used on its own, or in R Markdown and now Quarto, but I don't know how it behaves when used in Jupyter. And it seems it would benefit some better support to improve above behavior.

Maybe there are way, knitr could make IRdisplay aware of some outputs. As of now IRdisplay is not way to knitr_kable output. For knitr engine, knit_print knows about kable(), but for Jupyter engine, display does not know about it. That is the main problem.

It could also be something at the Quarto or knitr + quarto level so that Quarto processing detects this type of output.

Anyhow, for R cell, quarto works best with engine: knitr for now.

Hope this helps clarify and see the context of all this.

Some more notes about what I looked into.

Regarding knitr output in ipynb cell, This is what I get in Positron IDE using R kernel

and in jupyter notebook using ir kernel

Regarding what quarto sees from notebook execution

---
jupyter: ir
keep-ipynb: true
---

```{r}
knitr::kable(head(iris))
```

we get only one display_data output, marked as text/plain (and not text/markdown)

  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cb268274",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "\n",
       "| Sepal.Length| Sepal.Width| Petal.Length| Petal.Width|Species |\n",
       "|------------:|-----------:|------------:|-----------:|:-------|\n",
       "|          5.1|         3.5|          1.4|         0.2|setosa  |\n",
       "|          4.9|         3.0|          1.4|         0.2|setosa  |\n",
       "|          4.7|         3.2|          1.3|         0.2|setosa  |\n",
       "|          4.6|         3.1|          1.5|         0.2|setosa  |\n",
       "|          5.0|         3.6|          1.4|         0.2|setosa  |\n",
       "|          5.4|         3.9|          1.7|         0.4|setosa  |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "knitr::kable(head(iris))"
   ]
  }

Using display_markdown() does the expected change for text/markdown

  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2a263867",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "| Sepal.Length| Sepal.Width| Petal.Length| Petal.Width|Species |\n",
       "|------------:|-----------:|------------:|-----------:|:-------|\n",
       "|          5.1|         3.5|          1.4|         0.2|setosa  |\n",
       "|          4.9|         3.0|          1.4|         0.2|setosa  |\n",
       "|          4.7|         3.2|          1.3|         0.2|setosa  |\n",
       "|          4.6|         3.1|          1.5|         0.2|setosa  |\n",
       "|          5.0|         3.6|          1.4|         0.2|setosa  |\n",
       "|          5.4|         3.9|          1.7|         0.4|setosa  |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }

This display_markdown is the right way to format the Markdown table in output as this is even what to do in Jupyter notebook.

[tzakharko]

Thank you for clarifying, @cderv. It would be great if the documentation can be updated to highlight these two points:

• For asis machinery to work, one should explicitly print the data and not rely on implicit printing
• Using IRdisplay package when working with the Jupiter R kernel

On that note, is there a standard, documented way to detect execution from within a Jupyter kernel? IRdisplay seems to rely on the jupyter.base_display_func option to be set. But ark, for example, does not appear to support any of the related options.

To anticipate the likely comment, while knitr is the established and well-supported engine, I think there is also a lot of interest in Jupyter as an industry-standard interface. Modern editors now support cell syntax and inline execution of code, which is great for incremental development for all types of codes. And Posit in particular does a lot of great work in bringing first-class Jupiter kernel support and tooling for R users. This seems like a great opportunity to ensure a good foundation going forward.