Make `doom +org tangle` CLI load any available `ob-<lang>` libraries for the source langs of noweb-addressable blocks by ambirdsall · Pull Request #8406 · doomemacs/doomemacs · GitHub | Latest TMZ Celebrity News & Gossip | Watch TMZ Live
Skip to content

Make doom +org tangle CLI load any available ob-<lang> libraries for the source langs of noweb-addressable blocks #8406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Make doom +org tangle CLI load any available ob-<lang> libraries for the source langs of noweb-addressable blocks #8406

wants to merge 1 commit into from

Conversation

ambirdsall
Copy link
Contributor

@ambirdsall ambirdsall commented May 29, 2025 •
edited by hlissner
Loading

the issue here

It's a three-prong problem:

  • Doom CLI functions like doom +org tangle run in a minimal environment in which only strictly necessary libraries are preloaded
  • tangling a document which contains noweb-enabled blocks may trigger an org-babel evaluation of any noweb-addressable block
  • any such org-babel evaluation will cause tangling to fail with a rather loud error if the requisite ob-<language> library has not been loaded

Put it all together and, well, 💥

how to fix that

This updates the code inside tangle CLI's org-babel-tangle-collect-blocks function; its per-block let* form gets a new src-noweb-ref binding to store noweb-addressable labels (i.e. any #+NAME: or :noweb-ref associated with a block) when iterating through the source document's src blocks; when such a label and a corresponding ob-<labeled src block's lang> library can be found, it requires the library, making it available for future noweb expansion.

no pre-existing issues or PRs for this

A search for tangle cli in the repo's issues shows no results of any sort besides #4273, which is pretty specifically centered on the config bootstrapping behavior of doom {install,sync}.

  • I searched the issue tracker and this hasn't been PRed before.
  • My changes are not on the do-not-PR list for this project.
  • My commits conform to Doom's git conventions.
  • Any relevant issues or PRs have been linked to.

how I found it, plus a minimal repro and some test output in a dropdown

I personally ran into this when experimenting with using doom +org tangle to automate generating some dotfiles; it was specifically triggered by a kmonad config I tangled from an org file so I could git-track and share a single, hardware-independent1 source of truth between an old macbook and a thinkpad while still dynamically generating a hardware-specific kmonad.kbd for each.

To reproduce the bug I made this minimal version of the offending kmonad literate config:

#+name: keyboard-device-file
#+begin_src shell :tangle no :results tangle
find /dev/input/by-{path,id} -name '*-kbd' | head -n 1 | tr -d "\n"
#+end_src

#+begin_src kbd :tangle kmonad.kbd :noweb yes
(defcfg
  input (device-file "<<keyboard-device-file()>>")
  output (uinput-sink "My KMonad output")
  fallthrough true)

(defsrc caps)

(deflayer qwerty @cap)

(defalias cap (tap-hold-next 333 esc lctl))
#+end_src

I put those lines in a file—I used /tmp/org/test.org—and tried tangling it via CLI on both the master and require-all-potentially-needed-ob-libraries-in-org-tangle-cli branches2. I've included the shell output of my own test run below:

Shell output of manually reproducing the bug and testing the fix (not in that order, though) 
┌ ~/.emacs.d [require-all-potentially-needed-ob-libraries-in-org-tangle-cli|✔] d1ff35bb6 🌑
└➣ ls /tmp/org
test.org

┌ ~/.emacs.d [require-all-potentially-needed-ob-libraries-in-org-tangle-cli|✔] d1ff35bb6 🌑
└➣ doom +org tangle /tmp/org/test.org                                                                                
> Reading /tmp/org/test.org...
  ✓ Tangled to /tmp/org/kmonad.kbd

┌ ~/.emacs.d [require-all-potentially-needed-ob-libraries-in-org-tangle-cli|✔] d1ff35bb6 🌑
└➣ cat /tmp/org/kmonad.kbd                                                                                           
(defcfg
  input (device-file "/dev/input/by-path/pci-0000:00:14.0-usb-0:5.1:1.2-event-kbd")
  output (uinput-sink "My KMonad output")
  fallthrough true)

(defsrc caps)

(deflayer qwerty @cap)

(defalias cap (tap-hold-next 333 esc lctl))

┌ ~/.emacs.d [require-all-potentially-needed-ob-libraries-in-org-tangle-cli|✔] d1ff35bb6 🌑
└➣ rm /tmp/org/kmonad.kbd

┌ ~/.emacs.d [require-all-potentially-needed-ob-libraries-in-org-tangle-cli|✔] d1ff35bb6 🌑
└➣ git checkout master                                                                                                               
Switched to branch 'master'
Your branch is up to date with 'upstream/master'.

┌ ~/.emacs.d [master|✔] 8406c1ff2 🌑
└➣ !ls                                                                                                               
ls /tmp/org
test.org

┌ ~/.emacs.d [master|✔] 8406c1ff2 🌑
└➣ doom +org tangle /tmp/org/test.org                                                                                
> Reading /tmp/org/test.org...

Error: error ("No org-babel-execute function for shell!")
  error("No org-babel-execute function for %s!" "shell")
  org-babel-execute-src-block(nil nil ((:results . "none")))
  org-babel-ref-resolve("keyboard-device-file()")
  
  [...SKIPPING PASS THE COLOSSAL ELISP STACK TRACE...]
  
  x There was an unexpected runtime error
  Message: No org-babel-execute function for shell!
  Backtrace:
    (error "No org-babel-execute function for %s!" "shell")
    (org-babel-execute-src-block nil nil ((:results . "none")))
    (org-babel-ref-resolve "keyboard-device-file()")
    (#[257 "r\303q\210\306 \307\310\"\216\311\312\"\311\313\"\314\315\316\317$\266\203\211\203H\32...
    (replace-regexp-in-string "\\(.*?\\)\\(<<\\([^ 	\n]\\(?:.*?[^ 	\n]\\)?\\)>>\\)" #[257 "r\303q\21...
    (org-babel-expand-noweb-references ("kbd" "(defcfg\n  input (device-file \"<<keyboard-device-file()>>\")\n  ou...
    (org-babel-tangle-single-block 2)
    (let* ((block (org-babel-tangle-single-block counter)) (src-tfile (cdr (assq :tangle (nth 4 block)))) (file-na...
    (cond ((member "notangle" tags)) ((let* ((tags (seq-group-by #'(lambda (%) (equal (car %) "--or")) tags)) (or-...
    (let* ((tags (org-get-tags-at)) (info (org-babel-get-src-block-info 'no-eval)) (src-lang (nth 0 info)) (src-tf...
    (if (or (org-in-commented-heading-p) (org-in-archived-heading-p)) nil (let* ((tags (org-get-tags-at)) (info (o...
    (let ((full-block (match-string 0)) (beg-block (match-beginning 0)) (end-block (match-end 0)) (lang (match-str...
GNU Emacs     v30.0.93         ce50a1d3c18bcf0e5f51f4ed49f292f7be31010d
Doom core     v3.0.0-pre       HEAD -> master, upstream/master, upstream/HEAD 8406c1ff2 2025-05-24 17:32:27 +0200
Doom modules  v25.06.0-pre     HEAD -> master, upstream/master, upstream/HEAD 8406c1ff2 2025-05-24 17:32:27 +0200
  ! Wrote extended backtrace to ~/.emacs.d/.local/state/logs/cli.doom.250529140816.50920.error

Footnotes

  1. well, the hardware has to run linux ↩

  2. a fun sideplot to dipping into CLI code is that the same sandboxed environment that makes doom CLI's startup time doom-config-independent means there's no need to wait on any doom sync runs when A/B testing a change. ↩

@ambirdsall ambirdsall marked this pull request as ready for review May 29, 2025 23:18
@ambirdsall ambirdsall requested a review from a team as a code owner May 29, 2025 23:18
@ambirdsall ambirdsall changed the title fix: Load ob-<lang> libs for noweb-usable blocks Make doom +org tangle CLI load any available ob-<lang> libraries for the source langs of noweb-addressable blocks May 29, 2025
@hlissner hlissner added is:bug Something isn't working as intended module:lang/org Pertains to Doom's :lang org module labels May 31, 2025
@hlissner hlissner added this to the modules v25.07 milestone May 31, 2025
hlissner
hlissner requested changes Jul 1, 2025
View reviewed changes
modules/lang/org/cli.el Outdated
;; evaluated even if it will not be tangled per se
(if-let ((src-noweb-ref)
(ob-library-name (concat "ob-" src-lang))
(locate-library ob-library-name))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is binding ob-library-name to locate-library, it isn't calling the locate-library function. Wrap in another set of parentheses:

                     (if-let ((src-noweb-ref)
                              (ob-library-name (concat "ob-" src-lang))
-                             (locate-library ob-library-name))
+                             ((locate-library ob-library-name)))
                         (require (intern ob-library-name)))

modules/lang/org/cli.el Outdated
(cdr (assq :noweb-ref (nth 2 info))))))
;; a block which can be referenced via noweb may need to be
;; evaluated even if it will not be tangled per se
(if-let ((src-noweb-ref)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please swap out if-let with when-let* when the else branch isn't used.

@ambirdsall
fix: Load ob-<lang> libs for noweb-usable blocks
9802a65 
For performance reasons, Doom CLI runs in a minimal environment wherein
no `ob-<language>` libraries are initially loaded; but tangling a
document with noweb-enabled blocks can trigger an org-babel evaluation
of any noweb-addressable block; and any such evaluation will fail
tangling with an error unless the correct `ob-<language>` library has
been loaded.

So. This changes the tangle CLI function to note any noweb-addressable
labels (i.e. any `#+NAME:` or `:noweb-ref` associated with a block) when
iterating through the source document's blocks; for each block where one
is found, it conditionally attempts to `require` the corresponding
`ob-<src-lang>` library.
@ambirdsall ambirdsall force-pushed the require-all-potentially-needed-ob-libraries-in-org-tangle-cli branch from d1ff35b to 9802a65 Compare July 1, 2025 21:23
@ambirdsall
Copy link
Contributor Author

Good catches both! I tested some noweb-injected content with #+begin_src welp, and sure enough, the top-line error improved:

# with the original `(locate-library ob-library-name)`:
Error: file-missing ("Cannot open load file" "No such file or directory" "ob-welp")

# with the fixed `((locate-library ob-library-name))`:
Error: error ("No org-babel-execute function for welp!")

Fixed up the commit and re-pushed.

@ambirdsall ambirdsall requested a review from hlissner July 1, 2025 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hlissner hlissner Awaiting requested review from hlissner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
is:bug Something isn't working as intended module:lang/org Pertains to Doom's :lang org module
Projects
None yet
Milestone
modules v25.07
Development

Successfully merging this pull request may close these issues.
2 participants
@ambirdsall @hlissner

TMZ Celebrity News – Breaking Stories, Videos & Gossip

Looking for the latest TMZ celebrity news? You've come to the right place. From shocking Hollywood scandals to exclusive videos, TMZ delivers it all in real time.

Whether it’s a red carpet slip-up, a viral paparazzi moment, or a legal drama involving your favorite stars, TMZ news is always first to break the story. Stay in the loop with daily updates, insider tips, and jaw-dropping photos.

🎥 Watch TMZ Live

TMZ Live brings you daily celebrity news and interviews straight from the TMZ newsroom. Don’t miss a beat—watch now and see what’s trending in Hollywood.