Not so long ago (if you live in year 2025) I have finally managed to get hooked on Emacs. This was my 3rd attempt to migrate from VS Code. This time we can call it a success! It can be attributed to System Crafters and particularly Emacs From Scratch series. Bit by bit I got comfortable enough to even start hacking my own little things in Emacs. I went on a lot of tangents including picking up Guile or trying to implement my own Scala source block for Org Babel but the most damaging (to my free time) was my discovery of Denote.

For uninitiated, Denote is a note taking system (but not only) for Emacs, created by Protesilaos Stavrou. I don't know how anyone would land on this page without prior knowledge of Denote, but no shame in that 😆. I recommend reading Denote Overview and perhaps some of SystemCrafters videos on YouTube might interest you. And of course Protesilaos talks about Denote himself on his channel.

Even though I have started using Emacs for programming, I think I knew that what I really want from it is a tool to organise my life. I spread myself thin across too many hobbies/projects and tend to have multiple browser windows open containing hundreds of tabs. Now I can file everything in Denote and forget about it till I can allocate time for a particular research. No other personal wiki or brain-mapping software have ever had so much success with me. 🥳

Now let's get closer to the actual "why" part for this project. It was SystemCrafters again. I watched a stream on Generating a Blog from Denote entries and thought, that would be cool! The stream didn't end with a complete solution though. So I decided that this can be a nice project to learn more about Emacs/Org programming, and so here we are… months later… 😅

Given a directory with Denote notes, take a subset of notes meant to be published and generate a static site from them.

If I have learnt anything from the SystemCrafters' stream is that there is unlikely to be a clean integration with Org Export/Publish system. To limit the amount of problems, supporting region/buffer/file exports without setting up an Org Publish¹ project is out of scope.

One ASCII tree is worth a thousand words, this is what we are after.

website
├── index.html
├── media
│   ├── blog_image.png
│   └── favicon.png
├── robots.txt
├── styles
│   ├── code-dark.css
│   ├── code-light.css
│   └── site.css
├── post1
│   ├── index.html
│   └── plot1.png
├── post2
│   └── index.html
└── post3
    └── index.html

Nothing unexpected, the tricky part is to produce it from Denote notes with Org Publish… To keep things sane, here are the rules for the solution:

1. Every file that is to become a page/post, gets its own directory under the site root. Directories are named after the title part of the full Denote filename. We achieve two goals with this:
   1. Nice looking, short URLs without any extra HTTP server work. Just check your URL address bar. 😆
   2. Create a place for page-specific resources. I assume pages are going to have dynamically generated data from code blocks. I like to play with Jupyter² and Julia³ source blocks, so this is a very common scenario for me.
2. If a link points to a media file, we will rewrite its URL to point to a shared location identified by a user defined variable (/media in the tree above). We need to handle both denote: and file: schemes, the first one is obvious, the latter can point to a Denote resource e.g. an inline image. There is an exception though, we can also encounter files coming from the Sitemap⁴ generator. Therefore, for file: scheme the rules are:
   • Page-generated resource - set a link to be relative to the source page e.g. ./plot1.png.
   • Sitemap - we can't know who/what generated a link, but if we detect that it points to a page, generate an absolute URL to a page e.g. /post1.
   • For all other cases act according to :with-broken-links⁵ export setting. Basically, this is an error state, we assume all files in a Denote directory are Denote resources or page-generated files.
3. My Denote files have mostly flat hierarchy. I don't want to have complex mapping rules even though having a bit of nesting certainly helps with the site navigation/structure. I want to keep the project scope under control, to be addressed later. To summarise:
   • Assume Denote source directory is flat, at least the parts that are meant to be published.
   • Full denote filename is quite long, contains a timestamp and therefore conflicts are unlikely. That is definitely not the case for the page names we have here. If a conflict happens, it is a user error. ❤️

The function below handles all the link rules we stated above. Just like that, half of the project is done already! 🎊 It took me a month of research to write though! 🤪 Org file AST, Org Element API, just regular elisp troubles… 100 open tabs in my browser burning my CPU to ashes…

Anyway, to the function:

(org-export-define-derived-backend 'ky-html 'html
  :translate-alist
  `((link . ky--publish/html-link)))

(defun ky--publish/html-link (link desc info)
  "Link transcoder for custom denote/file link handling.

See `org-html-link' for LINK/DESC/INFO description."
  (let* ((link-type (org-element-property :type link))
         (link-path (org-element-property :path link))
         (file-path (if (string= "denote" link-type)
                        (denote-get-relative-path-by-id link-path)
                      link-path))
         (file-name (file-name-nondirectory file-path))
         (file-ext (file-name-extension file-name))
         (denote-file-title (denote-retrieve-filename-title file-path)))
    (pcase link-type
      ("denote"
       (let* ()
         (pcase (ky--publish/classify-denote-file file-name)
           ;; any page uses absolute URL e.g. `/page1'
           ('page (ky--publish/make-link (format "/%s" denote-file-title) desc))
           ;; any site resources go into shared resource directory e.g. `/media/<file>'
           ('media (ky--publish/make-link
                    (format "/%s/%s.%s"
                            ky-publish/media-destination-dirname
                            denote-file-title
                            file-ext)
                    desc))
           ('no-access (ky--publish/handle-no-access-file file-path info))
           (_ (ky--publish/handle-unknown-file file-path info)))))
      ("file"
       (if denote-file-title
           ;; If we match Denote extension (e.g. org) - assume it is a link to a post otherwise
           ;; treat as a shared resource - set href to the resource directory. Links have to be
           ;; relative, otherwise html backend will create a `file:' link. We don't want to handle
           ;; potentially inlined resources, let 'html deal with this.
           ;;
           ;; NOTE: perhaps mutating `link' is a bit dirty, but `org-element-copy' doesn't copy
           ;; properties set on a link e.g. `ATTR_HTML'
           (if (string= file-ext (or denote-file-type "org"))
               (ky--publish/make-link (format "/%s" denote-file-title) desc)
             (let ((publish-path (format "../%s/%s.%s"
                                         ky-publish/media-destination-dirname
                                         denote-file-title
                                         file-ext)))
               (org-element-put-property link :path publish-path)
               (org-export-data-with-backend link 'html info)))
         ;; a link to a regular file
         (let ((publish-dir (ky-publish/destination-directory)))
           (if (file-in-directory-p file-path publish-dir)
               ;; file is already in the publishing-dir, assume this is a dynamic resource generated
               ;; by a page with `ky-publish/dresource', form links relative to containing page:
               ;; <publish-dir>/<page>/<resource-path> -> ./<resource-path>
               (let ((publish-path
                      (apply
                       'file-name-concat
                       (cdr (file-name-split (file-relative-name file-path publish-dir))))))
                 (org-element-put-property link :path publish-path)
                 (org-export-data-with-backend link 'html info))
             ;; otherwise bail out
             (ky--publish/handle-unknown-file file-path info)))))
      (_ (org-export-data-with-backend link 'html info)))))

Here we first check if we got a denote: link, if so, we get some information about the file it points to and decide how to generate an HTML link based on the result from ky-publish/classify-denote-file.

(defun ky--publish/classify-denote-file (file-name)
  "Determine how FILE-NAME should be published.
Returns one of the following: 'page, 'media, 'no-access,
nil"
  (when (denote-file-has-identifier-p file-name)
    (let ((classify (lambda (regexp)
                      (ky--publish/denote-file-accesible-p file-name regexp))))
      (cond
       ((funcall classify (ky-publish/denote-page-regexp)) 'page)
       ((funcall classify (ky-publish/denote-media-regexp)) 'media)
       ('no-access)))))

The function takes a couple of predefined rules to sort files into 4 categories:

1. page - that's a post.
2. media - anything else that matches the regexp, a site resource.
3. no-access - any private resource that is not meant for publishing.
4. nil - not a Denote file.

And a couple of helper functions to check file access, generate HTML link elements and handle 'no-access and unknown links.

(defun ky-publish/filter-denote-files (regexp)
  "Get a list of denote formatted filenames filtered by REGEXP."
  (mapcar
   'denote-get-file-name-relative-to-denote-directory
   (denote-directory-files regexp)))

(defun ky--publish/denote-file-accesible-p (file-name regexp)
  "Use REGEXP to determine if FILE-NAME can be published.
See `ky-publish-vars.el' for REGEXP definitions."
  (member file-name (ky-publish/filter-denote-files regexp)))

;; TODO: handle IDs
(defun ky--publish/make-link (href desc)
  "Create <a> element pointing to HREF with description DESC."
  (format "<a href=\"%s\">%s</a>" href desc))

(defun ky--publish/handle-no-access-file (file-path info)
  "Handle no-access links according to settings in INFO.
FILE-PATH is a relative path to a file returned by Denote."
  (pcase (plist-get info :with-broken-links)
    (`nil (user-error "Unable to resolve link for: %s, no access" file-path))
    (`mark (format "<span class=\"no-access-link\">[NO ACCESS: %s]</span>" file-path))
    (_ nil)))

(defun ky--publish/handle-unknown-file (file-path info)
  "Handle links to unknown files according to settings in INFO.
FILE-PATH is a relative path to a file returned by Denote."
  (pcase (plist-get info :with-broken-links)
    (`nil (user-error "File does not match any type: %s" file-path))
    (`mark (format "<span class=\"unknown-link\">[UNKNOWN FILE: %s]</span>" file-path))
    (_ nil)))

If we take a look at ox-html we can see a couple of functions that provide export and publish functionality:

1. org-html-export-as-html exports current buffer to an HTML buffer.
2. org-html-convert-region-to-html converts selected region of an Org buffer into HTML in-place.
3. org-html-export-to-html export current buffer to an HTML file.
4. org-html-publish-to-html publishes a supplied Org file.

We are after org-html-publish-to-html, that's the one we will use with Org Publish. org-html-export-to-html is good for quick testing though, so we a bit of copy-pasting from ox-html we can have both.

(defun ky-publish/export-denote-to-html (&optional async subtreep visible-only body-only ext-plist)
  (interactive)
  (let* ((extension (concat
                     (when (> (length org-html-extension) 0) ".")
                     (or (plist-get ext-plist :html-extension)
                         org-html-extension
                         "html")))
         (file (ky--publish/setup-publish-filename #'org-export-output-file-name extension subtreep))
         (org-export-coding-system org-html-coding-system))
    (org-export-to-file 'ky-html file
      async subtreep visible-only body-only ext-plist)))

(defun ky-publish/publish-denote-to-html (plist filename pub-dir)
  (advice-add 'org-export-output-file-name :around #'ky--publish/setup-publish-filename)
  (org-publish-org-to ky-publish/backend filename
                      (concat (when (> (length org-html-extension) 0) ".")
                              (or (plist-get plist :html-extension)
                                  org-html-extension
                                  "html"))
                      plist pub-dir)
  (advice-remove  'org-export-output-file-name #'ky--publish/setup-publish-filename))

We can also try running ky-publish/export-denote-to-html on any Org file, preferably with denote: links and marvel at how everything is broken! Every single link points to a non-existing location. This is why we need an Org Publish project(s) to put files in the proper locations that we assume in ky--publish/html-link.

Before we go to projects setup though, we still need to fix output structure for generated pages. Default org-export-output-file-name function is not going to create <title>/index.html structure for us. There are multiple ways to address the problem. For example output filename can be altered with EXPORT_FILE_NAME property set in Org file. There are problems with this though. The property doesn't support macros. Perhaps we could include an elisp source block that would set a value for the property automatically¹⁰ when it detects our backend, but even then the function doesn't support creating new directories. To my knowledge there is no way to influence how Org Publish project writes output files. We can't rename them and apart from setting :publishing-directory, there is no option for a more sophisticated output control.

It is not all lost though, to make this work we need just one small hack:

(defun ky--publish/setup-publish-filename (orig-fn &rest args)
  "An override for `org-export-output-file-name'.
Instead of using full Denote-style filenames extract `title` part
and create <title>/index.html entries. <title> directory has to
be created here too as a side-effect, as Org Publish expects it
to exist. Since the resulting directory structure is flat,
<title> part of the filename is assumed to be unique.
ORIG-FUN/ARGS are the original function/args passed by
`advice-add'."
  (let* ((orig-res (apply orig-fn args))
         (html-ext (string-remove-prefix "." (nth 0 args)))
         (pub-dir (nth 2 args))
         (denote-title (denote-retrieve-filename-title orig-res)))
    (if denote-title
        (let ((output-dir (file-name-concat pub-dir denote-title)))
          (make-directory output-dir t)
          (file-name-concat output-dir (concat "index." html-ext)))
      orig-res)))

Now we get the structure and naming we are after, things start to take shape! We are missing all the media resources though, and there is no root index.html page, so at best you can get a file listing if your HTTP server is setup to do that. I am testing locally with darkhttpd and file listing is exactly what I get.

These days I am busy learning Julia, even though what I should really be doing is looking for a job… 😅 Anyway, I use Jupyter to run Julia code blocks, it works pretty well and I can also pick different kernels and just as easily run SageMath¹¹. None of my mathematical/scientific endeavours are worth sharing, but I though it would be cool to make sure I am prepared! 😆

Luckily it is pretty easy to publish generated files. The function below alters the destination directory if it detects that it runs under Org Export. Generated files are put under their parent page directory. There is a condition in ky--publish/html-link to handle them and rewrite links accordingly. If there is no export running this function is mostly transparent.

I also tried this approach with Gnuplot and it works just as well, but overall I have no idea if it supports every single source block out there.

(defun ky-publish/dresource (file-name &optional fallback)
  "Can be used to generate :file param for Babel SRC blocks.

If export backend matches one we defined and the current buffer is
a denote page, generate filename as `<page_name>/FILE-NAME' in
the `:publishing-directory', `ky--publish/html-link' will set
links accordingly. In all other cases fallback to default SRC
block behaviour with :file set to FALLBACK."
  (let ((backend (when (boundp 'org-export-current-backend) org-export-current-backend)))
    (if (eq ky-publish/backend backend)
        (when-let* ((buffer-name (buffer-file-name))
                    (parent-dir (denote-retrieve-filename-title buffer-name))
                    (out-dir (file-name-concat (ky-publish/destination-directory) parent-dir))
                    (file (file-name-concat out-dir file-name)))
          (make-directory (file-name-parent-directory file) t)
          file)
      fallback)))

If you are wondering why can't I generate files before publishing and then publish as any other resource - I guess, it is my personal preference to not litter in my notes with generated files. For the same reason e.g. Jupyter puts files under .ob-jupyter by default.

What's left is a sitemap or simply an index page. This part is even more shabby than what you have seen so far, but nevertheless with this last task out of our way it is going to be a functional site.

According to the documentation all we need to do is set a couple of properties on denote-pages project to get an auto-generated index page. For anything but testing purposes you will need more control over the process. Luckily Org provides hooks for that, namely :sitemap-function and :sitemap-format-entry. Good news are - we are not going to use them! We would need to generate custom HTML inside these functions but our project is not about assuming particular design choices.

With that in mind, let's generate the simplest sitemap we can have.

(defun ky-publish/make-denote-pages-plist ()
  `(
    :base-directory ,(denote-directory)
    :base-extension ,(or denote-file-type "org")
    :publishing-directory ,(ky-publish/destination-directory)
    :publishing-function ky-publish/publish-denote-to-html
    :exclude ".*"
    :include ,(ky-publish/filter-denote-files (ky-publish/denote-page-regexp))
    :sitemap-filename "index.org"
    :sitemap-style list
    :auto-sitemap t))

Now another run of (ky-publish/publish (get-site-projects) t) completes the assignment. 🤪

Results are - it does what I want it to do. At the moment of writing it is a very fresh code. I am sure it will not take long for things to start breaking. Also, as stated in the goals, this is a minimal setup to make it possible to publish Denote files with denote: links. The ox-html generated result is still far from anything pretty (at least for me).

On my site I use Matcha CSS for styling. Matcha is a semantic CSS library, I like the simplicity of the approach both in terms of CSS and the clarity of HTML structure. It does take some extra work with ox-html output though. My site project has more custom transcoders to rewrite some parts into semantic equivalents and it is an ongoing process.