Building a development notes blog with denote and org-publish Notes
Building a blog with Emacs, customizing it, etc, can be quite confusing for the first timer.
It can also be little bit confusing for me as well, after I come back to it after a year or so.
That is why I have built this step by step documentation of how this blog was built. If others find benefit in this - even better.
Why You Should Always Teach What You Learn - https://medium.com/accelerated-intelligence/explanation-effect-why-you-should-always-teach-what-you-learn-9800983a0ea1
1 Why
1.1 Why build a website for development notes anyway
I have a lot of notes. Over a few systems(at home, at work, under descriptions of various projects).
So I wanted to gather everything in one place and somehow link them together. It is also nice if I can share that with others. Also, as of
, having everything about you/your way of working, etc written in one place, is a perfect for AI systems to grab. All in one place, no need to scrape the whole Internet. So the AI will for sure know how I like to work and my weak spots(what I usually write here is what I just learned or don't want to forget).Also, all of my notes are in .org files written in emacs, so of course my website will also be written in Emacs.
There are also many inspirational websites that look nice, are clean and simple, but not necessarily do the purpose of "development notes storage". Unless maybe this one - http://tips.defun.work/, which is also my initial inspiration.
Have your devnotes reachable anywhere + be able to share them with others(in a readable html format in a form of a blog). I am in.
1.2 Why use emacs for it
2 How
2.1 Choosing an Emacs tool to build such website
Emacs has both built in and external packages that help you to generate .html files from .org note files.
Some of them are:
- org-export-dispatcher(export one file at a time) Codeacademy example
- ox-publish(alist, export all pages of a project) (ox-publish, emacs blog, pkcwiki, pkcwiki2, my notes about ox-publish)
- create a blog with emacs-easy-hugo
- create a website with ox hugo
- org-static-blog, example
2.2 ox-publish VS org-static-blog
At first(org-static-blog
, almost had the blog setup and ready, converted around 50
pages to html already, displayed them under my domain. But then kept seeing
some features that ox-publish has and org-static-blog does not. Thought okay,
let's rethink that. Asked ChatGPT for help:
The choice between using the default ox-publish functionality in Org Mode or the org-static-blog package to create your Emacs website depends on your specific requirements and preferences. Let's explore both options:
ox-publish in Org Mode:
Pros
:Flexibility: ox-publish is part of Org Mode and provides considerable flexibility in customizing your website. You can define various publishing options, including different output formats and configurations.
Integration: It seamlessly integrates with Org Mode, making it easy to manage content directly within Emacs.
Customization: Allows extensive customization using Emacs Lisp, enabling you to tailor your website to your preferences.
Cons
:Steep Learning Curve: Setting up and configuring ox-publish might have a learning curve, especially if you're new to Emacs Lisp and detailed configuration.
org-static-blog Package:
Pros
:Simplicity: org-static-blog is designed specifically for creating simple blogs or websites using Org Mode. It's relatively straightforward to set up and use, making it ideal if you want a quick solution.
Minimal Configuration: It requires minimal configuration, making it accessible for beginners.
Focused Purpose: Primarily built for blogs, it might suit your needs if you're looking for a straightforward blogging platform within Emacs.
Cons
:Limited Features: It might lack certain advanced features or customization options compared to ox-publish, especially if you need more complex functionality.
Less Flexible: As it's tailored for static blogs, if your website needs evolve beyond blogging, you might face limitations.
Decision Factors:
Complexity vs. Simplicity: Consider how complex or simple you want your website setup to be.
Customization Needs: Evaluate how much flexibility and customization options you require for your website.
Future Growth: Consider whether your website's needs might expand beyond a blog and if you need features beyond what a blogging-centric package provides.
Familiarity with Emacs Lisp: Assess your comfort level with Emacs Lisp, as ox-publish might require more familiarity.
Recommendation:
If you prioritize simplicity, quick setup, and your website's focus is primarily a blog, org-static-blog could be a good starting point.
If you foresee needing extensive customization, flexibility, or plan for features beyond a blog, investing time in learning and using ox-publish might be more beneficial in the long run.
Experiment with both to understand their functionalities and how they align with your specific website goals. You can start with org-static-blog for a quick setup and later explore ox-publish for more intricate customization and flexibility if needed.
These were my thoughts AFTER I have read the ChatGPT response:
Of course after reading this I leaned toward
ox-publish
instead of org-static-blog. True, I have noticed that it lacks configuration options and is kind of restraining. I read other people's tutorials on how they use ox-publish and all the settings they have and I can not try to reuse them with org-static-blog. Not nice..Also there is this issue that I am facing today(re renders each post). - https://github.com/bastibe/org-static-blog/issues/126
I just hope that tag feature works in ox-publish(it should) - then I am all good. Let's dive in.
Unfortunately, the tags don't work in ox-publish way of making a website, so I will have to stick with org-static-blog. Not complaining. It will force me to read thought the source code more often and and see how I can use pieces of ox-publish in org-static-blog configuration.
Build a website with emacs and ox-publish. I will have ALL the features of org mode and standard publishing tools, exports will be clean, I will also be able to have multiple configs for each blog/website and. I will simply not have the tags.. which I will build on top by myself with the inspiration of =org-static-blog=(someday in the future). The notes will not be rendered into html that often, they won't be categorized by tags, but that's not a big deal. Primary audience for these notes is still - me. I need to be able to search through them and find the information in them. If it's not as easy to do that on the web - so be it. I will work on that later.
The issue I have mentioned above still bugs me. I can use tags in my website, BUT I can not compile it from the comfortable place of a custom script(instead of having the config in my main config), it is not native Emacs, stuff is built on top of it.. I once again decided to switch my mind and build my website with built in tools -2.3 What is "ox-publish"
Best description in a video format is here.
First things first of course:
- Publishing Org-mode files to HTML - https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html
- HTML specific export settings - https://orgmode.org/manual/HTML-Export.html#HTML-export
- My personal notes about building websites with
ox-publish
- Build a website with emacs and ox-publish I recommend to get to know to the source code if you like:
/org/ox-html.el.gz /org/ox-publish.el.gz
C-c C-e #
(org-export-dispatch) plus "template" - ALL possible in-file optionsorg-html-preamble-format
some useful %a %c %T functions- defaults here - https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html
IF you want to add html into an org file (not code block), can do it like such:
#+HTML: <div">Updated: #+HTML: hello #+HTML: </div>
3 Build
3.1 Installation of "ox-publish"
This package is already installed with Emacs by default.
3.1.1 Decide where to place the configuration
Ox-publish configuration might get large if you start modifying the head, preamble, postamble.. Also if you have multiple blogs/websites that you want to publish this way, you need to have two separate configurations.
Eventually you realize that you can't store multiple configurations in your emacs config file.
So the solution to this is to create a "build" folder and in it have such files:
devnotes/build/ ├── build.sh └── build-site.el
build.sh should be made to be executable chmo o+x build.sh
and it's contents
are:
#!/bin/sh emacs -Q --script build-site.el
and build-site.el
content would contain your ox-publish configuration(we will
remove the configuration from our emacs config and place it in this file with a
couple of additions).
That's it. Now when we will run the bash script (./build.sh
), it will fetch
the dependencies into the /build directory. It will then do the html conversion
from org files as you have specified and at the end it will print out "Build
complete!"
3.2 Creating .org files to be rendered
3.2.1 Separate repositories
We have one dir/repo for source files, another for exported html files. Having
them in separate repos allows us to use ripgrep search easier, allows us to use
project.el
simpler, not seeing duplicates of files(one .org, one .html). We
can version control the changes of the .org files as well as html files
separately.
/home/nixos/GIT: drwxr-xr-x 6 nixos users 4096 May 27 15:42 . drwx------ 6 nixos users 4096 May 27 15:44 .. drwxr-xr-x 5 nixos users 20480 May 28 15:28 arvydasg.github.io # html files lie here drwxr-xr-x 6 nixos users 20480 May 28 15:30 devnotes # source .org (could be other type) files lie here
3.2.2 Folder structure of source dir
When we have at least one .org file in the source directory, we can think about exporting it to .html.
Structure of devnotes folder, an example:
/home/nixos/GIT/devnotes: drwxr-xr-x 6 nixos users 20480 May 28 15:35 . drwxr-xr-x 6 nixos users 4096 May 27 15:42 .. drwxr-xr-x 2 nixos users 4096 May 28 13:59 build # folder in which the build script lives drwxr-xr-x 8 nixos users 4096 May 28 15:36 .git # git folder drwxr-xr-x 3 nixos users 4096 May 27 15:00 media # images folder drwxr-xr-x 4 nixos users 4096 May 27 15:00 static # css/js/ico folder -rw-r--r-- 1 nixos users 1404 May 27 16:30 20231125T235251--api-tools__api.org # denote file -rw-r--r-- 1 nixos users 646 May 27 15:00 20231126T001334--raspberrypi-first-setup__raspberrypi.org # denote file -rw-r--r-- 1 nixos users 790 May 27 15:00 20231126T001417--magic-mirror-setup__raspberrypi.org # denote file -rw-r--r-- 1 nixos users 324 May 27 15:00 20231126T001642--raspberrypi-pins__raspberrypi.org # denote file -rw-r--r-- 1 nixos users 225 May 27 15:00 contact.org # simple page that -rw-r--r-- 1 nixos users 164 May 27 15:00 .dir-locals.el # read "denote features" page for explanation to this, has no influence to blog -rw-r--r-- 1 nixos users 800 May 27 15:00 freelancing.org # random page -rw-r--r-- 1 nixos users 814 May 27 15:00 index.org # index(front) page of the website -rw-r--r-- 1 nixos users 1505 May 27 15:00 projects.org # custom list of projects, created the links myself -rw-r--r-- 1 nixos users 36 May 27 15:00 readme.md -rw-r--r-- 1 nixos users 23584 May 28 15:28 sitemap.org # automatically generated sitemap -rw-r--r-- 1 nixos users 817 May 28 15:06 statusupdates.org # random page -rw-r--r-- 1 nixos users 2728 May 27 15:00 uses.org # random page
3.2.3 Creating new notes with Denote
I am using Denote package, so I can use it's functionality to create a new entry. Here are some denote features.
I use Denote for my personal note taking, now I will also use it for development note taking. Win win.
3.3 The build process
3.3.1 "ox-publish" config file itself
Now since we have this list of files and folders, we are ready to export them to .html.
For that we will need the build script
. It's is a single file in which we
describe HOW we want our build configuration to look like.
There are many options available. I have written ALL of them (don't get intimidated by that) so I know what is available. It's better to know what is available than not to know :)
So here is the build script (latest version of it):
(require 'ox-publish) ;; run this with 'emacs -Q --script build-site.el' command (setq org-publish-project-alist '( ("notes" :base-directory "../" :base-extension "org" :publishing-directory "../../arvydasg.github.io" :recursive t ;; use :noexport: tag if you want headings that have it not to be exported ;; :exclude ;; :include :publishing-function org-html-publish-to-html ;converts org files to HTML ;; :preparation-function ;; :completion-function ;; ----------------------------------------------------------------------------------- ;; the DETAILS: options set within the individual .org file can ;; override all of the settings below: ;; you can specify majority the options that you see below in each ;; individual file. Find out syntax for it in =C-h v ;; org-publish-project-alist= and choose the configuration you want to ;; see the syntax for. :author "Arvydas Gasparavicius" ;; :creator :email "arvydas.gaspa@gmail.com" ;; :exclude-tags "noexport" :headline-levels 3 :language "en" :preserve-breaks nil :section-numbers t ;; :select-tags :time-stamp-file t ;; :with-archived-trees :with-author t :with-creator t :with-date t ;date from the filename? ;; :with-drawers :with-email t :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-inlinetasks t :with-latex t :with-planning t :with-priority t :with-properties t ;; :with-smart-quotes :with-special-strings t :with-statistics-cookies' t :with-sub-superscript nil :with-toc t :with-tables t :with-tags t :with-tasks t :with-timestamps t :with-title t :with-todo-keywords t ;; kita dalis :auto-sitemap t :sitemap-filename "sitemap.org" :sitemap-title "Notes" :sitemap-style tree :sitemap-format-entry ag/org-sitemap-date-entry-format ;; :sitemap-function ;; :sitemap-sort-folders :sitemap-sort-files anti-chronologically ;; :sitemap-ignore-case ;; :makeindex ;; :body-only t ;; ----------------------------------------------------------------------------------- ;; ox-html.el.gz :html-doctype "html5" :html-container "div" :html-content-class "content" :description "welcome to my site" :keywords "emacs antanas geles" :html-html5-fancy t ;; :html-link-use-abs-url ;; :html-link-home "sitemap.html" :html-link-home "index.html" :html-link-up "sitemap.html" ;; :html-mathjax ;; :html-equation-reference-format ;; :html-postamble nil ;; :html-postamble auto ;with-author/email/creator/date ;; :html-postamble t ;from :html-postamble-format :html-postamble "<hr/> <footer> <div class=\"copyright-container generated\"> <div class=\"copyright\"> Copyright © 2023-2024 Arvydas Gasparavicius </div> </div> <p class=\"date\">This org file is <u>exported</u> to HTML: %T</p> <p class=\"date\">This org file is last <u>modified</u>: %C</p> <p class=\"date\">This org file is <u>created</u>: %d</p> <div class=\"generated\"> Created with %c on <a href=\"https://www.gnu.org\">GNU</a>/<a href=\"https://www.kernel.org/\">Linux</a> </div> </footer> <button onclick=\"topFunction()\" id=\"myBtn\" title=\"Go to top\">Top</button> <script src=\"./static/js/generic.js\"></script> <script src=\"./static/js/scroll-to-top.js\"></script> <script src=\"./static/js/lightbox.js\"></script> <script src=\"./static/js/test.js\"></script>" ;; :html-preamble t ;insert a default one, which in none :html-preamble "<div id=\"updated\">Updated: %C</div>" ;; :html-head :html-head " <link rel=\"stylesheet\" href=\"./static/css/org-html-style-default.css\" type=\"text/css\"/> <link rel=\"stylesheet\" href=\"./static/css/generic.css\" type=\"text/css\"/> <link rel=\"stylesheet\" href=\"./static/css/taingram.css\" type=\"text/css\"/> <link rel=\"stylesheet\" href=\"./static/css/lightbox.css\" type=\"text/css\"/> <link rel=\"stylesheet\" href=\"./static/css/scroll-to-top.css\" type=\"text/css\"/> <link rel=\"icon\" href=\"./static/ag.ico\"> " ;; :html-head-extra "<link rel=\"stylesheet\" href=\"https://arvydas.dev/static/style.css\" type=\"text/css\"/>" ;; :html-head-extra "<link rel=\"stylesheet\" href=\"./static/style.css\" type=\"text/css\"/>" ;; :subtitle "this is a subtitle" :html-head-include-default-style nil ;turning them off, but adding in a separate file called org-html-style-default.css :html-head-include-scripts nil :html-allow-name-attribute-in-anchors nil :html-divs ((preamble "div" "preamble") (content "div" "content") (postamble "div" "postamble")) ;; :html-checkbox-type :html-extension "html" :html-footnote-format "<sup>%s</sup>" :html-footnote-separator "<sup>, </sup>" :html-footnotes-section "<div id=\"footnotes\">\n<h2 class=\"footnotes\">%s: </h2>\n<div id=\"text-footnotes\">\n%s\n</div>\n</div>" ;; :html-format-drawer-function ;; :html-format-headline-function ;; :html-format-inlinetask-function :html-home/up-format "<div id=\"org-div-home-and-up\">\n <a accesskey=\"h\" href=\"%s\">NOTES </a>\n \n <a accesskey=\"H\" href=\"%s\"> HOME </a>\n</div>" :html-indent nil ;; :html-infojs-options ;; :html-infojs-template ;; :html-inline-image-rules :html-link-org-files-as-html t ;; :html-mathjax-options ;; :html-mathjax-template :html-metadata-timestamp-format "%Y-%m-%d %a %H:%M" ;; :html-postamble-format ;; :html-preamble-format ;; :html-prefer-user-labels nil :html-self-link-headlines nil :html-table-align-individual-fields t ;; :html-table-caption-above t :html-table-data-tags ("<td%s>" . "</td>") :html-table-header-tags ("<th scope=\"%s\"%s>" . "</th>") :html-table-use-header-tags-for-first-column nil :html-tag-class-prefix "gaid" :html-text-markup-alist ((bold . "<b>%s</b>") (code . "<code>%s</code>") (italic . "<i>%s</i>") (strike-through . "<del>%s</del>") (underline . "<span class=\"underline\">%s</span>") (verbatim . "<code>%s</code>")) :html-todo-kwd-class-prefix "" :html-toplevel-hlevel 2 ;; :html-use-infojs :html-validation-link nil ;; :html-validation-link "<a href=\"https://validator.w3.org/check?uri=referer\">Validate</a>" :html-viewport ((width "device-width") (initial-scale "1") (minimum-scale "") (maximum-scale "") (user-scalable "")) :html-inline-images t :html-table-attributes (:border "2" :cellspacing "0" :cellpadding "6" :rules "groups" :frame "hsides") :html-table-row-open-tag "<tr>" :html-table-row-close-tag "</tr>" :html-xml-declaration (("html" . "<?xml version=\"1.0\" encoding=\"%s\"?>") ("php" . "<?php echo \"<?xml version=\\\"1.0\\\" encoding=\\\"%s\\\" ?>\"; ?>")) :html-wrap-src-lines nil :html-klipsify-src nil ;making it to t - jupyterhub like code blocks. Insane! ;; :html-klipse-css ;; :html-klipse-js ;; :html-klipse-selection-script ;; :infojs-opt ;; :creator :with-latex t ;; :latex-header ) ("static" :base-directory "../static" :publishing-directory "../../arvydasg.github.io/static" :base-extension "css\\|js\\|ico\\|svg" :recursive t :publishing-function org-publish-attachment ;copies the files verbatim ) ("media" :base-directory "../media" :publishing-directory "../../arvydasg.github.io/media" :base-extension "png\\|jpg\\|jpeg\\|pdf" :recursive t :publishing-function org-publish-attachment ;copies the files verbatim ) ("build-site.el" :components ("notes" "static" "media")))) (defun ag/org-sitemap-date-entry-format (entry style project) "Format ENTRY in org-publish PROJECT Sitemap format ENTRY ENTRY STYLE format that includes date." (let ((filename (org-publish-find-title entry project))) (if (= (length filename) 0) (format "*%s*" entry) (format "{{{timestamp(%s)}}} [[file:%s][%s]]" (format-time-string "%Y-%m-%d" (org-publish-find-date entry project)) entry filename)))) (setq org-export-global-macros '(("timestamp" . "@@html:<span class=\"timestamp\">[$1]</span>@@"))) ;; publish all the projects described in this file :components part (org-publish-all t) ;; publish only specific project ;; The t parameter tells it to regenerate all ;; files regardless of when they ;; were last generated)so no need to add the ;; force prefix with C-c C-e C-f (org-publish "my-org-site" t) (message "Build complete!")
You can see everything in it… How I specify directories, how I name the author, what is my nav(preamble), what is my footer (postamble), how I added css files, how I added js files, how I added a theme, how I created folders for images and css/js files, etc….
3.3.2 Running the build (generating the .html files from .org files)
After you have studied the config file and you understand what's happening, we
can now launch this script while in /home/nixos/GIT/devnotes/build
. Write
./build.sh
in shell and it will run the exportation.
./build.sh
is basically creates a "naked" emacs instance which runs our
configuration file:
#!/bin/sh emacs -Q --script build-site.el
After the export - check the html file folder, you should see the html files generated as you like. Open them in the browser to check. Commit if happy, modify the build script if not happy.
3.3.3 Previewing the generated site
Now that we have a few .org file notes in our directory and we have exported them into html, we can preview them. Open the .html file and it will open in the browser.
- Preview the html file in the browser
If the file on my wsl lies here:
file://wsl.localhost/NixOS/home/nixos/GIT/devnotes/html/index.html, I would open the whole html directory instead of a single file (file://wsl.localhost/NixOS/home/nixos/GIT/devnotes/html/) to preview the contents of it in the browser. This allows me to quickly open the files that I want and see what has changed.
how to implement - here https://systemcrafters.net/publishing-websites-with-org-mode/building-the-site/
- Preview with simple-httpd
Or you can use
simple-httpd
. It's a package that can host your files as a website on your local machine so that you can pull it up in your browser.You can install simple-httpd from MELPA using M-x package-install or by putting the following snippet in your Emacs configuration if you have use-package installed and MELPA configured as a package source(install it globally in your emacs config, not in build-site.el file):
(use-package simple-httpd :ensure t)
Now you can run M-x httpd-serve-directory. It will prompt you for a directory to serve from within Emacs.
Select the path of your public/ directory and then open your browser to http://localhost:8080 to see the preview of your site! Set httpd-port to change the default port if necessary.
Any time you regenerate the site files, you can just reload the page to see the result of changes you made! (if you need to do a refresh, then it's the same as opening in the browser…). That's why I don't bother with simple-httpd.
4 Publish
I am using github pages. It is possible to host static html pages from it for free. Here is a tutorial how it is done - deploy a static website with github pages.
To later add a domain name to your site, follow this tut - setup custom domain for github pages
4.1 Manual workflow
Good if you want to preview the changes locally first, only then commit.
- I first make changes in my devnote directory
- Run ./build.sh locally
- Check the output(html files) in arvydasg.github.io local repository
- If all is good, commit the .org files to devnote directory
- Commit the .html files to arvydasg.github.io directory
This was the primary way of working before setting up github actions before
.At first I was trying to create this workflow myself, manually, dealing with two repos, figuring out the paths, realizing that I need to use some secret keys, etc… but it was a struggle and then I found this - https://cpina.github.io/push-to-another-repository-docs/, but it did not work for me as expected, so I went back to manual way of creating a github action that I want. Each mistake led me to the direction I needed to go. Did not give up and made it :)
Here is how I made it:
4.2 Automated workflow with github actions
Took me the whole day today (
) to figure this one out…I learned about stepping into the github action containe midrun, a very cool way to debug your github action.
I learned that I must create a personal access token here -
https://github.com/settings/tokens (mark repository tick) NOTE:
regenerated this token for 60 more days. and then add the
token to the devnotes repo(from which you are running the action) here -
https://github.com/azegas/test/settings/secrets/actions. Name the secret
variable GH_PAT
.
And here is the first working config (current version):
name: Publish to arvydasg.github.io on: push: branches: - master jobs: publish: runs-on: ubuntu-latest steps: - name: Checkout devnotes repository uses: actions/checkout@v3 - name: Install Emacs run: | # install terminal version of emacs sudo apt install emacs-nox --yes - name: Run build.sh Script run: | cd build/ ./build.sh # directory after build.sh run script - /home/runner/work/devnotes/devnotes # source directory - /home/runner/work/devnotes/devnotes # destination directory - /home/runner/work/devnotes/arvydasg.github.io # tmp directory - /tmp # test repo to which to push stuff - https://github.com/azegas/test.git - name: Clone arvydasg.github.io repo run: | cd /tmp git clone https://github.com/azegas/arvydasg.github.io.git ls -la - name: Copy content from source to arvydasg.github.io repo run: | cd /tmp/arvydasg.github.io cp -r /home/runner/work/devnotes/arvydasg.github.io/* . ls -la - name: Authenticate myself to git run: | cd /tmp/arvydasg.github.io pwd git ls-remote --get-url origin git config --global user.email "azegaspa@gmail.com" git config --global user.name "BOT" git config --global -l - name: Commit the changes run: | cd /tmp/arvydasg.github.io git add . git commit -m "yay" git status - name: Push changes to arvydasg.github.io repo (using GitHub personal access token) env: GH_PAT: ${{ secrets.GH_PAT }} run: | # so it's only necessary to set up the github secret for devnotes repo, not for arvydasg.github.io repo cd /tmp/arvydasg.github.io git remote set-url origin https://$GH_PAT@github.com/azegas/arvydasg.github.io.git git push origin master # - name: Setup tmate session # uses: mxschmitt/action-tmate@v3
Now by having this in .github/workflows/deploy.yml
file, whenever you make a
commit to devnotes
repo, arvydasg.github.io repo will get updated with the
html content, effectively updating the arvydas.dev site.
So now I will never have to manually do a push again. Its good that I dont have to push to see the site, can develop locally, test locally, push only when the changes ready.
4.3 Fix commit message
I want to make the devnotes
repo commit link to show in arvydasg.github.io
repo commit that was automatically made with github action.
Tried checking out this documentation - https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
And then from it, tried using some of the "context" variables like so:
# env: # ORIGINAL_COMMIT_MESSAGE: ${{ github.event.head_commit.message }} # GITHUB_RUN_ID: ${{ github.run_id }} # git commit -m "Automated commit. Original commit: $ORIGINAL_COMMIT_MESSAGE - GitHub Actions Run ID: $GITHUB_RUN_ID" # Automated commit. Original commit: mod - GitHub Actions Run ID: 93562……34702 # git commit -m "$GITHUB_SHA, $GITHUB_WORKFLOW_SHA, $GITHUB_TRIGGERING_ACTOR" # bed1324733be364f7e7f61417c77f57e3873da08, bed1324733be364f7e7f61417c7……7f57e3873da08, azegas
But I did not like that there were no hyperlinks, only some jibberish. But from the example above, you can find t he commit with it - bed1324733be364f7e7f61417c77f57e3873da08. But no hyperlink…
I finally found a way to make a commit message with a hyperlink:
- name: Commit the changes with dynamic message env: GITHUB_SHA: ${{ github.sha }} REPO: ${{ github.repository }} run: | cd /tmp/arvydasg.github.io COMMIT_URL="https://github.com/${REPO}/commit/${GITHUB_SHA}" COMMIT_MESSAGE="Automated commit for - ${COMMIT_URL}" git add . git commit -m "${COMMIT_MESSAGE}" git status
Added this to my main config.
4.4 Problem with "date modified"
Commit that did this change - https://github.com/azegas/devnotes/commit/f5471491056b4e055fbc8a9556d18fc141ffd2f5
At the top of each page, I would have this rendered for example in the preamble - "Updated: 2024-06-04 Tue 04:46". Basically an identification when the particular post was last updated.
Now this will be removed, and I will explain why below.
But first, a few issues with this "updated date" box in the first place:
- It shows wrong time, not the current Lithuanian time
- IF a small change is made in the pramble or postamble of the whole site, the pages get REGENERATED and therefore, their date when they are updated get changed. Even though the content of the page did not change. I don't like that. The idea was to track only the date modified of the post, not of the html that surrounds it.
- IF you delete some files in the arvydasg.github.io repo and then regenerate them again - "updated" date will be inaccurate again . Not good.
Basically the last point is the main reason why I have to remove this feature.
4.4.1 Workflow before github actions
Before github actions, only the pages that have changed would get regenerated.
Now - since it's happening in the pre-commit and is temporary, all the files are being freshly generated by each github action run. Therefore rendering this "updated" field useless.
So I will simply leave the "created" field at the bottom of the page for those who are interested and remove everything else.
5 Bits and pieces
5.1 Add a "google translate" button to the site
5.2 Individual files customization options
We are specifying all the configuration in the build-site.el
file, however,
you can still make some changes to the individual files.
Check like it's done in index.html
Here are the things you can modify:
#+title: Freelancing #+date: <2020-01-01 Sun 01:01> #+description: #+filetags: #+OPTIONS: toc:nil #+OPTIONS: num:nil
5.3 Generating pages with code blocks
So far we've been looking at a very simple example page that doesn't really have much on it. What happens when we try to generate a more elaborate Org file containing code blocks?
Let's try it out with another file, a version of my literate Emacs configuration called Emacs.org!
If you're generating a site that features code blocks like a coding blog or a literate Emacs configuration, you'll probably see an error like this when you generate the site:
Cannot fontify source block (htmlize.el >= 1.34 required)
To resolve this issue, you will need to install the htmlize package from MELPA. We can automate the process of installing this package by adding the following snippet to our build-site.el file:
;; Set the package installation directory so that packages aren't stored in the ;; ~/.emacs.d/elpa path. (require 'package) (setq package-user-dir (expand-file-name "./.packages")) (setq package-archives '(("melpa" . "https://melpa.org/packages/") ("elpa" . "https://elpa.gnu.org/packages/"))) ;; Initialize the package system (package-initialize) (unless package-archive-contents (package-refresh-contents)) ;; Install dependencies (package-install 'htmlize)
In this snippet, we load Emacs' package manager and configure the package-user-dir to be a sub-directory of our project folder. This allows you to install packages for the script without mixing them up with the packages of your personal Emacs configuration!
The next thing we do is add MELPA to the package archive list and refresh the package archive so that htmlize can be found. Finally, we call package-install to install it!
In a future video I'll show you how to convert your Emacs color theme to a CSS file that you can use to colorize source blocks with exactly the same colors used in the theme!
5.4 Important information about linking between .org files
Github link to issue - https://github.com/protesilaos/denote/issues/369
As of
denote links do not work with exporting to html. Talked with Prot about it, there is no fix for it now. So what I do now is I create file links(C-c C-l file: <and your filename>) instead of denote links, these do work. Also, if you get a link wrong and it is not possible to render it, the compilation of html pages will break. That's good, because by checking the logs you will see the broken link and can fix it. If you don't want the compilation to stop, you can add this in your config:(setq org-export-with-broken-links 'mark)
You will simply see this in your html page(I prefer to know that there are broken links during the html compilation, so I don't put the code above in my config).
[BROKEN LINK: denote:20231125T212326]
Naming the links:
Example taken from here - https://orgmode.org/manual/Links-in-HTML-export.html
#+ATTR_HTML: :title The Org mode website :style color:red; https://orgmode.org
5.5 CSS support in HTML export
Official docs for css support in HTML export - https://orgmode.org/manual/CSS-support.html
There are a few options when it comes to custom CSS on your website generated with Emacs.
5.5.1 Use the default styles from ox-html
You have to do nothing, ox-html
- org-html-style-default
got you covered.
And it looks pretty good already by default.
But you can do like I did, went to org-html-style-default
and copied the CSS
from there, placed in my own stylesheet. I can customize the look myself now a
little. Don't have to rewrite everything - just the parts that I want.
Also it's good to have the default styles as a reference close to me rather than somewhere deep in the emacs docs.
5.5.2 Add custom classes to blocks of text/code
5.5.3 import custom css
Default html export engine already has us covered with basic css, you can enable/disable it like so:
:html-head-include-default-style t/nil
But you might want to change the default css. Best bet is to do something like this:
:html-head "<link rel=\"stylesheet\" href=\"../static/style.css\" type=\"text/css\"/>"
Basically whatever you put in style.css
now will end up in every html page of
yours. We can add any information you like there. But for now I am concerned
about css and favicon. This will link to my own style.css file, but you can
also use CDN's or bootstrap or whatever else.
Use your own config from another website/blog for example:
:html-head-extra "<link rel=\"stylesheet\" href=\"https://arvydas.dev/static/style.css\" type=\"text/css\"/>"
5.5.4 org-html-themes
ORRR last thing that you can try when it comes to styling your website, try using one of these, org-html-themes.
You can point to one of these themes like such, by placing this code at the top of your.org file:
Or if you have downloaded the code(more future proof solution), add such line at the top of your .org file.
#+SETUPFILE: PATH/TO/GIT/REPO/org/theme-NAME-local.setup
Of course we can also modify our setq org-static-blog-page-header
to contain
the org-html-themes
, but I won't do that this time since I know I will be
using my own custom stylesheet.
5.5.5 Note to self
Some css examples here(built with ox-publish though) - https://github.com/arvydasg/emacs_blog/blob/master/css/style.css
5.6 Javascript support in HTML export
Official docs - https://orgmode.org/manual/JavaScript-support.html
Add the .js files the same way as you would add .css file.
5.7 Images in HTML export
5.7.1 Display images
NOTE: gif's are not rendered in github pages for some reason (julijaconsulting and facebook project gif's)
Official docs - https://orgmode.org/manual/Images-in-HTML-export.html My docs - images in org files
5.7.2 One option is to add images as plain html
But then you can not change the size of it in lightbox.
5.7.3 Another option is to include a link to a file(the usual way)
To toggle the images(show/hide), use org-toggle-inline-images
.
Okay, so no now we know that we can create a yasnippet for this(I have mine
bound to img
) and then I can drop images wherever.
5.7.4 Org download package for inserting images in org files
Optional, does not really work if in WSL - using org download package in WSL
5.7.5 Add a lightbox effect on the images
5.8 Preamble(nav) and postamble(footer)
Docs about them - https://orgmode.org/manual/HTML-preamble-and-postamble.html
Postamble - https://taingram.org/blog/org-mode-blog.html Postamble - nice footer and more? - https://ict4g.net/adolfo/notes/emacs/linking-my-html-pages-to-source-code.html
Check other inspirational websites that are build with Emacs for some inspiration.
Then simply modify - :html-preamble and :html-postamble in your config.
5.9 Add a search to the sitemap.org
So my thought was that… if I create lots of small files and all of those
small files are listed in one list in sitemap.org
, then it will be difficult
to locate the files.. The tags are currently not visible in this list, you only
have to search by keywords that you know. For example you search for Django,
but then you get a search bar with results of 20 entries. Have to go through
each one of them… Pain.
What if I had a simple search bar in sitemap.org
file which would allow me to
filter the list results by the words I write in the search bar?
5.9.1 Print "hello world" first in the sitemap.org
Sounds good, but the problem is that I do not control the behavior of the
sitemap.org
generation. At least not originally. But it's emacs, so I can.
I see that my build configuration has :sitemap-function
option, which is
currently commented out. Perhaps this is the place that is responsible for
creating the sitemap.org
file?
I go to
/nix/store/jm91p103vzqyvjcgk44ga6r5p5fhj45i-emacs-29.3/share/emacs/29.3/lisp/org
and find ox-publish.el.gz
package. In it I see org-publish-sitemap-default
function that looks like this:
(defun org-publish-sitemap-default (title list) "Default site map, as a string. TITLE is the title of the site map. LIST is an internal representation for the files to include, as returned by `org-list-to-lisp'. PROJECT is the current project." (concat "#+TITLE: " title "\n\n" (org-list-to-org list)))
First I want to simply print "hello world". Let's modify this function like so:
(defun ag-org-publish-sitemap-default (title list) "Default site map, as a string. TITLE is the title of the site map. LIST is an internal representation for the files to include, as returned by `org-list-to-lisp'." (concat "#+TITLE: " title "\n\n" "* Hello World\n\n" (org-list-to-org list)))
Then in the build config, let's use this function:
:sitemap-function ag-org-publish-sitemap-default
Build the site.
You should see the hello world in the sitemap.org
. Great, we have found a way
to control the sitemap!
ag-org-publish-sitemap-default
5.9.2 Add the search bar in plain html
To include an HTML search bar in the
sitemap.org
file generated by our custom function, we need to embed the HTML directly into the Org file. Org-mode supports including raw HTML, which will be preserved when the Org file is exported to HTML.
Let's modify our custom function to include the search bar:
(defun ag-org-publish-sitemap-default (title list) "Default site map, as a string. TITLE is the title of the site map. LIST is an internal representation for the files to include, as returned by `org-list-to-lisp'." (concat "#+TITLE: " title "\n\n" "#+OPTIONS: toc:nil\n\n" ;; Disable the table of contents for better formatting "#+BEGIN_EXPORT html\n" "<div class=\"search-wrapper\">\n" "<label for=\"search\">Search</label>\n" "<input type=\"search\" id=\"search\" data-search>\n" "</div>\n" "#+END_EXPORT\n\n" (org-list-to-org list)))
Now create a search.js
file in static/js
folder. Add it in the
html-postamble
.
Content of the search.js
:
// This code waits for the entire HTML document to load before running document.addEventListener("DOMContentLoaded", function() { // Select all list items (notes) within the unordered list with class "org-ul" const allNotes = document.querySelectorAll(".org-ul li"); // Select the search input field with the data attribute "data-search" const searchInput = document.querySelector("[data-search]"); // Add an event listener to the search input field that listens for "input" events searchInput.addEventListener("input", function(e) { // Get the current value of the search input field and convert it to lowercase const value = e.target.value.toLowerCase(); // Loop through each note in the list of notes allNotes.forEach(function(note) { // Get the text content of the note and convert it to lowercase const noteText = note.textContent.toLowerCase(); // Check if the note's text includes the search input value if (noteText.includes(value)) { // If it does, display the note note.style.display = ""; } else { // If it doesn't, hide the note note.style.display = "none"; } }); }); });
Rebuild the site. Search should work.
5.10 Extra's
Extra tips:
5.10.1 You can also include other content from other files!
Like such - https://orgmode.org/manual/Include-Files.html
I have done this to include recent posts into index.org page.
5.10.2 Creating a HTML page with my .emacs configuration
Simply made a symlink to my .org configuration file and it renders it together with the rest of the .org files.
[nixos@nixos:~/GIT/devnotes/build]$ ln -s ../../dotfiles/.emacs.d/init.org init.org
UNFORTUNATELY - if you will chose to deploy to github pages, it will not understand the symlink, the build will fail with such message:
Error: No such file or directory @ rb_check_realpath_internal - /github/workspace/org/init.org
So I ditched this idea of having a symlink to my init.org. IF I will ever host this site on another platform - I might try this workaround again.
5.10.3 Adding a date to post title in sitemap
Inspiration here - https://taingram.org/blog/org-mode-blog.html
We can customize the entry format with a sitemap-format-entry
function. In my
case I wanted to show the date inline with the blog post listings:
(defun ag/org-sitemap-date-entry-format (entry style project) "Format ENTRY in org-publish PROJECT Sitemap format ENTRY ENTRY STYLE format that includes date." (let ((filename (org-publish-find-title entry project))) (if (= (length filename) 0) (format "*%s*" entry) (format "{{{timestamp(%s)}}} [[file:%s][%s]]" (format-time-string "%Y-%m-%d" (org-publish-find-date entry project)) entry filename))))
Notice above we insert an Org macro called timestamp, it is defined as follows:
(setq org-export-global-macros '(("timestamp" . "@@html:<span class=\"timestamp\">[$1]</span>@@")))
We then also must inform org-static-project-alist
about this change by using
our custom function in:
:sitemap-format-entry ag/org-sitemap-date-entry-format
Re-render your .org files.
This macro adds some HTML around the timestamp for CSS styling, it has to be done as a macro as otherwise Org escapes the HTML tags. The results are:
#+BEGIN_EXPORT html
6 Customize the automated build
I want to remember the ways it's possible to modify the export process.
- Official docs - https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html
A single line does this -
#+INCLUDE: "sitemap.org" :lines "12-20"
This will include 12-20th lines of the
sitemap.org
page. Can simply open sitemap.org page to check what lines you want to include.Useful when you want to add a list of sitemap in index.html page for example.
#+INCLUDE: "sitemap.org" :lines "-6"
See a default suggested template
C-c C-e # (org-export-dispatch)
# html template: #+options: html-link-use-abs-url:nil html-postamble:auto html-preamble:t #+options: html-scripts:nil html-style:t html5-fancy:nil tex:t #+html_doctype: xhtml-strict #+html_container: div #+html_content_class: content #+description: #+keywords: #+html_link_home: #+html_link_up: #+html_mathjax: #+html_equation_reference_format: \eqref{%s} #+html_head: #+html_head_extra: #+subtitle: #+infojs_opt: #+creator: <a href="https://www.gnu.org/software/emacs/">Emacs</a> 29.3 (<a href="https://orgmode.org">Org</a> mode 9.6.15) #+latex_header: # Default template #+options: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline author:t #+options: broken-links:nil c:nil creator:nil d:(not "LOGBOOK") date:t e:t #+options: email:nil f:t inline:t num:t p:nil pri:nil prop:nil stat:t tags:t #+options: tasks:t tex:t timestamp:t title:t toc:t todo:t |:t #+title: *Org Src 20231215T182523--building-a-development-notes-blog-with-denote-and-org-publish__denote_emacs_websites.org[ org ]* #+date: <2024-06-06 Thu> #+author: nixos #+email: nixos@nixos #+language: en #+select_tags: export #+exclude_tags: noexport #+creator: Emacs 29.3 (Org mode 9.6.15) #+cite_export:
- Export settings - https://orgmode.org/manual/Export-Settings.html
- HTML specific export settings - https://orgmode.org/manual/HTML-specific-export-settings.html
Source code:
/org/ox-html.el.gz /org/ox-publish.el.gz
org-html-preamble-format
some useful %a %c %T functionsIF you want to add html into an org file (not code block), can do it like such:
#+HTML: <div">Updated: #+HTML: hello #+HTML: </div>
Create editable text areas Docs - https://orgmode.org/manual/Text-areas-in-HTML-export.html
If you ever want to try using this feature, here is an example: