One of my critical projects is making a new knowledge-centered website that not only makes my works-in-progress notes available to all, but also functions as a working lab notebook. I’ve tried document managers such as Quiver, Evernote, and Scrivener and am just starting to review digital research tools that are used in the Sciences. Nothing quite fits my peculiar workflow, and so I’ve decided to finally roll my own. I’ve wanted to make something for at least a decade, but it’s only now that I think I have all the skills to start building it.
This post is the running log of what I’m doing with this project. I’m going to try to keep the relevant decision-making in this post, and update it as I go. This will help me simulate the use case where I’m using my blogging system as my laboratory log.
The basic requirement I have is that I can write as fast as I think and have the environment deconstruct and recompile my writing into a well-structured, easily-browsed website. This new website will replace my current public website.
Here are my current criteria as of July 8, 2018:
- Plain text files that use Markdown formatting for content, and a simple front-matter format for metadata.
- Meta-tagging extensions to mark areas of document as a certain kind of knowledge or idea
- Automatic generation of good-looking documents as web pages.
- Easy branching, forking of information nodes.
- Support for a composite document model, similar in spirit to OpenDoc or OLE.
- Preprocessor that scans and extracts useful subnodes nodes from source material, replacing the source material with subnodes that can then be shared.
- Compiles useful navigation links, indices, and glossaries.
- Compiles to a static website on my laptop.
- Easily edited in a text editor like VS Code
- Media support using my ancient image plugin, Lazy Image Layout
I’m informally calling this project KM-SSG. SSG stands for static site generator, which is a type of software that creates a bunch of
.html pages from source materials. This kind of software has become more popular in recent years as computers have gotten fast enough to make the “site generation” part of the software practically instantaneous. Static sites also have the advantage of being inherently more secure than a system that uses “dynamic pages” (e.g. WordPress) instead of “static pages”. Back in the days of WordPress 1.0, the other major competitor was a static site generator called Moveable Type which is still around. After 12 years of blogging, though, my needs have become more specialized and so I want to make my own system.
What is different NOW is the sheer ubiquity of open source software libraries that you can use to build software, along with the code collaboration tools that make the process of rolling-up something vastly simpler. What’s hard now is picking the right one, and that is what this post is about.
Reviewing the Candidates
There are four popular Static Site Generators that I picked from the list on staticgen.com. I’ve grouped them by underlying language:
- Jekyll – The most popular SSG with a lot of support. It however runs on Ruby, a language I’m not familiar with. Since it’s so popular, it’s likely that I can find the support I need to figure things out. On the other hand, it would mean learning a new language and development environment.
- JUL 09 2018 – Mirolabs via Discord mentions coda.io and notions.so tools, described as really promising but not SSGs. Since we are both interested in KM I will look at them.
My strategy at the moment is to try to build a really basic content framework with Jekyll, importing all my old WordPress posts using the Jekyll WordPress Importer. If I get that working, I can try to implement another feature. I suspect at some point I’ll need to write my own custom scripts, and perhaps that will be the right time to change to Hexo.
I copied all the imported WordPress posts, already converted to Markdown, into the root level of my Jekyll install and just tried
jekyll server --livereload --incremental. It took 9.5 minutes (556 seconds) to create the site…ooof. There were no indices or category pages. There were also issues with HTMLENTITIES (e.g.
&ndash) in the titles. I think I probably need to reprocess these MD files with my script first, then regenerate again.
I also read about how to generate indices, remap pages using the
permalink property, and something called “collections” to allow one to group pages and posts together.
That’s enough for today. For next steps, will look at fixing the metadata display (it shows up as a key-value array) and also make sure that the posts were completely processed through PanDoc before generating the site again. It’s a bit worrying how slow it is. I might want to find a WordPress-compatible theme and install that.