One of my critical projects is making a new knowledge-centered website that not only makes my works-in-progress notes available to all, but also functions as a working lab notebook. I’ve tried document managers such as Quiver, Evernote, and Scrivener and am just starting to review digital research tools that are used in the Sciences. Nothing quite fits my peculiar workflow, and so I’ve decided to finally roll my own. I’ve wanted to make something for at least a decade, but it’s only now that I think I have all the skills to start building it.

This post is the running log of what I’m doing with this project. I’m going to try to keep the relevant decision-making in this post, and update it as I go. This will help me simulate the use case where I’m using my blogging system as my laboratory log.

Requirements

The basic requirement I have is that I can write as fast as I think and have the environment deconstruct and recompile my writing into a well-structured, easily-browsed website. This new website will replace my current public website.

Here are my current criteria as of July 8, 2018:

• Plain text files that use Markdown formatting for content, and a simple front-matter format for metadata.
• Meta-tagging extensions to mark areas of document as a certain kind of knowledge or idea
• Automatic generation of good-looking documents as web pages.
• Does not rely on a dynamic runtime environment to build pages (no PHP or databases). Uses Javascript and NodeJS as the underlying build engine.
• Easy branching, forking of information nodes.
• Support for a composite document model, similar in spirit to OpenDoc or OLE.
• Preprocessor that scans and extracts useful subnodes nodes from source material, replacing the source material with subnodes that can then be shared.
• Compiles useful navigation links, indices, and glossaries.
• Compiles to a static website on my laptop.
• Easily edited in a text editor like VS Code
• Media support using my ancient image plugin, Lazy Image Layout

I’m informally calling this project KM-SSG. SSG stands for static site generator, which is a type of software that creates a bunch of .html pages from source materials. This kind of software has become more popular in recent years as computers have gotten fast enough to make the “site generation” part of the software practically instantaneous. Static sites also have the advantage of being inherently more secure than a system that uses “dynamic pages” (e.g. WordPress) instead of “static pages”. Back in the days of WordPress 1.0, the other major competitor was a static site generator called Moveable Type which is still around. After 12 years of blogging, though, my needs have become more specialized and so I want to make my own system.

What is different NOW is the sheer ubiquity of open source software libraries that you can use to build software, along with the code collaboration tools that make the process of rolling-up something vastly simpler. What’s hard now is picking the right one, and that is what this post is about.

Reviewing the Candidates

There are four popular Static Site Generators that I picked from the list on staticgen.com. I’ve grouped them by underlying language:

• Jekyll – The most popular SSG with a lot of support. It however runs on Ruby, a language I’m not familiar with. Since it’s so popular, it’s likely that I can find the support I need to figure things out. On the other hand, it would mean learning a new language and development environment.
• Next and Gatsby – Both of these use Javascript/NodeJS as the underlying environment, which is my current platform. Interestingly, they specifically are designed to output React apps, which is another technology I’ve been getting my head around. Choosing either of these SSGs would serve to broaden my development horizons. However, it’s not clear to me that I want to actually use React as my front-end because React best practices change so often. I don’t want to create a maintenance headache for myself.
• Hexo – This also uses Javascript/NodeJS, but instead of React it uses the venerable standard template engines EJS and PUG to create final pages. I am leaning toward this, because I don’t think I necessarily want to make React apps or get stuck in the React world.

In the past I also tried Metalsmith, which is more like a SSG toolkit that works similar to once-popular Javascript build tool Gulp. However, it’s not a very popular tool these days and requires considerable digging around to figure out how to use it. It also has not seen any recent development, so I suspect it is fading away.

OTHER CANDIDATES

• JUL 09 2018 – Mirolabs via Discord mentions coda.io and notions.so tools, described as really promising but not SSGs. Since we are both interested in KM I will look at them.

DEVELOPMENT LOG

JULY 08

My strategy at the moment is to try to build a really basic content framework with Jekyll, importing all my old WordPress posts using the Jekyll WordPress Importer. If I get that working, I can try to implement another feature. I suspect at some point I’ll need to write my own custom scripts, and perhaps that will be the right time to change to Hexo.

I copied all the imported WordPress posts, already converted to Markdown, into the root level of my Jekyll install and just tried jekyll server --livereload --incremental. It took 9.5 minutes (556 seconds) to create the site…ooof. There were no indices or category pages. There were also issues with HTMLENTITIES (e.g. &ndash) in the titles. I think I probably need to reprocess these MD files with my script first, then regenerate again.

I also read about how to generate indices, remap pages using the permalink property, and something called “collections” to allow one to group pages and posts together.

That’s enough for today. For next steps, will look at fixing the metadata display (it shows up as a key-value array) and also make sure that the posts were completely processed through PanDoc before generating the site again. It’s a bit worrying how slow it is. I might want to find a WordPress-compatible theme and install that.