No More Website Generators

I never really liked dynamically generated websites. As a guy who lives on the command line, the slow, GUI-required, must-be-constantly-connected approach just doesn't work well for me. What's old is new again, and much of the Internet is going back to static websites. There are a few generators that are worth looking at, but I've had problems with them as well. I'm moving back towards how I used to do web development: no tools at all.

WordPress

When I first started this site, I used WordPress. I was never really happy with it. I couldn't edit my page offline and upload it later. I don't need multi-user support. The layout is a pain to edit. Attaching pictures is tedious.

The worst part was that in order to get any kind of good performance from the site, I needed to cache the pre-generated site from the WordPress database. Therefore, I was ultimately doing everything I could to work around WordPress, meaning that I was using the wrong tool for the job.

If I'm doing everything I can to avoid PHP, mandatory online editing, and dynamic loading, I'm taking WordPress's square peg and forcing it into a round hole. The "features" that define WordPress were getting in my way.

WordPress has its strengths. It allows someone who just wants a basic website to instantly create one with no knowledge of HTML. As someone whose primary HTML editor was Notepad before this (and enjoyed it!), the security vulnerabilities and editing difficulties just aren't worth the trade-off.

Jekyll

I've been using Jekyll to generate this site. It hasn't been too bad. I can easily write an article and upload it to my server. If I have 30 pictures to upload, I can simply drag-and-drop them all in a file browser. Rather than using a "sandbox" plug-in or something of that nature, I can generate the site locally on my hard drive to test how things look.

Then, it started adding features and plug-ins. This isn't terrible; I can choose not to use them. I admit, one or two are nice additions, though. I use postfile which allows me to semi-easily add pictures and other files to the same directory as the article itself rather than keeping thousands of pictures in some "asset" folder.

Then, when I posted two articles in one day, I found that there is no way to tell Jekyll their chronological order. It sorts by filename. Jekyll is open source, so I patched my copy to accept a time as well as a date.

While I may have added a few work-arounds over time to undo or ignore some of Jekyll's functionality, it hasn't been too bad. If this had been the end of the story, though, this article wouldn't exist.

The first time Jekyll broke on me was when I installed it on a new computer. I didn't know there was a new version, and its command line interface had changed. I now needed to run jekyll build rather than the old command. This is purportedly a one-time "future-proofing" change. This is completely understandable.

It also broke all of my plugins and changes. One was a nice break, though: post_url no longer needed to be patched in; it is now part of Jekyll's core. Nothing else worked, though.

I spent some time getting them to work again. While I can't hold this completely against Jekyll itself, I don't know the language Jekyll is written in: Ruby. Fixing these plug-ins required me to hack in a language I didn't know with no plug-in API documentation. It seems that to write a plug-in, you already need to know how Jekyll does things internally. Everything I found on the Web was for the old version.

It's fine. The hurdle's jumped. It comes with a software tool's rise in popularity and becoming bigger. Jekyll was trying to make itself more extensible and easier to use. Having every command begin with jekyll rather than using a loose collection of other keywords is a good idea. After all, someone may already have a non-Jekyll related serve command on their system.

Then, there is something that isn't Jekyll's fault, but it comes with its design decisions. As mentioned, Jekyll is written in Ruby. It requires many packages and libraries to run, including the Ruby interpreter itself.

Ruby updated.

This caused Jekyll to completely break. While not being Jekyll's fault, it did add another layer of unreliability to using Jekyll. I couldn't add to or modify my site because of this until Jekyll was fixed. Eventually, everything was fine again, but it took a while. I debated on leaving Jekyll.

When I switched from WordPress, I had to almost rewrite every article I had. All of the text formatting had to be redone from WordPress's format to either Markdown or Textile. Naturally, once I completed this, a new feature was added the Jekyll to import a site from WordPress.

After comparing my two options, I chose Textile. I liked the syntax, and it gave me the versatility and power over my formatting that comes with using raw HTML. As a basic example, there is no way to center anything in Markdown.

Now is the future, and naturally I chose incorrectly. While I believe Textile to be superior from a formatting standpoint, there is now a huge amount of support for Markdown. Textile is practically unheard of at this point. If I am going to switch from Jekyll, I have to reformat everything again.

I looked at Discount, a C implementation of Markdown. It's extremely fast and includes one or two additions to Markdown that I believe make the language much more practical, including the ability to center. I started to write a Textile parser in C, but I don't think I'm going to bother. It's a dying language. I feel like I'd be the only person on the Web still trying to use it.

Jekyll has one other problem: it's slow. Each time Jekyll is run, it regenerates the entire site, and I'm pretty sure it only handles one article at a time. I, like most people, have a multi-core system. Jekyll should be capable of multi-threading if it runs this slowly.

After all this, I realized that I'm going through the same motions as with WordPress, albeit to a lesser extent. I finally admitted this to myself when I hit my latest Jekyll/Ruby problem:

/usr/local/lib64/ruby/gems/2.0.0/gems/execjs-2.0.2/lib/execjs/runtimes.rb:51:in `autodetect': Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes. (ExecJS::RuntimeUnavailable)
        from /usr/local/lib64/ruby/gems/2.0.0/gems/execjs-2.0.2/lib/execjs.rb:5:in `<module:ExecJS>'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/execjs-2.0.2/lib/execjs.rb:4:in `<top (required)>'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/coffee-script-2.2.0/lib/coffee_script.rb:1:in `<top (required)>'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/coffee-script-2.2.0/lib/coffee-script.rb:1:in `<top (required)>'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/jekyll-coffeescript-1.0.0/lib/jekyll-coffeescript.rb:2:in `<top (required)>'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/jekyll-2.0.3/lib/jekyll.rb:73:in `<top (required)>'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/lib64/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
        from /usr/local/lib64/ruby/gems/2.0.0/gems/jekyll-2.0.3/bin/jekyll:6:in `<top (required)>'
        from /usr/local/bin/jekyll:23:in `load'
        from /usr/local/bin/jekyll:23:in `<main>'

It's time for a change.

The New Tool: A Shell Script

Common to Jekyll: Using Markdown

I recently read an article, It's OK Not to Use Tools by Jonas Downey.

Today, a basic HTML/CSS site seems almost passé. But why? Is it because our new tools are so significantly better, or because we’ve gone overboard complicating simple things?

While there are many markup languages now, they all are just simplifications of HTML. There are a few reasons to use a higher-level language than raw HTML, though:

HTML to extremely verbose. Even a simple list like this one is more complicated than it has to be.
There are special characters that need to be replaced. For example, "&" has to be written as &
Although rare, there are changes to the HTML standard. To bold text, <b> is now written as <strong>, but <span style="font-weight: bold;"> also works.

I've bundled the source code for the current release (2.1.6) of Discount with my web content. This means that no amount of upgrading will break my site anymore. If Discount 3 ever appears, it won't break my site until I choose to update my site to use it, if ever. I no longer rely on rdiscount. With Discount, I also get SmartyPants punctuation, which I prefer.

The longest part is to convert all of my current archive to a unified format.

Other Required Features

There are a few other things I need to make life easier:

Templates - I want to attach a header and a footer to every page automatically. If I change the template, all the pages should update on their own without manually copying and pasting into hundreds of files.
Page-specific variables - I should be able to set the title in the <head> section of the outputted file in the article itself without maintaining a separate database.
RSS feed, sitemap, and index generation - I'm not going to manually create the RSS feed; I will forget. Also, it's tedious. The point of using a script is to allow me to write without dealing with site creation tedium and preventing human error.
Macros - I prefer to write something similar to <youtube xxxxxxxx> rather than pasting the embed code in. This is because the embed code changes with time. Now that YouTube is using HTML5 with Flash as a fallback, I'd rather my embedded videos use the new player without hunting for them all and modifying each one. Using a macro that I modify in one place to automatically update all the others is great.

This can all be done with a simple shell script. I wrote one that ended up being remarkably similar to sta.sh. The only real difference is that I don't really use awk for anything, so I don't really know it. Mine is completely sed based. I also rely on bash 4 features rather than being sh based. I also use rsync rather than removing and recopying the entire destination directory on each build.

My final script uses symlinks rather than copying the files, so I won't have multiple copies of pictures and large files on my hard drive.

Comparison

	Jekyll	Shell Script (Copying)	Shell Script (Symlinks)
First Run	40 seconds	20 seconds	1-2 seconds
Rebuilding with 1 File Changed	10 seconds	< 1 second	< 1 second

What Does This Mean?

While the script is highly customized to my workflow, being that I wrote it, it is extremely fast and extensible. It is still single-threaded, but it runs so fast that I don't care.

I had originally created the script as a GNU Makefile, but make 3.82 broke a lot of what I use at work. make 4 is coming soon, and I'm expecting a nightmare. I'd rather keep it simple. The reason for creating this script is to remove my reliance on ever-changing tools, and the parallelization advantage I may have gotten is pointless. I'm now limited to the speed my hard drive can manage, and sub-second run times are more than adequate for a hobby site. I wasn't expecting such performance.

This means that the drafts I've been writing and the bookmarks I've been keeping for the past few months may finally be used now. I can write again!