Github Pages and Jekyll - A New Platform for LACNIC Labs

Problem Statement

Our “LACNIC Labs” website has been hosted on a Drupal CMS since 2010. While this platform has provided good service to us, we believed it was time for a change.

Our requirements for a CMS platform are perhaps a bit different than other people’s. We are small team and we are always struggling with the maintenance of a complex platform involving PHP, MySQL and a variety of Drupal plugins for different purposes (WYSIWYG editor, access statistics, comment moderation, etc).

For example, we were interested in:

  • A platform requiring minimal maintenance, presenting minimal attack surface. Ideally it should be database-free and requiring a bare minimum of plugins and extra software installation.
  • Simple support for offline post edition. Although composing simple posts on a browser window is ok, longer documents are best edited offline, proofread and then uploaded.
  • Support for light publication workflows . While we feel the need for some publication QA control and post approval, we did not want to be encumbered by “heavy” approaches as supported by most CMS platforms.
  • Versioning. Some form of file versioning and change control is necessary for us, but most database-based CMSs struggle with it.

These requirements point to a stafic file-based platform. Plain files integrate well with normal software development practices.

Meet Jekyll, the Static Website Compiler

Jekyll (https://jekyllrb.com/) is a Ruby-based tool that works as a “compiler” of sorts. In “blog” mode posts are written using either Markdown or full-blown HTML. It has support for theming and layout control and produces a static file tree than can be simply served off by Apache, NGINX or any other webserver. No database is required

Jekyll favors “convention over configuration” making it relatively simple to configure and maintain. Since all material is stored in files, a Jekyll site can be stored on a git or svn repository.

Markdown offers an easier writing experience than using full HTML. Markdown can be edited easily using plain text editors but, for those so inclined, there are a variety of apps and web pages offering WYSIWYG Markdown editing tools.

GitHub Pages

Github, the very well-known social coding platform offers a service called Github Pages that can be used to serve a set of static HTML files from a Github repository.

Using Github Pages it would be possible to leverage Github’s online Markdown editor as well as its “Issues” trouble-ticketing system as a way of implementing change control. Versioning comes naturally as everything is hosted on git repositories.

Github Pages websites can be anchored to custom DNS names using CNAME records.

Platform Choice and Solution Architecture

At first sight Github Pages seems to offer all we need. However, we discovered a requirement that proved not so easily satisfied. We need to anchor LACNIC Lab’s website to both “www.labs.lacnic.net” and to “labs.lacnic.net”. While the first can get a CNAME and be served off Github, the latter cannot as CNAME records are not allowed at the zone apex.

Another issue was that by using Github pages and a custom domain name we would not be able to provide a TLS certificate (as Github provides certificates covering many *.github.com and *.github.io subdomains, but not custom CNAMEs). In that scenario we’d have to serve all our traffic through plain HTTP, something we’d obviously wanted to avoid.

Serving the site locally off one our servers solves both issues easily, but we would lose some of the workflow benefits we got from hosting directly on Github Pages.

Luckily, this is one of those situations where you can get the best of both worlds. Since everything is hosted on a git repository, we just need an automatic way of running git pull && jekyll build when the site’s github repository is commited or pushed to.

Github’s “webhooks” come is our tool of choice here. A webhook in short is an HTTP request triggered by an event.

  1. Upon push or commit, github invokes a webhook
  2. When our website server receives the webhook runs a script
  3. The script runs:
    1. git pull
    2. jekyll build

diagrama

Webhooks in Github

Webhooks in general are HTTP requests triggered by platforms when certain events ocur. In GitHub, webhooks are configured per repository and carry a rich JSON payload with information about the triggering event.

Webhooks, general documentation :

https://developer.github.com/webhooks/

Event “PageBuildEvent” : https://developer.github.com/v3/activity/events/types/#pagebuildevent

Tools for building webhook servers

We need an application listening on our server waiting for the incoming webhook calls. While this could be implemented using older technologies like CGI scripts, webhooks have become such a fundamental element in web architectures that specific tools have been developed. Just to mention two we have “goexpose” and “webhook”.

  • goexpose:
    • https://github.com/phonkee/goexpose
  • webhook:
    • https://github.com/adnanh/webhook

In our case we settled for webhook. It provided enough features covering our use case and looked easy to configure. We are running it off an “user level” SystemD service unit, listening on port 9000. User-level services allow for services and daemons to be easily run as non-root users. Internet requests are reverse proxied using NGINX.

Solution configuration snippets.

While a full tutorial of how the publication workflow was built is beyond the scope of this post, some configuration snippets are included as reference:

Webhook URL:

sausalito:dns-workshop-carlos carlos$ echo "es diciembre, papá"|md5 
636787f7644d22533cd699409632205e
sausalito:dns-workshop-carlos carlos$ echo "es diciembre, papá"|shasum
4948fae9c2a21c34154562335e9219c0c95de70d  -
sausalito:dns-workshop-carlos carlos$

Git Credential Cache:

git config --global credential.helper 'cache --timeout=3600'

NGINX Reverse Proxy Configuration:

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        location /hooks/ {
                proxy_pass http://127.0.0.1:9000/hooks/;

        } 
}

Webhook “weblabs_hooks.json” file:

[
   {
          "id": "4948fae9c2a21c34154562335e9219c0c95de70d",
          "execute-command": "/home/weblabs/lacnic.github.io.git/redeploy.sh",
          "command-working-directory": "/home/weblabs/lacnic.github.io.git/"
   }
]

Webhook command line invokation:

webhook -ip 127.0.0.1 -hooks weblabs_hooks.json -verbose

Automatically running “webhook”

Enable user linger:

 sudo loginctl enable-linger weblabs

SystemD user-level unit:

[Unit]
Description=Weblabs hooks for autopull && build
After=network.target

[Service]
Type=simple
WorkingDirectory=/home/weblabs
ExecStart=/usr/local/bin/webhook -ip 127.0.0.1 -hooks weblabs_hooks.json -verbose
Restart=on-failure
PrivateTmp=true

[Install]
WantedBy=default.target

References

  1. https://www.brendanlong.com/systemd-user-services-are-amazing.html