Building Static Sites in 2017: Cloud-Hosted, CMS-Backed, and API-Driven

In this post, we survey modern static site generators and how they work with content management and APIs for dynamic data. As an example, we’ll develop a content site using Jekyll for templating, Jekyll Admin as a CMS and Baqend’s real-time API for syncing comments.

TL;DR

Static sites provide an unmatched level of scalability and performance by completely avoiding an active backend. Modern static site generators such as Jekyll, Hugo and Hexo make it very easy to build static sites from template languages and largely technology-agnostic. For content-driven use cases like blogs and publishing sites, content management systems for editing static sites have evolved. And for the use cases that require integration of 3rd party services, data storage, and user management, Backend- and Function-as-a-Service APIs provide an option for implementing server-side logic with little code. This survey gives an overview of the static site technology stack that incorporates static site generation, content management and opt-in APIs for dynamic content. We will review the different approaches and walk through a real-world example of using the stack.

Introduction

Frontend, network, and backend performance used to be tough. But with the right technologies (e.g., CDNs, build tools, and HTTP/2) things are getting much easier (see our survey). However, most performance problems stem from single-node systems such as WordPress or Magento. Both are based on server-side rendering and data storage in a single MySQL instance and are therefore notoriously hard to scale.

It’s not unusual for a WordPress site to have a time-to-first-byte (TTFB) of 1–3 seconds. At the same time, 49% of users expect websites to load in 2 seconds or less, according to a survey by Akamai. And if that time is already spent for server-side rendering, there is little chance of hitting that performance goal. What makes it worse: Google uses the TTFB as a metric for their search ranking.

The good news is: you can go from your existing WordPress, Drupal, etc. to a fully static site that is still CMS-backed, as we’ll survey in this article.

Going Static

Static sites have a significant advantage: They do not require server-side rendering or database queries, can be deployed virtually anywhere and are great for caching.

Static Website Architecture

Static site generators are great tools for building static sites that use rich templating in order to keep code clean and avoid duplication. They run on your local machine and compile a static site from a bunch of content files stored in your file system. The most popular ones are:

  • Jekyll (Ruby-based, 30132 Github Stars) is the most popular static site generator. It has a huge community, many plugins and great support for blogs.
  • Hugo (Go-based, 18057 Github Stars) excels in building speed: complete projects can be generated in milliseconds. It uses a single binary, making it easy to install for blogs and documentation.
  • Hexo (Node.js-based, 16875 Github Stars) is getting more and more popular as it fits nicely into the Node ecosystem and is easy to integrate with Octopress and Jekyll plugins.
  • Gatsby (Node.js-based, 10041 Github Stars) builds single-page application blogs based on React.js with reusable components.
  • Metalsmith (Node.js-based, 5948 Github Stars) is very low-level and highly extensible, so with the right tweaking it can build almost any kind of website.
  • GitBook (JavaScript-based, 15605 Github Stars) publishes ebooks from markdown upon pushes to Github.
  • MkDocs (Python-based, 4135 Github Stars) is a static site generator specifically made for building technical documentations from markdown.
  • Octopress (Ruby-based, 1576 Github Stars) is a Jekyll fork particularly geared towards hackers.

In our example, we will use Jekyll, which is the system that also powers Github Pages. In fact, it was created by Github co-founder Tom Preston-Werner back in 2008 as an engine for building static blogs. It is based on the Ruby toolchain and uses the Liquid templating language.

Building a Jekyll-based Content Site

What we’d like to build as an example is a simple content site where we maintain a repository of bachelor and master thesis topics for computer science students:

https://thesis.app.baqend.com/

So here is how you build a Jekyll site:

  1. Do you have a static website? Then you already have a valid Jekyll project.
  2. Install Jekyll. I’m running Windows, so what I did was: a) install the package manager Chocolatey, b) install Ruby: choco install ruby -y c) install Jekyll: gem install jekyll
  3. As I found Jekyll’s default theme a little boring, I opted for the nice-looking theme called Minimal Mistakes. Jekyll has a large ecosystem of designs and boilerplate projects.

To use and extend the example setup of this article, check out the Github project:

git clone git@github.com:Baqend/thesis.git

Project Structure

Here is what the Jekyll project structure looks like. The most important files and folders are:

  • The _config.yml contains global configurations settings in the YAML format, e.g. the site’s title
  • _site contains the rendered static site
  • _posts stores all the the blog-type content files and _pages the content for individual pages
  • assets holds your images, videos, etc.
  • In _data, YAML data is shared across views, e.g. author information
  • _layouts, _includes, _sass contain the design and templates

The great thing about Jekyll is that it not only builds your static site, but also comes with a local, live-reloading server.

Running Jekyll

Let’s run the Jekyll live server:

bundle exec jekyll serve

The command will start a server on port 4000 that serves your current version.

Adding Content Management

The reason why many people use WordPress, Drupal, Joomla etc. is that these frameworks are great for Content Management. Even unskilled users can easily create articles and landing pages using WordPress. The good news is: There are Content Management Systems (CMS) for static sites, too. Let’s add Jekyll Admin to the project.

To add a Jekyll plugin, the Gemfile is extended, which is Ruby’s equivalents to Node’s package.json: gem 'jekyll-admin', group::jekyll_plugins

Append /admin to the URL and you have your CMS:

Jekyll Admin CMS Interface

Jekyll is based on Markdown. When you create a post in the CMS, it will simply create a <post>.md file in your _posts folder.

Let’s walk through how Jekyll works, indepedent from how content is managed. The upper portion in a content file is called front matter. It contains (YAML) metadata used in the template, for example, tags of the post. The body will be rendered from markdown.

We specified to use the layout single which will use the Liquid templating engine to render to HTML:

This uses the<post>.md files’s front matter and body to output the static HTML page. For example, to add the post’s title, render markdown, and strip HTML and line breaks, we can use a Liquid variable and pipe it through helpers:

{{ page.title | markdownify | strip_html | strip_newlines }}

Adding media can be done by using the Static Files section in Jekyll Admin or by copying files into your asset folder. You can then reference them from you markdown to include them.

Liquid (developed by Shopify) does not allow any code in the template, so if more advanced logic is required, you need to create helper classes.

Jekyll Admin is also not the only CMS for Jekyll. For example, Forestry and Cloudcannon are SaaS solutions for doing static content management on top of Jekyll.

Building the Jekyll Site

Time to output a rendered site:

bundle exec jekyll build

This will put a deployable version into the _site folder. It uses many subfolders with containing an index.html to allow pretty navigation and permalinks. The _site folder can be deployed to almost any platform, for example, an Nginx Webserver or Amazon S3.

Deploying and Hosting the Static Site

Now, we simply have to upload the Jekyll project. Baqend has CDN-backed file hosting with automatic cache invalidation so let’s upload it there. All we need is the Baqend CLI, which can be installed from npm (alternatively you could also drag & drop the files in the dashboard).

npm install -g baqend #install CLI
baqend register #create a new app backend
bundle exec jekyll build #build the Jekyll project
baqend deploy -f _site thesis #deploy the _site folder to Baqend
baqend open thesis #open the deployed site in the browser

Adding APIs to Jekyll: Real-Time User Comments

The limitation of any static site is, of course, user-generated content (e.g., comments) and protected business logic (e.g., a shop checkout). This is where APIs of third-party services come into play, for exmaple, Disqus for comments, or Facebook for social data. If more backend logic is required, serverless architectures are the paradigm of choice: The frontend orchestrates different services (e.g., Paypal, Disqus, AWS Lambda) and contains most of the business logic.

What we want to have for our example, is a customizable comment section, where users can post comments and see any new comments from others users in real-time:

Storing Data from Jekyll

Combining Jekyll with JavaScript APIs is very powerful. You can use Jekyll to generate page information into your scripts and handle input data dynamically:

Therefore, let’s look at how we can use a Backend-as-a-Service to store and query data from Jekyll.

Backend-as-a-Service systems are designed to make serverless development easy, by having server-side code execution combined with standard APIs for things such as:

  • Registering and logging in users
  • Running scalable, server-side business logic (Function-as-a-Service)
  • Giving access to a database-(as-a-service)
  • Authorizing users and protecting content
  • Sending push notifications and performing synchronization
  • Hosting and delivering files

We are building Baqend, which is Backend-as-a-Service geared towards web performance. Baqend is a spin-off from database research and heavily based on caching algorithms.

Let’s add a commenting system to our static site using a Backend-as-a-Service. In order to display and post comments we have to do three things:

  1. Set up a data model of how to store comments in our backend.
  2. Save a comment, when the comment form is submitted.
  3. Sync the comments of a post across browsers in real-time.

Setting up the Data Model

Configuring the data model is extremely simple. Just head of to the dashboard of your Baqend App:

Now, expand the Data Section and hit + to create a new model. Let’s give it just a few String properties to store the poster, the (markdown) message, and the page (slug). A created_at and updated_at property are maintained automatically.

That’s it. We can now save comments. To try it, let’s open the data tab and add a few comments:

Saving Comments from the Static Site

Now let’s head back to our Jekyll project and add the code to post a comment in the custom.html file for custom comment handling.

First, let’s include the Baqend SDK and library for rendering Markdown and connect to our app. This will transparently download the schema and caching information.

Now let’s add the (plain old jQuery) code to take input from the comment form and save it as a new object:

Here we use the Liquid templating syntax with {{ page.slug }} to generate the current page id into the script on each post. The save() call simply inserts the new object over the REST API.

Querying and Syncing Comments

Now that we have the data in the database, we need to query it. There are two ways to do this:

  • A standard, pull-based query fetches data from the database and allows us to render it in the client (with jQuery, React, Angular, etc.).
  • With a real-time query, we establish a websocket connection to the server, perform a full-fledged MongoDB query and receive any updates to the result set in real-time. We call this a self-maintaing query.

The great thing is: Both look exactly the same in the code and are very easy to handle.

So as soon as we are connected (DB.ready callback), we use the query builder to retrieve comments for the current page, order them by creation date and call our UI-handler each time the set of comments is changed (e.g. a comment is inserted, removed, updated etc.).

And that’s it: Now we have our realtime-capable comment system that we can customize and extend however we like, while still having the abilities and simplcity of Jekyll for creating and content and generating the site.

Let’s deploy the commenting logic:

bundle exec jekyll build; baqend deploy -f _site

So there it is: our static website, with a local CMS, and APIs for on-demand dynamic functionalities. If you look at the developer console you can see comment updates coming in via websockets and static data being loaded via HTTP/2:

You can also configure your custom domain in the dashboard and let Baqend provision the SSL certificates for HTTP/2. By default you have an <app>.app.baqend.com domain that is already SSL-protected.

Securing Access

There is one last thing: we do not want to allow everyone to delete or update other people’s comments. So let’s secure the app by adding a schema ACL to only allow inserts and queries. You can do this in the ACL tab:

The last question is: How can we scale this project across many developers and many content editors?

Collaborating on a Static Site Project

Since a static website has all its context in the file system, sharing it is very easy. Content editors need to have Jekyll installed to use the Jekyll Admin CMS UI and they also need some way of sharing content files. The two wide-spread mechanisms to share the static project are:

  • Share the project folder via Dropbox, Google Drive, etc. This has the advantage of being very easy for non-technical users. However, there is no proper versioning, rollbacks, commits, etc.
  • Using Git with Github, Gitlab, etc. For obvious reasons, this is usually the preferred choice for developers and tech-savvy users.

The Baqend/API part is also easy to share, as you can simply add developers to the app. Each of the collaborating developers can edit the data model, view data, deploy the site, etc. Throug build tools (e.g., Gulp, Grunt, Webpack, npm scripts), the building, staging and deploying of the site can be made very accessible.

Migrating to Jekyll from Another Project or CMS

If you already have a static site project, you can simply start using the jekyll commad line for static site generation. You can simply embed Liquid templates in your HTML e.g.,{{page.title}}.

If you are coming from another CMS, Jekyll has importers that generate markdown files in your _posts folders. This works for Behance, Blogger, CSV, Drupal 6, Drupal 7, EasyBlog, Enki, Ghost, Google Reader, Joomla, Joomla 3, Jrnl, Marley, Mephisto, Movable Type, Posterous, RSS, S9Y, S9Y Database, Textpattern, Third-party, Tumblr, Typo, WordPress, WordPress.com.

For example, say you already have a WordPress blog. Simply run this locally, to pull content data from WordPress’s MySQL:

ruby -rubygems -e 'require "jekyll-import"; JekyllImport::Importers::WordPress.run({})'

That will import posts and pages. You can then use them in any of the numerous Jekyll themes or build a new frontend.

Resources on Static Site Generators

To learn more about static site generators, have a look at these:

Summary

The message of this post is: Static websites are much more flexible than their name indicates and inherently scalable. They can be combined with content management systems for creating posts and pages and with cloud APIs for handling non-static concerns like comments, payment, and user login.

The stack we proposed in the example project uses three central technologies:

  • Jekyll as a versatile static site generator with rich templating.
  • Jekyll Admin as a dead-simple local CMS.
  • Baqend for storing and querying data and hosting the site.

To use and see the stack in action, simply clone the example project from this Github repo.


Don’t want to miss our next post on building static websites? Get it conveniently delivered to your inbox by joining our newsletter.

Posts created 22

Leave a Reply

Related Posts

Begin typing your search term above and press enter to search. Press ESC to cancel.

Back To Top