In the last blog post about previous versions of MétEX, I talked about how I'd always used server-side frameworks such as CakePHP and Rails. These are excellent for creating heavily interactive, stateful, database-dependent applications but they seemed a bit wasted on MétEX. After all, the content doesn't change often, and there's no user state to keep track of. So for the current version of MétEX I decided to look into static site generators.
The idea here is to move when the heavy lifting occurs. On traditional server-side applications, the server does all the work for every single request. Every single user that requests a page, the server interprets the code, queries the database, compiles the HTML, and serves it back to the user. For data that changes minute by minute, user by user, that works great. But MétEX doesn't need that flexibility. So instead, all the heavy work is only done once - when the site is built, or compiled. This way, the server only ever has to serve static files and not do any computation, speeding up page loads immensely and improving caching.
There are several established static site generators used mainly for blogs, but I wanted to stick with a JavaScript framework I'm very familiar with - Angular. I find Angular an excellent front-end framework for single-page apps, and the move from the server-side MVC frameworks was very easy. And as luck would have it, a static site generator for Angular apps has recently been getting mature enough to use in production.
I quickly decided that Scully would fit perfectly. It has both its own blog/markdown file parsing, as well as being able to cache HTTP/AJAX requests. It has good plugin support, being able to create sitelists and having the option to actually disable the Angular code on generated pages, which is a good idea if you don't have any real interactivity on your site. Unfortunately MétEX does require JavaScript for image carousels and the upcoming Journey feature, so Angular does still have to load.
MétEX has always been database-driven. There's always been a relational database at the heart of the site, and I still think this is the best choice going forward. I did look at using Markdown files with Scully and using front matter for metadata, but I decided this would get far too messy. A relational database is definitely what MétEX needs, since Stations belong to Lines, Lines are linked by Movements, and the data needs structure and enforcement in order to work properly. That's where the headless CMS comes in.
If you've used something like Wordpress, then you know what the interface looks like - it's both a front-end for editors to add content to the site, but it also exposes an API that any script can query, just like a database. Strapi is great because as a developer I can easily configure the structure to make sense for my content. I can now add Lines, Stations, Interchanges, Movements, and Places in a friendly user interface.
On the API side, I wanted to experiment with GraphQL, so that's how all data queries are made. The Angular front-end makes GraphQL queries to the Strapi API, requesting the Station and Line data as necessary. But hang on, isn't that starting to sound a lot like the old style MétEX, where every page load incurs a database query and a lot of elaboration? Well, only in development. Because on production, Scully does its magic - when the site is compiled, Scully scans the site for every single possible request that the user can make, then requests that information from Strapi. This data is saved in JSON files, and it's these files that Angular use to serve up the site. If you open Web Inspector or your browser's equivalent, you'll see that no external requests are made. The headless CMS is only queried when the site is built, and not at all when a user hits the site.
Angular has built-in support for PWAs, essentially websites that can be installed like apps. Try it now! You'll get a MétEX icon on your home screen or desktop. And what's more, all the static content (minus the images and audio) is pre-loaded so navigating between stations or lines is super-fast as no network requests are made. Additionally, once you've downloaded the images for a station, they're cached and should work offline.
MétEX's PWA support is only at its initial stage, so I hope to improve its performance and utility in future updates. But as long as you install the site once, you'll automatically get any future updates!
One of the great disadvantages of using a server-side framework based on PHP or Ruby is that you need a hosting provider that supports these languages, along with a compatible database. You also need to make sure that everything's kept up to date and security patched. One recently popular solution to these problems is to use a cloud-based solution, such as AWS or Azure. You don't have a traditional web server anymore, instead you rent processing time from one of these companies. In theory this is great, since you don't have to worry about the infrastructure anymore, just your own code. But running an MVC app on one of these providers can end up being quite costly since a web server has to be permanently running, ready for requests.
I've been using Heroku as my cloud provider of choice since the Rails version of MétEX. As the site doesn't get that much traffic, the free tier has always been enough for my needs, but that comes with a heavy penalty - the server essentially shuts down when it's not being used, and takes about 30 seconds to spin back up when a request comes in. This means that the old version of MétEX would sometimes take ages to respond to the first request, which isn't ideal. If I wanted to upgrade to the next tier of service, I'd be paying around $7 a month - which doesn't compare favourably to a traditional web host (keeping in mind the low traffic this site gets).
But that's where the genius of using a headless CMS comes into play. Remember, the CMS is only ever used by editors (so... just me), and by Scully at compile time. And that's it. So I just make sure that Scully doesn't time out too soon and the performance penalty is no longer an issue for users.
As for the front-end, I've been playing with Netlify for a while, and it's a great static site host. Like Heroku, it has great Github integration, so whenever I make a code change (or even a content change - Strapi has good webhook support), Netlify will grab the latest code from Github, compile the site, and serve it automatically. I'm not even using half the features Netlify offers and - once again - I'm still only using the free tier.
OK, so - Headless CMS on Heroku. Front-end on Netlify. But where does the media come from? The photos and the audio? That's all stored in an AWS S3 bucket. I'm not really a fan of Amazon's naming scheme since it's impenetrable for most people but essentially it's a bit like a web server that you can upload to using an API. I mean, I still use my favourite FTP application to manage it, but Strapi can also directly upload files to it from the editor's interface. Again, the main advantage here is cost since the most I've ever paid for this so far is £0.02 a month. Literally pennies. And if MétEX should ever get so popular that it needs more bandwidth, it's trivial to increase its availability - on all three of AWS, Heroku and Netlify.
So I hope this has been easy enough to follow and understand. Just to make things clear, here's the entire pipeline for a code change:
Yes, it's more complicated than a traditional server-side web app, but in the long run I think that MétEX will be easier to update, quicker for users, and also cheaper to run! Now that the site's architecture has been completed, it's time to get to work on new features!