Building The Joomla.com Hosting Platform

When SiteGround was chosen by OSM to be the official hosting provider for Joomla.com – the new free website service by Joomla, we were really excited. We knew that creating an infrastructure that needs to host hundreds of thousands of sites will not be an easy task. This was a huge and complicated project, but our passion for complex technical challenges was what made us participate in the RFP in the first place. Of course, we were additionally motivated by our love for Joomla and by our previous long-year experience of hosting thousands of Joomla sites on our own servers. Many people from SiteGround and from the Joomla! community have done some amazing job on the different aspects of this new service like creating the website, designing the templates and developing the extensions. However, in this blog post I will concentrate especially on the technical challenges that our DevOps team encountered while designing, developing and implementing the hosting platform for Joomla.com.

The Goals

If you check the official Joomla.com site’s footer you’ll see that users get access to modified version of Joomla! CMS hosted on a special hosting infrastructure built by SiteGround. We had to develop this special hosting infrastructure because this project has some unique requirements:

1. It must be extremely efficient, so that hosting millions of websites for free is sustainable.
2. The websites should be blazing fast and highly secure, so that the user never associate Joomla CMS and SiteGround hosting with negative experience.
3. Starting your new Joomla website should be as easy and fast as possible, so that people do not get discouraged on the first step.
4. The infrastructure should be highly scalable, so that it grows seamlessly with the number of its users.

Below I will go over the technical decisions we made to achieve these goals.

One Joomla! to Rule Them All (efficiency and safety)

The main efficiency challenge that our team had to overcome was that hosting a big number of Joomla websites, even if they are just inactive default installs, requires quite a lot of space. Just to give you some numbers, the latest Joomla! Installation archive (3.3.6) contains 6952 folders and files. On a Linux server every file or folder uses an inode. The size in MBs of these files is not big, but the number of inodes on a server is limited. Even people that are not big mathematics fans will understand that installing a new Joomla! app for every new client will quickly cause the drives on the servers to become full.

We needed to think of an original solution how to minimize the number of the inodes needed and what we came out with was that only one Joomla! CMS application will be shared by all the clients. The idea was to create a Joomla! CMS skeleton that contains the core Joomla! files and then link all sites to this one skeleton. We identified the folders and files that should be part of the skeleton and we created it. Of course, we also considered the folders, such as the images directory, that should be unique for every site. Once the skeleton was ready, creating new accounts was easy because we just had to create symbolic links to it.

On top of the resource usage efficiency that resulted from this decision, we achieved two additional things that highly increased the security of the system. (1) Using a single Joomla made the upgrade process really easy. Once a new Joomla! version is released we just upgrade the skeleton’s codebase and all clients are instantly upgraded to the new release. (2) Using a single Joomla meant that the users themselves will have limited access to mess up with the system files. The upload of external extensions and templates is not allowed as a result and this makes the security of the system much easier to maintain.

Custom Caching (efficiency and speed)

Another one of the key technical solutions that allowed us to achieve even more efficiency from the system and to ensure that the users’ websites are really fast is the custom caching that we developed specifically for this platform. We used the famous Nginx web server and PHP-FPM to cache information and speed up the delivery of static resources and dynamically generated data. We installed Nginx and configured the PHP-FPM and then we found out that we’ll have to patch them in order to use them. The first problem that we faced was that we did not have the ability to purge the cache only for a specific domain name. We downloaded the ngx_cache_purge module, extended it and tested it on our end. The issue with PHP-FPM was that it lacks support for chroot and this was essential for us due to security reasons. Again, it took us some time to do it but our DevOps engineers patched and extended the code and we compiled our own PHP-FPM.

Dynamic DNS (immediate provisioning)

Now that we have created fast, secure and efficient environment, we had to think of a way to allow clients to use their new Joomla! sites right after they created their accounts. When one creates a new Joomla.com account he/she chooses a unique sub-domain name. We all know that the DNS sometimes needs time to update all DNS records in all databases used all over the world. Many internet service providers cache DNS data and in general it is not cool to wait to use your newly created free Joomla! site. To solve this issue we decided to use something called Dynamic DNS. This technology allows us to quickly map domain names to IP addresses and maintain certain accessibility levels using well-known addresses. The result was immediate account activation and access to your website without the need to wait for any propagation time.

Linux Containers (infinite scalability)

As I mentioned, we had to find a solution that should be highly scalable. At the same time we needed something that is easy to configure and automate the whole process. We decided to use Linux containers mainly because they offer almost unlimited scalability options. For example, we can configure a container to automatically scale-up if it uses more than 80% of the available RAM. The same is valid for the storage capacity and the available CPU power. On every container we also use two MySQL instances to ensure that the servers will be able to handle more concurrent connections.

Now that Joomla.com is already functioning we have started welcoming a big number of Joomla! sites on the infrastructure described above. I am pleased to say that so far we haven’t faced any major issues and the platform is handling flawlessly the traffic towards the existing sites, as well as the launch of the new ones. We are looking forward to the bright future ahead of the project!