28 March 2019

The Equalture Story (5): Why and how we rebuild our software.

The Equalture Story. A series of stories to give you a look behind the scenes of our company. Milestones. Failures. Lessons we’ve learned. Just the honest story about a SaaS startup trying to conquer the world.

In this very first tech blog, our CTO Jaap explains why and how we rebuild our software.

They said ”YES”!

I joined Equalture in the summer of 2018. At that point we had already started selling the product as it was ‘finished’. I started by assessing the quality of code, scalability of the setup, security, documentation and overall resilience of the platform. What I found was a product that was extraordinary from a functional perspective, but left a lot to wish for from a technical perspective.

So we sat down at the first weekly session with the new team (all 3 of us) and I that in order to support the ambitions of this company, we’d need to rebuild our platform. And while this is not the first time I’ve told a company that they should rebuild or refactor their software, it was the first time that I was met with a resounding “YES”!

So, I started planning the architecture and the team to support it. But first, we had to make sure that the current product would remain stable for the time to come.

Meet Chip’n’Dale

Chip and Dale are our CakePHP backend and AngularJS frontend. They were deployed on shared hosting by running git pull through SSH. Config management was done through editing files using anyone’s favourite text editor (vim, only vim).

In both Chip and Dale, we saw a plethora of code duplication, lack of refactors and code removal, inconsistent data structures and somewhat poor design. This is all the result of a year of software development under high pressure with limited knowledge of what the end product would look like, performed by developers with very little knowledge of each others’ respective domain.

And like the actual cartoon characters, it’s hard to tell the difference between Chip and Dale. Dale is located in a folder called “front-end” and Chip is not, but it’s never been totally clear which concern or piece of logic goes or resides where. This has led to a lot of confusion which limited the ability to keep our code clean and maintainable.

Phase 0 – upgrading, sanitisation and integration

So, we started hiring developers to help us stabilise the situation. We started by upgrading Dale to the latest version of NodeJS, replacing Gulp with Webpack and Bower with NPM. This took 1 person about a month to complete. After which the frontend was much more stable, predictable, faster and easier to build.

We also started removing redundant code from Chip by introducing some rules. We agreed that every change in code should improve the state of the software. We had no measurements for code quality, but code review helps a bunch when trying to keep each other on top of their game.

We also introduced automated build pipelines using GitLab CI which decreased the time spent on deployments and other tasks related to shipping code. Furthermore, we introduced Docker for local development.

All this helps, but we’re still not happy with the state of the architecture and infrastructure. Quite frankly, CakePHP is just not a framework I’d recommend for a new project in this day and age and AngularJS has been long deprecated.

Introducing Tom and Jerry

That’s why we started rebuilding our software. Key elements we’ve been looking for are scalability and maintainability, which meant we started looking at software development languages and frameworks that supports modular design and concurrency with clear and concise syntax and sane conventions.

For the backend (named Jerry), we quickly landed on Phoenix (Elixir). The combination of functional programming in Erlang with conventions stemming from over 10 years of lessons learnt by the Rails community appealed to us. We built a POC that connects to our current database which we were able to bootstrap in a very short timeframe.

For the frontend (named Tom), we picked React. We know and trust it and it supports a very modular design which enable us to use components which we can include in our bootstrapped version of Jerry. Also, the current state of React is very clean and the new React context API combined with hooks may actually reduce the headache that came with previous attempts at state management.

Phase 1 – building and deploying the admin scope

So far, we’ve been working on building a management interface for the admin scope in Jerry (using Phoenix’s own templating engine, EEx) and we’ve been making huge progress. We are able to quickly rebuild our interfaces and have been working on increasing the team’s knowledge of Phoenix and best practices, while thoroughly researching all components of the framework and ecosystem, every step of the way.

We’ve also started migrating our infrastructure to the Google Cloud Platform using Kubernetes. We’ve integrated it with Gitlab CI and are now actually considering moving to GitLab flow much earlier than we could have anticipated because we can now start introducing a test-centered workflow that will eventually reduce the risk of regression.

So what’s next?

Next we’ll be looking at replacing the current admin scope with our new admin scope, meaning anyone who logs in as an admin through Chip’n’Dale will redirect automatically to Jerry. We will also be looking at getting started with building our first components in Tom and introducing Tom and Jerry to each other. Let’s hope they get along!

Cheers, Jaap