We deleted our service layer… 😱
In early April we kicked off our private beta to gather insights into the app and find any pesky bugs that weren’t squashed during internal testing. Amazingly there were only a few bigger issues reported. Of those three bigger issues, two were fixed over the course of a day or two. The last issue is a different story. Let’s talk about it.
ORMS & You 🤓
On paper, ORMs are a useful tool to help with modelling your service layer data and dealing with the persistence and relationships of those models. But not all ORMs are built equally, and in the case of Tower’s service layer (internally called Pillar,) this wreaked havoc when testing went from two accounts to three or more due to mis-using part of the ORM, and, to be transparent, SQL itself.
So here’s a warning to all those trying to decide on a tech stack: choose your ORM carefully.
The bug 🐛
Let me start by saying that the bug was an “us” problem. We mis-used part of the SQL language, specifically using a JOIN instead of a nested / sub-query. Looking back, it was a silly mistake but it went overlooked for so long because when testing with only two accounts a JOIN will work the way you expect it to.
After going through the reports on why some users were seeing “open” conversations when they had already requested to join them, it was very easy to see why this was happening, and a fix should be super straightforward. Right? RIGHT?!
We didn’t choose the right ORM 😢
After trying for days to figure out how to actually write the code to deal with the bug and coming up empty handed, we reached out to the authors of the ORM we used and, unfortunately, there wasn’t any way to do a sub-query on a relationship as a single operation while also keeping the ergonomics of the ORM’s pagination and relationship loading. We asked the authors when this type of thing might be possible and they were only just starting to add support and nothing would be officially released in the short-term.
So, lesson learned. We didn’t do enough research to our specific use case.
/Pillar. Right click. Move to Trash. 🗑️
Okay not exactly, but like… pretty much.
We dropped our previous tech stack for the service layer and made more calculated decisions. When Tower was first conceptualized, Vapor was an attractive stack, including their ORM, Fluent. There was a lot of nice features and functionality that leveraging Vapor unlocked, but looking back in never felt quite right. We probably “held it wrong”, admittedly.
So given that we dropped Vapor, what did we end up with? Something not very notable at all. But that was actually a selling point. Hopping on the new and fun trends that haven’t been thoroughly battle-tested bit us once already, so we went for:
ExpressJS. Woo. 🦗
Leveraging TypeScript gives us some of the functionality we care about, such as build-time type checking. We did go with something flashy, though: Prisma. We tested a number of ORMs and found Prisma to be the most ergonomic while developing.
With ExpressJS, Prisma, and TypeScript we get something that’s both modern and proven, and as massive benefit / byproduct is that our deployment times reduced by over 15 minutes. Our Docker image build times were upwards over 18 minutes, whereas now it’s down to 2.5 minutes.
Wrapping up 💫
Ultimately, the choice for your tech stack is incredibly specific to your product, but the moral of the story here, at least for us, is to be afraid to chalk up losses and pivot. While it’s almost never feasible in enterprise products, Tower is still at a stage where if we need to make changes we can. And we do.
Finally, keep an eye out for our upcoming public beta later this month. We are re-entering our private beta shortly with our new tech stack. If all goes well we’ll be moving to public beta soon after.
