One of the fantastic things about Go is the ease with which we can execute concurrent tasks using goroutines and channels. But using goroutines and channels in production env without proper context on how they behave can cause some serious effects.
Well, we faced one such impact where we had a leakage in goroutines that resulted in the application server bloating over time by consuming abundant CPU & frequent GC pauses affecting the SLA of multiple APIs.
What to expect from this article
We all know Django is an awesome framework that provides a default admin interface based on the models. The default functionalities that are supported by Django solve 95% of our use cases but there are quite some instances where the application needs certain functionality and we’ve to extend the inbuilt Django functionality the way we need and it's a nightmare to newbies.
One such instance is where we are desperately in need of the below functionalities.
Things I try to cover in this article
What is a UDF?
UDF means User Defined Functions which means the database has to run the user-defined function in its own context at run time and return the response back to the client.
The user-defined function can be written in any language that the database understands. In the case of Aerospike, UDF’s are written in Lua which is a scripting language. …
Out of hundreds of optimizations that a developer thinks of, reducing the docker image size will be given the least priority. So did we.
As a result, our docker image started to weigh around 2.4 GB in size which indirectly resulted in our developer’s productivity. Below are the issues we faced with larger docker image size
-> Delay in running automated Integration Tests
Whenever we raise a merge request to the release branch, we have an automated process that builds a new image out of the raised MR code, spins up a new container out of it, and fire thousands…
If there is one big transition that took place in the past decade or so in terms of application development style and deployment patterns then it has to be migration from Monolithic application development to Microservices.
The latter had provided us with various advantages by splitting the bigger application into various smaller chunks with each chunk being called a microservice.
When you ask a developer, what are the two key things that are required for better performance of an application, the answer will surely be
When it comes to monolithic architecture, we estimate the total CPU and RAM required for an application and pick one huge machine with a very huge number of CPU Cores as well as hundreds of GB’s of RAM.
The issue with such huge machines is they are pretty costly compared to multiple horizontally scaled lightweight machines as well as they are hard to get one instantaneously unlike lightweight machines…
Working for an E-commerce company with a pretty huge customer base, the primary thing that we always keep in our mind while we code is to keep it performant enough to serve huge traffic.
As a developer, we benchmark the application with lakhs of requests and try to come up with a breakpoint on what’s the max load that the application or database server can handle at any given point of time. We call that as SLA(Service Level Agreement) which states about the max requests the system can handle, 99% response time, uptime, and a few other stats.
One primary factor that attracts the end-user is how quickly the application responds back to the user request. Applications perform pretty well when the traffic is low but things start failing once the load increases on the application.
Before taking the application to production, there will be rigorous benchmark testing on what’s the max load that the application server or database server can handle. …
With a rapid transition from Monolithic application development to Microservices based application development, communication between microservices has become a crucial and core component. We need some sort of common language pattern where all dependent microservices should be able to communicate irrespective of programming language.
In continuation of my previous article where I’ve detailed on the limitations of goroutines, in this article, I would like to share how we achieved 70 million sets to Aerospike Database in 30 minutes by leveraging the entire system Cores with the help of Go Worker Pool Pattern.