Global Database (further GDB) is one of the business intelligence market leaders, that provides company details and other information such as email addresses financial, credit risks and due diligence checks, digital insights and a lot more. The company delivers its provisionings via an online platform that enables its users to filter down data according to various criteria, enrich their own data and use perishable up-to-date information via the platform’s Application Programming Interface.
GDB faced one of the most common challenges: the global issue of horizontal scaling. In only 3 years, the company has grown from a tiny startup to a fierce competitor within the business information markets, challenging industry giants such as Duedil or DiscoverOrg.
As their customer base grew, serving real-time data to a quite demanding audience became less of a luxury and more of a demand: “We approached EBS with a data processing dilemma. Since GDB handles hundreds of gigabytes, processing bottlenecks were starting to cripple our service and the only solution we could think of for the long term was increasing server resources. This has been a painful quick-fix because, in some instances, additional resources were simply not enough. We reached a point where we’ve been spending crazy amounts of money, just to keep the platform up. Service Latency has also been an issue and we understood that a simple patch would not make the cut.” – states Mr. Nicolae Buldumac, Managing Director, at Global Database.
At the other end, EBS Integrator’s leading system engineers were facing another dilemma: the one of building on top a live system while reducing outage incidents to a minimum. “Given our customer’s circumstances, we knew: time is of the essence. The entire development cycle could take over 6 months and by the time the new version of GDB was done, their stakeholders might have reached for a better service to their competition” – states Mr. Ion Aremescu, the Lead System Engineer for GDB project.
To solve GDB’s most obvious pain-points, EBS had to go back and rethink the entire system architecture. Since GDB’s platform was data critical, the solution would resemble a serious revamp on a live service, a risk few providers would take. After weeks of analysis, EBS decided to build an infrastructure defined by Kubernetes on Docker, Kafka, ElasticSearch, MongoDB, PostgreSQL and Ansible, while the CI/CD as well as monitoring tasks would be handled by Graylog / Zabbix and Jenkins.
The greatest advantages delivered by EBS’s infrastructure is the possibility of introducing Java and other secondary programming languages that work seamlessly, regardless of the state on another microservice or container.
The second most great advantage was the possibility to migrate the existing system to a container-based infrastructure gradually. “Rome hasn’t been built in a day and the dynamic migration process enabled us to go from a legacy monolithic system to a micro-service paradigm in a step-by-step manner to avoid massive data loss or service outages.” – stated Mr. Iulian Ciobanu, EBS’ Chief Information Officer. The gradual migration advantage enabled EBS to maintain an online service while migrating key components to microservice structures.
“To solve the most critical GDB challenges we’ve started with the way data is processed. Since our new infrastructure made use of Kafka, we could change the entire data paradigm and enable an asynchronous data processing model and this allowed us to deploy several clusters for Elasticsearch. As a result, query times have been reduced with 80% building premises for a healthier and more reliable processing environment.“ states Mr. Ion Aremescu, the Lead System Engineer for GDB project.
With no doubt, the most lucrative strategic decision EBS took was to migrate GDB to a microservice-based structure powered by Kubernetes. Before going this path, processing data arrays could take up to a minute. Now, GDB’s data processing metrics fell under 20 seconds per high-load queries and in some instances, these times go as low as 800 milliseconds. Given their fail-proof strategies, EBS engineers delivered a system aware of its own resource consumption. This allows resource adjustment based on specific cost policies processed an AI algorithm.
“What EBS did has certainly exceeded all our initial expectations. We’ve seen impressive cost savings on our server resources, an UX that is faster, snappier, more relatable to our end-users and we haven’t had an outage in months. Furthermore, the new system design saves us an average of €100 per day. At this rate, the GDB rewrite project will recoup itself within the next 3 years.” – states Mr. Ion Aremescu, the Lead System Engineer for GDB project.
The entire transition process took a bit longer than a system rewrite would, however, this build premises of making it possible on an always-on service. During 8 months of development, GDB has been up and running without any interruptions, delivering the benefit of interactive end-user feedback regards new features and improvements performed on the platform:
“Working on the live system in the manner EBS proposed, actually paid off. We got interactive feedback regards eventual features, the overall system performance and lucrative suggestions from live customers – this is how we got as close to our end-users’ needs as possible. Looking back, I honestly believe that working with EBS, has been and is a total bargain.” – states Mr. Nicolae Buldumac, Managing Director, at Global Database.
At the moment, Global Database is one of EBS’ centerpiece projects, enabled by Continuous Integration and Delivery. Since EBS started on the right foot, the team can now deliver new features and platform improvements in a non-intrusive manner – with a 99.99% uptime in service delivery.
Get in touch to find out more about how
our experts can help your business