The network in question is responsible for advertising performance of a service with entertainment content. For this task they employ a variety of solutions, including the Orbitsoft ad management system.
We were tasked with improving system performance while reducing maintenance and development costs. One solution was to change the programming language from PHP to Go. Here we’ll tell you how we solved this problem and what came of it.
-
01
Customer
a network that manages advertising for a service with entertainment content.
-
02
Business — tasks:
- reduce cost of maintaining servers required for ad management system
- conserve resources for system updates
- improve system performance
- reduce processing time of ads display requests
-
03
Solution
replace programming language PHP with Go
-
04
Results:
- $ 21,600 savings on system maintenance per year
- Cut response time from 250 milliseconds to 150 milliseconds
- Now just 3 servers are needed instead of 6 for ads display
Impressions growth from 30 million to 800 million per day
The network is responsible for the effectiveness of the ads displayed by the entertainment content service. We have been working with this network for 11 years.
We developed an ad management and display system. It can be conditionally divided into three large parts:
- a control panel where advertisers and publishers (website owners) operate
- a set of services that are responsible for ads display
- service tools
The system load is constantly growing. When we started, the service had from 20 to 30 million ads per day, a year later they had 100 million, and at the beginning of 2021 an astounding 800 million. To ensure maximum system performance, we regularly update processes, internal solutions, and anything else that can help the network and service earn more from advertising.
Problem: PHP language limitations and expensive server maintenance
Until 2020 the ad management system worked in PHP 5.6. We chose PHP to save on customer expenses. For example, the cost of development and specialist services for PHP is lower than for Java and C++. Additionally, system customization in this language requires less time than the same customization in Java.
Expensive service. For a time, the system handled the given tasks well enough. But as the load grew, problems appeared. The first problem was that service for the customer became more expensive and more complicated.
Insufficient response speed. It takes time to request and process ads, and if it takes too long, the ad network risks reputation and revenue.
Let’s say an ad banner is placed at the top of a page. If the ad management system is slow to respond, the banner may not appear immediately on the website.
And while the system is working on a response, a user can scroll down the page and leave without seeing the ad.
Officially, the ad was shown, so the advertiser has to pay. In fact, the advertiser is simply paying for air, and no one likes that.
Slow response speed affects relationships with partners. If the system takes a long time to respond, sites load slower, and this in turn reduces revenue. Most likely, a client will begin looking for a different ad network.
System maintenance costs also depend on response time. The faster the system processes a request, the faster server resources are freed up to process the next request, so more requests can be processed on the same hardware. For the customer, this means savings on servers and maintenance.
Technical block: system metrics in PHP
6 servers, excluding storage.
Peak consumption:
- CPU — at the level of 75%
- RAM — 90%
Response time: 250 ms
Configuration of servers: Dual Hexa-core Xeon E5−2620 v3 2.4 GHz 48.00 GB RAM 2x SSD 256 GB.
Solution: switch from PHP to Go
Over 11 years working with the client, we optimized the system to the maximum possible. We had reached a point where the only way to increase performance was to change the language. We considered both Go and Java, as we have experience and knowledge in working with both languages, but in the end we chose Go. In our opinion, this is the more cost-effective choice and easier for the customer to adapt to.
In addition to changing the platform, we implemented a few more changes:
- abandoned a monolithic application and switched to a microservice architecture
- implemented a full-fledged CI/CD
- provided extensive coverage of unit code and integration tests
- created scripts for automatic deployment of updates
Altogether, these changes allow the team to scale each component individually, deploy updates faster, and conserve server resources. All of this also helps to significantly reduce customer expenses and increase the effectiveness of advertising.
When developing and launching the new system, we did not turn off the existing servers and system, so the customer avoided downtime: everything worked as usual. To achieve this, the team sequentially moved each of the functions into separate services and then launched them individually.
Process: how components were separated from a monolithic structure and output to separate services
-
Analysis
- Code state
- Dependencies
-
PlanHow to separate the functions of a component into a separate service
-
Development
-
Testing
- Automated
- Loaded
-
Optimizing bottlenecks
-
Development and launch
- Scripts for automatic service deployment
- Service implementation and performance monitoring
Result: Reduced customer costs and accelerated system response
Before updating the system, we agreed with the customer regarding a goal of two tasks: reducing the cost of system maintenance, and achieving better performance for auction purchases. This is what we achieved:
| $ 21,600 | 3 servers | 150 ms |
| …savings on system maintenance per year | …instead of 6 for system maintenance | …response time instead of 250 ms |
In total, the project took 8 months, during which we — as a permanent external team — solved the tasks agreed upon with the customer and completely rebuilt their system.
Technical block of the project
Technologies:
- language: Go 1.13
- service interaction protocol: GRPC
- Service Discovery: Consul
- collection of metrics and monitoring: Prometheus
- CI/CD: Bamboo
- deployment: Ansible
Impact of language change
After we rewrote all server components in Go, the load indicators dropped to the following values:
- CPU — 30%
- RAM — 40%
Additional changes implemented:
- abandoned monolithic application in favor of microservice architecture
- implemented a full-fledged CI/CD
- provided extensive coverage of unit code and integration tests
- developed scripts for automatic deployment of updates
