Companies waste tens or even hundreds of thousands of dollars as a result of doing their own development. This has to do with the particulars of the advertising business.
Online advertising management is a resource-hungry process: you need to find a relevant audience and site, and deal with traffic and setting up content display. It’s impossible to put together an audience profile right away, it’s impossible to compose other elements — it’s always an enumeration of hypotheses, models, and algorithms. This process takes time and costs money. And if there’s no team that has already worked with these kinds of mathematical systems then there are delays, or you don’t achieve the desired results.
This article discusses how companies deal with the problem of developing and analyzing hypotheses and why they decide to outsource development to OrbitSoft.
Briefly About Projects
- advertising agency
- content provider
- operator of advertising on the cryptocurrency market
- test hypotheses and algorithms before investing money in them
- automate the analysis of advertising according to the specified criteria in order to purchase on more favorable terms
- acquire an algorithm for finding a relevant audience
Budget protection. Advertising agency refused to invest in an algorithm that didn’t work
Process automation. Content provider automated data analysis for advertising purchases
Ready algorithm. The company received an algorithm for finding a relevant audience
Advertising Agency Refused to Invest in an Algorithm That Didn’t Work
An agency from Spain makes money by selling ads. The company buys ads from platforms with the pay-per-impression model (CPM) and sells with the pay-per-action model (CPA) — when only targeted user actions are paid for, such as downloading, subscribing or registering.
The problem is there are no resources for hypothesis testing — neither a team with the necessary experience nor a ready-made software solution. One of the ways to increase income is to learn how to show ads to audience members that are more likely to perform the targeted action. This is helped by algorithms that set the criteria for displaying content.
Agency mathematicians have hypothesized a new algorithm. As planned, it should optimize traffic for smartphones and tablets so that users take more targeted actions.
A single hypothesis is obviously not enough to say with certainty whether the algorithm will work or not. Everything needs to be tested. Otherwise there is a risk of investing in a solution that doesn’t change anything.
Companies can test hypotheses using their own resources. This requires programmers, testers, and analysts. Another option is to hire an external team that specializes in checking algorithms.
The agency doesn’t have a hypothesis testing team, so it would have to be hired. In this case, testing would take about two years: first, you find specialists, immerse them in the process and features of the algorithms, develop a program for testing — and only then do you conduct tests. This decision is also affected by the budget: according to the agency, it would have to spend $ 200,000 on salaries.
Solution: Delegate Hypothesis Testing. The agency turned to OrbitSoft to test the hypothesis for the new algorithm. We already have a solution for such problems, so we went to work immediately.
Based on our Demand-Side-Platform Product (DSP), we developed technology that can test a mathematical model. It took us 1.5 months and $ 20,000 from the customer to develop it.
Result: With the help of ready-made technology, the customer tested the model on real users, in real time. It took 4 months to verify it, and during this time the company realized that the algorithm was economically unprofitable — they could not achieve the planned conversion with it.
Despite the fact that the algorithm didn’t work, the company still got their result. They discovered that their hypothesis didn’t help them target more accurately, so there was no point in investing in this algorithm. It would be better to develop and test another one.
The company also saved money. If they had tested the hypothesis using their own resources, they would have spent up to two years and up to $ 200,000. This way the project took less than six months and cost one tenth the price.
Comparison: Cost and Time for Algorithm Testing
Technical Features of the Project: Buying Traffic
CPM calculation method. Historical CPM (pay-per-view) data was used
- CPMA — average value from the start of the advertising campaign to the current moment with coefficient k1 CPMB — average value of CPM for the last hour with coefficient k2
- CPMC - average CPM value for the last hour with coefficient k3
Dynamic CPM = (k1 * CPMA + k2 * CPMB + k3 * CPMC) * ((100 — MARGIN) / 100)
The limits of the values of the coefficients:
- k1 = 0.1 … 0.25
- k2 = 0.3 … 0.45
- k3 = 0.5 … 0.75
Price Calculation for a Certain Percentage of Traffic Purchases
- 90% - at a fixed CPM price
- 10% for each advertising platform — at a dynamically determined CPM price
- 60% - at a fixed CPM price
- 40% of traffic for each advertising platform — at a dynamically determined CPM price
If it wasn’t possible to buy out at the dynamic CPM price on a certain site, then every 10 minutes the price increased to the threshold value. When the threshold CPM value was exceeded, the site was excluded from traffic purchases.
- 80% of purchases — at a dynamically determined CPM price
- 20% - at a fixed CPM price.
The dynamic CPM price is calculated separately for each site. During the process of buying traffic, the CPM threshold value is monitored: if the price exceeds the limit, the site is excluded from traffic purchases.
Content Provider Automated Data Analysis for Advertising Purchases
This company creates content for mobile operators, such as videos, GIFs, games, and so on. These partners are called content providers. End users buy their content through cellular operators.
The main way to attract customers is by displaying ads that convert users to customers. The provider receives a commission for each targeted action.
Paid content is games, videos, memes, and anything else that might grab users' attention. The more interesting it is and the more it meets the interests of users, the more willing they are to pay for it.
Problem: Ads are Analyzed Manually. The provider’s income depends on the ratio: the cost of advertising and the number of subscriptions that follow. Conventionally, the lower the price and the greater the number of targeted actions, the higher the earnings.
First, the customer contacted CPA networks, which buy traffic for impressions and sell for actions. The problem is that they optimized their ads mechanically: they bought traffic, manually analyzed statistics, and manually bought what was more efficient. It took a lot of time, and it was impossible to avoid mistakes.
Solution: Automate Ad Scoring. The customer wanted a data analysis tool that would help them earn more. Such a tool is called a model.
General idea: the model examines the data and, based on this, determines which ad is more profitable to buy, taking into account the price and conversion. Moreover, it works automatically, without human intervention.
The company turned to OrbitSoft to develop the model. We used our product and wrote a predictor for it. With the predictor’s help, the program understands when and what to purchase for relevant traffic.
The predictor studies the behavior and reactions of users to the ads they see. For example, what sites the user visits, at what time, what he or she looks at more often, for how long, what he or she reacts to, and when he or she closes them. For a more accurate analysis, the predictor processes information from 54 traffic sources — SSP and Ad Exchange.
Process Diagram for Automated Ad Buying
The predictor analyzes the dataThe predictor analyzes historical data about the user, his or her behavior, and reactions to advertisements
A prediction model is createdThis is used to train and optimize the analysis process
The program makes a decision regarding purchase of advertising
- The program transfers data to the model
- It gets an estimate of this data
- Based on this, it makes a decision regarding the purchase of traffic
Results: The provider automated the analysis of advertising and accelerated the process of ad buying, which affected the performance indicators for all work with clients' content (mobile operators).
OrbitSoft Predictor Helped Increase Conversions, and other Advertising Metrics Work with Content
Results for the example of one mobile operator over three years
236 thousand subscribersConnected to paid content
10.6 million rebillsNumber of daily payments for paid content
182 million rublesTurnover from displaying advertising content
20 thousand dollarsWhat the operator spent on servers and buying traffic
Subscribe to our newsletter
Technical Features of the Project: Stages of Work
1. To choose a platform for analyzing and building the predictive model. We had to decide between Vowpal Wabbit and TensorFlow. To make a decision, we:
- conducted experiments on historical data, processing information about past impressions and clicks
- we compared which of the tools better predicts events (clicks) with our data
Based on the results of the experiments, we chose TensorFlow. The platform allows us to build more complex models, as well as showing better results with our data.
2. Select Data for Analysis. Our solution was to use:
- accumulated historical data on impressions and user clicks
- parameters that can be obtained from an advertisement request
- accumulated data about the user
Among them are:
- Remote IP address
- Exchange ID
- Publisher ID
- Creative Size
- Placement ID
- Advertiser ID
- Campaign ID
- Creative ID
- Publisher Domain/Mobile Application Bundle
- Vertical Category ID (IAB Category)
- Ad Position
- Visitor Language
- Visitor Country
- Visitor Region
- Visitor City
- Visitor ZIP/Postal Code
- Visitor Lat/Lon
- Visitor ISP
- Visitor Gender
- Visitor Age
- Visitor Device Type
- Visitor Device Make
- Visitor Device Model
- Visitor Browser
- Visitor Browser Version
- Visitor OS
- Visitor OS Version
- Visitor Connection Type
- Day of week
- Time Zone
3. Choose an Algorithm. We considered several methods for building the model:
- Simple logistic regression
- FiBiNET (Feature Importance and Bilinear feature Interaction NETwork)
- FLEN (Field-Leveraged Embedding Network)
We chose FLEN because the model showed the best results with our data.
4. Select the Criteria for the Quality of the Model.
The data we use is not balanced, so two criteria were used to evaluate the model, which show:
Precision — the proportion of objects that the classifier identified as positive and which are so
Recall — the proportion of objects of a positive class from all objects of a positive class found by the algorithm
5. Integrate Solutions onto our Advertising Platform. Using the Go language, we developed a service for the client’s goal. Here’s what it does:
- downloads the model trained on our ad platform
- issues forecasts
- integrates with other platform components that are involved in the decision to buy traffic and display ads
Model Performance Indicators. When training the model, we used data collected for several weeks — about 1.2 billion events. The model makes decisions for 60 million events every day.
Company Got an Algorithm for Finding a Relevant Audience
This company worked with cryptocurrency and investments, and this niche was profitable. As another source of revenue, the owners chose to place advertisements for clients on cryptocurrency platforms.
Problem: It was not clear how to find a relevant audience. Customers were willing to pay for advertising, but the company didn’t understand how to build the process, i.e. where to place the advertising, under what conditions, how often to display it, and so on.
Solution: create an algorithm for finding an audience. The company didn’t waste resources developing the technology for the analysis and selection of the audience themselves, and immediately turned to professional help.
To solve the problem, we used our own product — OrbitSoft DSP, and developed an algorithm specifically applicable to the client’s situation. Thanks to this, the company received this in a short amount of time — just 4 months.
The algorithm works with 10 traffic sources, for example Google AdX, Smaato, MoPub, AOL, and OpenX. It analyzes the incoming audience and determines with what probability the user fits the desired segment. The more user profiles there are, the higher the accuracy of decision making in subsequent analyses.
The Algorithm Work Scheme
We developed an algorithm similar to the one that works with Lookalike Audiences on Facebook
Gathered the audience
Created an audience segment that is interested in cryptocurrency
Segment was uploaded to OrbitSoft DSP
We attached a machine learning algorithm to the platform, which selects the audience according to key parameters
Result: Thanks to the algorithm, the company gained a relevant audience base and increased its revenue. It began to buy traffic for 10 thousand dollars — this is the daily spend of advertisers. The larger the amount, the more commission the company receives. The result allowed the company to enter the top 10 advertising agencies in the field of cryptocurrencies and fintech in the countries of the former CIS in six months.
Technical Features of the Project: Work Stages
1. To gather an audience. To do this, we compiled a database of sites that write about cryptocurrency. It included, for example:
- Blogs about cryptocurrency
- News portals
- Rating systems
- Analysis applications
- Crypto wallets
All sites were analyzed for:
- purchases of audience segments that are interested in finance and new technologies
- evaluation of matching for sites using synchronization pixels (Cookie Matching)
2. To launch ads. The collected data was used for broadcast advertising campaigns. The work was divided into two stages:
- A/B tests of advertisements, which helped to gather the target audience. The higher the interest in cryptocurrency, the higher the rating in the segment
- Narrowly targeted advertising campaigns for users with high ratings i.e. those most interested in the topic of cryptocurrencies. If interest persisted, the rating increased even more.
3. To find a similar audience. For the search, ready-made purchased segments were used. Our task was to find a group of users in these segments that were similar in terms of target criteria. For this we:
- compared users of a certain parameter
- monitored thresholds — how well the audience meets the criteria for these parameters
For the assessment we used:
- Gender and age
- Income level
- User interests
- Short-term intentions of users
- Geographic location
Thresholds values within which users were considered similar — 0.68 to 0.85.
OrbitSoft Is a Pragmatist That Doesn’t Fall in Love with Ideas
We test ideas every day — both our own and those of our clients. In our experience, no more than one percent of them end up working. And that’s okay. It is impossible to come up with something that will definitely work all the time.
It’s good when there are a lot of ideas. But ideas are just hypotheses to be tested. If they turn out to be inefficient, you need to abandon them — and go on to the next one. If they work, great — it’s time to develop them. The important thing is not to engage in a fool’s errand.
Many businesses fall in love with their ideas, and it becomes difficult to abandon them: they want to continue to tinker with them, think them over, or launch them in the hope that somehow they will pan out. Therefore, my advice is to involve a partner in hypothesis testing. These are not their hypotheses, so they don’t hold on to them with every last bit of their strength. Instead they methodically — one by one — test and weed out all those that don’t work.
OrbitSoft is a pragmatic company that doesn’t fall in love with ideas. We are very sober-minded and don’t allow businesses to waste time on testing dead-end hypotheses. We help you to move fast and efficiently.