A small technological marvel occurs on almost every visit to a web page. In the seconds that elapse between the user’s click and the display of the page, an ad auction takes place in which hundreds of bidders gather whatever information they can get on the user, determine which ads are likely to be of interest, place bids, and transmit the winning ad to be placed in the page.
How can all that happen in approximately 100 milliseconds? Let’s explore the timeline and find out what goes on behind the scenes in a modern ad auction. Most of the information I have comes from two companies that handle different stages of the auction: the ad exchange AppNexus and the demand side platform Yashi. Both store critical data in an Aerospike database running on flash to achieve sub-second speeds.
The cast of characters in an ad auction are:
- The user
- The web server being visited
- An exchange, such as AppNexus (a few big sites such as Facebook and Google have their own exchanges)
- A bidder or Demand Side Platform (DSP), such as Yashi
- Third-party sites that collect demographic and behavioral data on users (e.g., what we shop for)
- Advertisers
Routine page loads
Let’s set a bit of context for the magic of ad displays by looking at how much time a web page takes to load.
Steve Souders’s classic High Performance Web Sites (released seven years ago) reported that the top ten U.S. web sites in 2007 required download times that ranged from 1.7 to 22.4 seconds. Things have certainly improved since then — although it’s hard to say how much resulted from faster Internet connections and how much from performance tuning.
Rolling forward to this year, Akamai recently released its quarterly State of the Internet Report. Akamai does Real User Monitoring (RUM) through a product called Aqua Ion it released last year. Average page load time is not very impressive, although of course it varies widely among regions. Typical load times are 3 or 4 seconds for broadband, and 5 to 10 seconds for mobile networks. (I’m making gross generalizations from the table on p. 57 of the report.)
Speeds are going to improve, and no site wants its ad auction to hold up the user. So let’s look at each step in the auction and how long it takes.
Preparing for the ad request
According to Geir Magnusson, Jr., CTO of AppNexus, an ad auction is triggered while the web browser is loading a page. When the user clicks on a link or enters a URL, the server returns a page with ad tags that contain JavaScript to invoke an auction with a particular exchange. The browser then issues a request to an exchange such as AppNexus, or to a supply-side platform (SSP) that then sends it to the exchange.
Andy McConaghie and Larry Nolan of Yashi explained to me that the browser passes the user’s IP address, the user agent string, and available cookies to the exchange (which shares this information ultimately with the DSP). The browser also passes information on the web page being visited and characteristics of the desired ad. These characteristics include whether it’s banner, pop-up, video, etc.; the size of the area where it will be displayed (which gives a good indication where it will appear on the screen); and whether a video will auto-play when loaded.
Each party in the auction (the exchange and DSP) tries to identify users, normally through cookies. Here, of course, is where so many privacy advocates see risks to the user. Indeed, both exchanges and DSPs try to find out as much about our preferences as possible by sharing their knowledge of the user with third-party sites that collect demographic data about us, information on our shopping habits, etc. However, none of these companies really wants to identify who we are — they just want to get us the sports equipment or gadget we like. In short, the exchanges and DSPs are uninterested in personally identifying information such as your name.
It’s getting harder these days to use cookies. For privacy reasons, many modern browsers disable third-party cookies by default. Some correspondents can use fingerprinting instead, hashing together everything known about the browser. The various combined characteristics of each browser (version, plug-ins, etc.) are unusual enough to be useful in uniquely identifying the user. However, AppNexus does not use fingerprinting because of privacy concerns.
Mobile devices also often have no cookies, and native apps don’t use cookies. A correspondent can use the device ID from Android or iOS as an identifier.
Before we move on to the auction itself, it’s worth noting that both exchanges and DSPs screen the ads in advance. Ad campaigns run for a fairly long period of time, such as a month, and are aimed at a particular user demographic or geographic location.
Auditing ads is standard practice across the industry. Magnusson said AppNexus employs “an army” of people to categorize ads so they can serve up bicycle ads to bicycle enthusiasts, solar panel ads to environmentally conscious people, etc. The AppNexus screeners also make sure the ads are not misleading (for instance, the notorious malware that looks like dialog boxes), and that they meet the websites’ standards for appropriateness.
DSPs also need to see ads in advance in order to match an ad accurately to the user. In general, the exchange and DSP store as much data in advance as they can to cut critical milliseconds off of the real-time auction process.
Timeline for an ad auction
Once an exchange receives the request for an ad, a window of 100 milliseconds starts. (Some auctions allow 150 milliseconds, and as we’ll see, some are especially complex.) At this point, events go as follows (see also the summary in Figure 1):
-
User identification (less than 10 milliseconds).
This is where the database plays its first role. A key/value store works quickly for the simple lookup of a cookie or other identifying information. The challenge is to get a response to such a query over hundreds of gigabytes, even terabytes, in a few milliseconds. In addition to matching users with ads, the identifier can be used for frequency capping — for instance, so a user doesn’t see the same ad several times a day.
-
Augment understanding of the user from third-party data (10 milliseconds).
These third parties are the companies that accumulate information about our purchasing habits. The time allowed for them to return data is so short that they often can’t spare time for network transmission, and instead co-locate at the AppNexus server site. In fact, according to Magnusson, the founders of AppNexus created a cloud server before opening their exchange.
-
Get bid request from the DSPs (50 milliseconds).
AppNexus handles 150 bidders (DSPs), sometimes more. Each DSP submits one ad. According to McConaghie and Nolan, a typical process at a DSP such as Yashi is:
-
User identification.
The DSP takes the identifying information on the user to find her in its database, just as the exchange did. In addition to the key/value lookup, Yashi also does geolocation lookups. For each auction they make hundreds of comparisons.
-
Augment understanding of the user from third-party data.
Like the exchange, the DSP passes the information it has about the user to third-party data providers. The data provider sends Yashi new data each hour on users for whom they have new information. Yashi stores the information in its database.
-
Choose the most appealing ad.
The DSP can return only a single ad from all the advertisers it serves. Having collected information on all ads — including, most of all, the price the advertiser is willing to pay — and having determined user characteristics (such as location, demographics, and preferences), the DSP picks the ad most likely to win the bid and sends it back to the exchange.
-
-
Order the ads by bid price.
Because the highest bid is likely to win the auction, the exchange uses this critical piece of information to line up the ads from each bidder.
-
Apply business logic.
The exchange still has to make sure the ad is appropriate for the website and meets its policies.
-
Select the winning ad and transmit it to the browser.
All the preparation and last-minute thinking that went into the auction results in returning the perfect ad that will, of course, send you or me into a state of joy where we have no recourse but to make a purchase.
At peak activity times, this whole process can be done by AppNexus three million times per second.
Figure 1. Ad auction timeline
Variations
So that’s an ad auction, in brief. Naturally, there are many variations, not all of which I can cover here.
For instance, when a site runs its own exchange (Facebook being a well-known example), it contracts with other exchanges such as AppNexus. So, the Facebook exchange performs all the steps in this document, then sends requests to AppNexus to repeat the steps. The other difference in this variation is that AppNexus does not send Facebook the actual content of the ad, but just an identifier. Should the AppNexus ad win the Facebook auction, Facebook passes back the identifier to get the ad.
When AppNexus works with other exchanges, it uses its database in a two-step process: first to map the ID sent by the other exchange to an AppNexus ID, and then to map the AppNexus ID to information AppNexus maintains about the user.
Moreover, AppNexus runs its own DSP. The founders did not plan on doing this, but found that other sites had trouble creating DSPs — the tasks required under such time-limited circumstances are difficult and very demanding. AppNexus serves many advertisers and treats its own DSP no differently from any other, showing no bias.
This demonstrates that, although many websites struggle to balance the need to engage users with the cost of longer load times, new database technologies and high velocity architectures make it possible to do this and more.
This post is part of a collaboration between Aerospike and O’Reilly Media. Read our statement of editorial independence.