Designing a scalable ad network (the money system)

[fe_81]

Hi,

 

We are building an ad network and we are wondering how it can be designed to scale. I cannot go into too much detail about the service here, but let’s pretend it is a simple ad network where there are 2 types of users:

1. Product Owners, who pay a certain amount of money in advance, and each time their banner is clicked/hovered/etc. some money is withdrawn.

2. Blog Owners, who themselves choose specific banners for their page. These banners are shown until the Product Owner run out of money. They are not automatically changed.

 

Some technical background, in case it matters: Our backend is built with PHP (Zend Framework) and MySQL. We plan to use MemCached for caching and the service will be hosted on Amazon.

 

We have a relatively good idea how to scale most parts of our service, but the ”money system” (in lack of a better word) seems harder. The problem is that we need to have up-to-date data about how much money a customer has left, and use this each time we display its ads (we don’t want to show ads for customers who have run out of money). We also need to update the amount of money often (after clicks, hovers and other events). Are there any “silver bullets” to solve this problem easily, that you know of?

 

The issue of whether to display ads can probably be tackled with the help of caches (or database replicas). The data does not need to be fresh, since it is probably acceptable to show a few ads, even though the customer does not have any money left. This already happens in a situation where a customer barely has enough money left for one click, and a page displaying the ad is loaded by several end users at the same time. Now the ad is shown for each end user, even though there is only money for one potential click. Now if several end users click, some clicks will go uncharged. It is inevitable that this will happen sooner or later; therefore, we believe it is acceptable to also use cached data (or data from replicas) for this at least to some extent. It will probably just result in a few more uncharged clicks/hovers/etc. What do you think?

 

The issue of updating this info is harder, since the master database needs to be updated. We have to do database writes very often (especially from hovers). We have a few different ideas how to solve this:

- We could use “shards”. Different accounts would be stored in databases on different servers. The downside of this is that shards generally increase code complexity and we do not have “transactional integrity” when transferring money from one customer to the other if they are located on different shards. There is also a risk that one customer grows too large.

- Another idea is to just log everything that has been clicked/hovered/etc on each web server and process (or aggregate) it locally at the web server and then send the aggregated data to a common server, where all data is aggregated and all accounts are updated.

- Report events with a certain probability and statistically determine how many events really happened.

- Some combination of these

 

Are there any design patterns, tools or common methods how these issues are usually solved?

 

We also wonder what is the best way to store log data (from clicks, hovers, etc), so that it can later be processed. Databases are slow and I don’t know if writing to a file is that good either, unless several events can be buffered and written at the same time. I heard that Facebook uses Scribe for real time aggregation of log data. Could it be used to solve any of our problems?

 

Thank you for your help!

 

-F