fe_81

Hi, I am building an own affiliate network and would like to hear your opinions about a few security related things. Does anyone know of any guidelines how to place cookies and track sales when building an affiliate network. I don't need clean links or anything, just a link to my own server with a specific get parameter, which redirects to the store (the e-commerce website). My idea is that I would place the cookie from my own domain on the redirecting-page and then let the store make a call to my server when a sale is made. This would be done by having the stores add an <img>-tag containing parameters in the url. In the php-script for this url, I check that the cookie is set. Will this work on all browsers? What about 3rd party cookie settings? Do I have to require that the stores sign the parameters in the img-tag url with the help of a secret key? It seems as if some big affiliate networks don't do this. If I don't sign, how can I protect against just going through a link (obtaining the cookie) and then making a call to the url of the img-tag? What about cookie stuffing? I can't really find any info about how to protect against it? Is there any way? Thank you for your help -F

Thank you for your answer! Does this have a risk that some real strings get casted to boolean or int? For example if we have a field username and someone writes "true" or "five" or something like that? One option that I have been thinking about is to write a short script that can be run from the commandline, which would generate classes from the database scheema that take care of the typecasting for each table. Do you know if there is something like this already for Zend Framework? -F

The problem is that the variables may come as either strings or numbers. Especially booleans. Check this json string for example: { "booleanFromDB":"true", "booleanFromPHP":true, "integerFromDB":"123", "integerFromPHP":123 } I would like to avoid all such code on the js as: if (someVariable == "true" || someVariable == true) { do something ... } I would like to find a somewhat generic way to send data from MySQL->PHP->JS and back (JS->PHP->MySQL) without having to do lot's of typecasting or checks. Thanks, F

Has anyone else this problem ever or something similar when getting data from the database and JSON-encoding it for some ajax requests. Or is it just something you have to live with (either you have to typecast the data in the php-code or you have to do it javascript (possibly with some additional checks).

Hi, I use PDO MySQL with Zend Framework database adapters. When I select data from the database, it is always returned as strings. From what I have understood, this is the normal behavior. But this creates a lot of unnecessary typecasting code in various places of the application. One example is when data is encoded into JSON and used in javascripts. Assuming that everything are strings won't work since some variables might have types if they were not pulled from the database. Is it possible to have some kind of automatic type hinting for database adapters so that all data comes in the right types from the database, or are there any other good solutions to this problem? Thank you for your help! -F

Thank you for your answer. So all MySQLs partitioning features only has to do with partitioning on one server, not splitting the database to many servers? I know how replication works and we are going to use replicas for searches and similar that we cannot cache with Memcached. We are looking for a solution where we can scale horizontally by adding more servers and not by buying better hardware. We need to come up with a relatively good application design that allows for both flexibility and scalability. -F

I forgot to say that by "partitioning" I mean vertical partitioning, i.e. splitting the database into several databases so that some tables are in one database one server and other tables in other databases, each database on an own server. I saw that MySQL has some kind of partitioning support (horizontal, i.e. splitting one table so that some rows are in one DB and other rows in other DBs), but nowere did it become clear whether you can split so that these DBs are on different servers. Does anyone know about this? Can someone give some advice? Thanks!

Hi, I have a question about what is the most suitable database and design principles when you want to build a scalable system, but you also want to keep the code flexible to changes. I know this depends a lot on the case, so I’m going to go into more details. Let’s say we have product database, with lots of products. Products are related to several other things, like stores selling them, manufacturer who builds them, customers who bought them and their details, etc. All these will have some own attributes. Now, we will have a very complex admin panel, where you can search products based on keywords and other attributes and sort them in lists based on different rules, such as which has sold most (number of customers), etc. This would be trivial to implement if we would not need to care about scalability. Most searches and sorting of lists can be easily implemented with a few joins and sorting by COUNT(…) or whatever we want. But the problem is that this won’t scale, or will it? We probably need to partition the database, due to high traffic, so we can’t really use joins. In worst case we need to split some table into different shards. This will require really complex php-code, and some of the features may not even be possible to implement. Making changes to the admin-panel and adding new features will take a lot of time. Are there any good solutions how to do these complex queries with joins and sorting by count when you have partitioned the database so that some tables are on one server and others on other servers. And what about shards? We are using Zend Framework, which comes with TableModel. Is it a good idea to use table model in our situation, or is it better to just write our queries by hand? Would some ORM framework like Doctrine work (not familiar with it). Can we overcome these issues by using some noSQL database instead of MySQL, like MongoDB, that takes care of all the scaling? Does anyone know if complex queries, with joins and sorting by count, work well in MongoDB? Different websites seem to have different opinions. Some say MongoDB is not good for complex queries others say it is. Can someone give some advice or point me in the right direction where I can find info about this? Thanks, -F

Thank you for your answer. Is memory going to be the bottleneck on the web servers even if we do not run Memcached on the same servers? What is it that needs memory? PHP and Apache does not use themselves that much memory, even if we use APC for opt-code caching. Is it so that the need for memory grows considerably when there is heavy traffic? Running Memcached on all web servers is hard to configure, since we want to have an easy way to add new web servers or remove existing ones (possibly use autoscaling at some point). This would require all web servers to be aware of each other. It is obvious that a micro instance cannot be as good as a larger one, but how much worse are they? They are 4 times as cheap, so if less than 4 micro instances are better than one small (default), then the micro instances are probably better. Or does this make any sense at all? -F

Hi, Could someone please give us some advice on which Amazon instances are suitable for which purpose? I know that this may depend on the type of application you are building, but are probably some general facts. We are building a website, and plan to start with the following setup: - 1 Elastic Load Balancer - 2 EC2 Micro Instance (EBS storage only), used for web servers - 1 EC2 Micro Instance (EBS storage only), used as a caching server (with Memcached) - 1 EC2 Micro Instance (EBS storage only), used for a mail server - 1 RDS Small DB Instance, used for database server What do you think about this setup? We will monitor the system and add more web servers when needed. The cache server and the database server will first be upgraded to better, before adding more of those. Our biggest question is that are micro instances enough for web servers? I have heard that the bottleneck in web servers is rarely CPU and we do not do any complex calculations at the web servers or use https. We use PHP with Zend Framework, which does not require that much memory either. So the question is should we go with several micro instances or fewer larger ones? What are people normally using? We appreciate all help we can get. Thanks!

Hi, We are building an ad network and we are wondering how it can be designed to scale. I cannot go into too much detail about the service here, but let’s pretend it is a simple ad network where there are 2 types of users: 1. Product Owners, who pay a certain amount of money in advance, and each time their banner is clicked/hovered/etc. some money is withdrawn. 2. Blog Owners, who themselves choose specific banners for their page. These banners are shown until the Product Owner run out of money. They are not automatically changed. Some technical background, in case it matters: Our backend is built with PHP (Zend Framework) and MySQL. We plan to use MemCached for caching and the service will be hosted on Amazon. We have a relatively good idea how to scale most parts of our service, but the ”money system” (in lack of a better word) seems harder. The problem is that we need to have up-to-date data about how much money a customer has left, and use this each time we display its ads (we don’t want to show ads for customers who have run out of money). We also need to update the amount of money often (after clicks, hovers and other events). Are there any “silver bullets” to solve this problem easily, that you know of? The issue of whether to display ads can probably be tackled with the help of caches (or database replicas). The data does not need to be fresh, since it is probably acceptable to show a few ads, even though the customer does not have any money left. This already happens in a situation where a customer barely has enough money left for one click, and a page displaying the ad is loaded by several end users at the same time. Now the ad is shown for each end user, even though there is only money for one potential click. Now if several end users click, some clicks will go uncharged. It is inevitable that this will happen sooner or later; therefore, we believe it is acceptable to also use cached data (or data from replicas) for this at least to some extent. It will probably just result in a few more uncharged clicks/hovers/etc. What do you think? The issue of updating this info is harder, since the master database needs to be updated. We have to do database writes very often (especially from hovers). We have a few different ideas how to solve this: - We could use “shards”. Different accounts would be stored in databases on different servers. The downside of this is that shards generally increase code complexity and we do not have “transactional integrity” when transferring money from one customer to the other if they are located on different shards. There is also a risk that one customer grows too large. - Another idea is to just log everything that has been clicked/hovered/etc on each web server and process (or aggregate) it locally at the web server and then send the aggregated data to a common server, where all data is aggregated and all accounts are updated. - Report events with a certain probability and statistically determine how many events really happened. - Some combination of these Are there any design patterns, tools or common methods how these issues are usually solved? We also wonder what is the best way to store log data (from clicks, hovers, etc), so that it can later be processed. Databases are slow and I don’t know if writing to a file is that good either, unless several events can be buffered and written at the same time. I heard that Facebook uses Scribe for real time aggregation of log data. Could it be used to solve any of our problems? Thank you for your help! -F

Hi, We are building a website with social networking features (people can sign up and invite others, etc.). We are using Zend Framework for it and coding most of it from scratch. We would need the following features, that I think can be easily made with some existing solutions. - Each user can have an own blog on their profile page - A normal discussion forum, where users who are logged into our service can ask questions. There should not be a separate login for the forum. - A shop where anyone can buy products. However, there should be a way to programmatically add/remove products from the shop (i.e. not from an admin panel, but from the rest of our application) Does anyone know of any open source or free components that can be used for this and are easy to integrate? Thanks! -F