How can I cache more than just simple variables in the environment?

[LLLLLLL]

What's the best way to cache things on the server? I've found some SetEnv things but this all seems to be a single key/value.

 

I'd like to cache an array of things that I can access later. Right now it's cached in $_SESSION which is fine, but still there's overhead on every first-time visitor to a site. Is there a way to add something to $_SERVER or elsewhere that ALL visitors to a website will ultimately be able to access?

 

I guess I want something like web.config in C#, where the first time a page is visited, the "app starts", as it were, and those things are now in cache on the server.

 

What's the best way to do this?

[kicken]

what kind of information are you looking to cache?

[LLLLLLL]

A bunch of application-specific stuff. The first thing I'd like to cache are all the text strings that are used for the website's user-facing text. Currently these are all read from the DB and stored in SESSION, but I'd like to store it somewhere else so it's not read as often.

[ginerjm]

I really don't think you want to "store ... in the environment".   Between the db and the SESSION array, there is no need to do anything else.  By environment I think you mean something like an .ini file.  But why?  Unless you have the world's worst db connection the speed of a db query and the transfer of any application specific settings to an array in the SESSION array is immediate.  A simple check for the array right after your session_start call will tell you if you have to do the query or not. 

 

In case you are not familiary here's an example:

 

$_SESSION['appl_vars']['name'] = "My Application Name";
$_SESSION['appl_vars']['setting1'] = 'my setting #1';
$_SESSION['appl_vars']['setting2'] = 'my setting #2';
etc.
etc.

 

In each of your scripts you would have this:

 

session_start();
if (!isset($_SESSION['appl_vars']))
{
     include($my_php_path."GetApplData.php");
     GetApplData();
}

[LLLLLLL]

I think you're misunderstanding my intention. When visitor A comes to the site, the session for that visitor loads up all the strings from the DB and stores them. No more DB calls for visitor A. When visitor B comes to the site, another DB call and storage in session. Visitor C, so on and so forth.

 

But there are thousands of strings, and the load time isn't terribly slow but it could be faster. Why can't I...

1. When visitor A comes, load the stuff from the DB and then store in $_SERVER or whatever
2. Visitors B and C will read from that same cache

It doesn't seem necessary to store things per-session if there's something more global that would cut down that time. But where is that storage?

[ginerjm]

Thousands of strings?  And how do you make sure that they are current?  What happens when visitor A arrives and the current set of the strings is read and stored.  Every body after that sees the same data.  But what about when you must alter some strings - how do the users see that updated info without you destroying the current copy of the settings?

 

Really?  "Thousands of strings"?  And you don't have a paid consultant working on this design for you?

[kicken]

For a shared memory cache you would typically use something like Memcached or APCu. You'd simply store your string as a serialized array or in json format.

 

Another alternative is to simply store the strings in a PHP file however and include it when necessary and store the data into a variable. For example:

In the strings file:

<?php return array(
  'string1' => 'Something'
  , 'string2' => 'Something else'
  //, ....
);

To use it:

$strings = include('strings.php');
echo $strings['string1'];

You could move the strings permanently from the DB to the file and manage them there, or you could generate the file on demand if it doesn't exist or needs updated.

 

Before any of this though, I'd wonder why you have these thousands of strings and why you feel it's necessary to load them all to begin with. Surely you'd only need a small subset of the strings on any given page and could simple query for those strings.

[mogosselin]

From what you said, something like 'web.config' equivalent would simply be a file that your PHP code would read. 

 

There are no default/native application variables in PHP due to it's nature (how it runs requests, start its process, etc.). It's not like C# or Java where you can have application variables.

 

So, you could setup an ini file and read it with parse_ini_file

 

If it's a matter of performance, you'll need to setup distributed cache server like Memcached or Reddis, for example. But this is another ball game

[ginerjm]

But when you start reading an ini file, you may as well be querying a db.

[gizmola]

I'll dog pile and 2nd Kicken's answer.

 

When you look into "caching" what that really means is "memory caching".  Memory caching is of course many times faster than either disk or db access in most cases.  I want to point out that by default php sessions are stored in small files on the server.  File IO is often pretty fast, especially with a system that has plenty of memory given over to disk buffering, but it's often not as fast nor as reliable as memory.  And as you already opined, if these are truly shared variables its wasteful to be replicating that data in sessions anyways.

 

Of the memory caching options, there are 2 basic varieties -- "per server" and "shared".  If you have a single server, you can setup any of the shared options and run them on the same server, and you essentially have a "per server" setup, but typically the "shared" caches are used in server clusters where one or more "server" caches service 1-n front end servers (Webserver +php) behind a load balancer.

 

The per-server cache that many people already have available is APC.  Primarily it's used for opcode caching, but it also can be used explicitly as a data cache.  Under *nix servers, APC grabs "shared memory" from the OS, so it's basically dedicated memory that is reserved for use by the system.  As it is essentially bound to php (which is often bound to the web server, as in a mod_apache setup) its not sharable across servers.  That might be fine, and you can kill 2 birds with one stone by reading all your configuration data and strings through the cache.  Once the cache is warmed, this data should come right out of memory as you would like it to.  Depending on the size of your code base, you might need to tweak the settings to insure you have enough available memory to keep your dataset in memory.  

 

Otherwise, one of the benefits is it's lack of complexity and extreme access speed -- no networking involved or getting in the way.

 

Alternatively, the most used server process caches are Memcache and Redis.  Memcache of course was made famous by Facebook who utilized it heavily in their early days.  Memcache is basically as simple as APC in most ways, but was designed for large datasets (often used to front relational database result sets).    While simple, it has significant gotchas you have to understand, but has many large installations.   More overhead than APC, but for name/value caching, it's been the goto product for a lot of the internet's most famous high traffic sites.

 

Redis can be used like Memcache, as a simple "key" (name/value) store, but it has a lot of other features that appeal to certain people.  First and foremost, it can be storage backed -- where you can utilize its ability 

 

Redis is similar and can be used for simple name/value storage, but it has some tricks that the other caches don't.  It has several options for persisting data, which can alleviate the need for "cache warming" on system startup, and programmatically gives you all sorts of features that might be useful for queueing, like pub/sub, "lists" which unlike memcache or APC can be manipulated (the value is not just a black box, but structured data you can sort, push pop and search)  amongst other uses.

 

As the saying goes, Redis is very fast as an in-memory cache, but it's not as fast as memcache or APC, and it's more complex and requires a larger footprint.

 

Hope this helps elaborate on the major players I've used in the php application space, and gives you a leg up on your research.  From the sound of it, any of the 3 will work for you, so it's just a matter of considering your use case, and doing some testing.

[gizmola]

Quick update -- recently I became aware that with php5.5 the opcode cache feature of APC is no longer relevant as the zend opcode cache made it into the 5.5 base.

 

If you still want to use the cache feature of APC you need to instead install APCU.  I haven't played with this yet, but I assume it's simple enough that api and features are probably still relevant to what I've already discussed in the previous post.

[mogosselin]

@ginerjm : If it's a standard small site/1 server setup, you're right, and that's what you assumed the OP has probably.

But, if you have more 'front servers' than 'DB servers', you could want to unload the DB. It all depends on your setup.  I doubt that's what the OP has, but you never know

 

@gizmola : great post!

[LLLLLLL]

Thousands of strings?  And how do you make sure that they are current?  What happens when visitor A arrives and the current set of the strings is read and stored.  Every body after that sees the same data.  But what about when you must alter some strings - how do the users see that updated info without you destroying the current copy of the settings?

 

Really?  "Thousands of strings"?  And you don't have a paid consultant working on this design for you?

 

I should clarify that it's hundreds of strings, but with multiple languages. So there are like 600 identifiers, for example "General.Cancel" represents the text "Cancel" in English, and whatever its counterpart in Russian, French, German, Chinese, etc. So it's thousands once all the languages are considered.

 

I'm not sure why you're asking about altering strings; you just update the cache as you would with any application, simple stuff. Not sure why I'd need to pay someone to work on something here. This design has been tried and tested in global POS applications for years with no issues.

 

I guess the bottom line is that I'll keep loading these and storing them in the session. It's only 600 rows. I was just hoping to load them once.

[CroNiX]

Oh my. I know you marked it as solved but this is really not the way to do it and is very inefficient and resource intensive. There is no reason to load all languages when only one will be used at a time. There is no reason to store them over and over again in session (in multiple languages) for each user.

 

Personally, I'd do something like create a directory to hold the languages.

/languages

 

Then for each language, have that be the filename.

/languages/EN.php

/languages/DE.php

/languages/ES.php

 

Each file would be an associative array, like for EN.php:

<?php
$lang['welcome'] = 'Hello there!';
$lang['goodbye'] = 'Thanks for visiting!!';

The other files would look the same and have the same keys, but just the translation would be different.

 

In your back end (I assume this is where you are creating the languages to store in the db), whenever you save a language/translation, have it create the above files.

 

Then when you determine the users language, just include the specific language file.

$lang = 'EN'; //This users language is English
include 'languages/' . $lang . '.php';  //load whatever lang file the user needs

echo $lang['welcome'];

I believe this would accomplish what you are after as it would eliminate querying the database (except when creating/editing a language) and they would be 'cached' for all intents and purposes. It would be faster and more efficient as you would only load the language you need and you wouldn't need to use session to store them at all.

Edited August 19, 2014 by CroNiX

[LLLLLLL]

A user can change from language to language at any point.

 

Sorry, but I've built an enterprise-level system with 30,000 stores on one system; it uses this model. It works just fine.

Edited August 19, 2014 by timneu22

[CroNiX]

Changing languages wouldn't be a problem.  Just load a different lang file.  If it works so well, why did you have the question to begin with? Sounds like you just don't want to get your hands dirty and improve performance. Lots of things "work", but can always be improved upon and made more efficient.

[LLLLLLL]

This post was not about my specific string caching. When asked what's being cached, I replied A bunch of application-specific stuff. There are all sorts of things that are loaded the first time they are needed. Display Preferences, Checkout Preferences, Shipping Preferences, etc. There's little performance problem with the strings (under .4 seconds, it turns out) and I don't know why you're trying to dissect an application that you know nothing about, when his wasn't my question in the first place.

 

Basically, I'm trying to reduce some DB calls (with what I mentioned, there are four calls the first time, if all those things are needed on a page) by finding a caching layer.

 

My question was about caching arrays of things in $_SERVER or something that's not $_SESSION. "Don't want to get my hands dirty"? Thanks for the insult.

Edited August 19, 2014 by timneu22

[kicken]

When asked what's being cached, I replied A bunch of application-specific stuff.

What kind of caching is appropriate is dependent on the nature of what that stuff is. Language translation strings for example do not change from user to user, so caching them in the $_SESSION array is not ideal. Caching them in a file you can include, or into some shared cache would be better as you're then not wasting space storing a separate copy of all the translation strings for each visitor. A user's preferences though are ideal to cache into the $_SESSION array as those are dependent on each user.

 

You have been provided plenty of information as to what is possible for ways to cache data on an application-level, but you seem to be choosing to ignore all that and instead continue to just dump everything into $_SESSION, while simultaneously getting upset at people for some reason.

[Psycho]

FYI: Keep in mind that a "cache" is not some magical entity - it is simply a file. A cookie and session data can be considered cache files although there are specific functionality on retention and the ability to set and get values. What has been proposed above about creating a file for each language is definitely the way to go and is, in effect, a cache file. Reading a language file is very fast and this pattern is used a lot. Plus, it is simple to allow a user to "change" their language and load a different file. The language selection should be saved as a cookie value whenever they make a selection.

 

On a side note, I know it is possible with some technologies to actually store such data in memory to be used from request to request even for different users. Maybe this is what you were really after. I don't have any experience with this in PHP, so I can't comment on that. But, I've used language files many time.

[LLLLLLL]

kicken said:

 

 

 

Caching them in a file you can include, or into some shared cache would be better as you're then not wasting space storing a separate copy of all the translation strings for each visitor.

 

And yes, that was the whole point of the thread. How can I implement a shared cache?

 

I think the answer is that there's no good answer in PHP. Not for what I want; things in C# cache easier because of the architecture (services running that hold stuff in memory). I can't use things like MemCache because some of this would be too tricky for an average user to set up on their server. That's why I was hoping there was a way to add to $_SERVER or something like that. Something universal.

 

The solutions like "read from a file" really don't apply to what I need. I have objects, arrays, other things that are eventually loaded. I just wish could load them once, not once per user, to reduce some DB calls. That's all.

[Psycho]

. . . That's why I was hoping there was a way to add to $_SERVER or something like that . . .

 

The solutions like "read from a file" really don't apply to what I need. I have objects, arrays, other things that are eventually loaded. I just wish could load them once, not once per user, to reduce some DB calls. That's all.

 

Here's my parting thought. I think you are making assumptions with no data to back it up. If you did understand all the benefits/drawbacks of using a cache file vs caching in memory I'm sure you would already know the answer to your question. Typically, you would only consider storing data in memory if you have already determine that there is a performance problem.

 

And, yes, PHP does have a way to do that. You're a smart guy. I'm sure you'll figure it out.

[gizmola]

@timneu22,

  You got a complete answer from both Kicken and myself.  You never bothered to reply to those answers or ask questions or anything.  Instead you seemed more intent on creating arguments with people for reasons I can't really understand.

 

Shared cache?  Yes, apcu, redis or memcache.  They all have client support in php and I've used all of them in different projects in the past.  They provide exactly what you were inquiring about.   

 

With that said, it seems pretty clear you don't have the application load or number of users where you are being forced to do something to insure performance.  Nobody is twisting your arm or demanding that you stop putting a load of static data into sessions (which by default get stored per session, in files on the web server fs).  Things that work fine for an intranet or small business system won't scale to significant user load, but if you aren't facing that scenario, then doing something inefficient and resource intensive may never be a problem you actually have to confront.

 

That doesn't mean there isn't a better solution, and for some reason you seem to have just dismissed the suggestions made in this thread, with the implication that you never received a solution, when in fact, you did.