Remember the last value

[etrader]

I want to save live data into file. My script fetch the data (one by one) and save it into file by amend method. The live data is not regular and every time it must be checked if the value has been updated or it is still the last value. One way to do so is to read the last line of the file and compare it with the fetched value. But in this case, I need to read the file for every check.

 

Is there a way to remember the last value and compare it with the coming value? to write only new value into the file.

[gizmola]

A file is not a database.  The most efficient way of doing this is to use fget to read through the file to the last line and compare that each time.  There are numerous caching addons available that might allow you to optimize this if it proves to be a major bottleneck.  You might want to look into memcached and apc.

 

People use databases because they can efficiently seek to individual rows and have provisions for multi-user access.

[Amit20]

"

save it into file by amend method

 

Did you mean append method???

[Dusaro]

http://www.tizag.com/phpT/fileappend.php

 

maybe that will help?

[gizmola]

 

 

maybe that will help?

 

He already stated he's appending messages  to the file. 

[Dusaro]

oops miss read >.<

[etrader]

Yes, I use append method to add new data as a new line.

 

The problem is that when I read the file to get the last line, php walks across the entire file. Since this repeats regularly, it is necessary to avoid heavy task.

 

I already use memcache (on nginx). Another way is to use mysql database. But the problem is that we regularly read and write data. Indexing mysql is good for the first one and bad for the latter.

[gizmola]

MySQL is optimized for inserts.  There aren't too many scenarios where this isn't going to be better done with a database based on your description.  You could also look at a nosql db like mongodb. 

[etrader]

As a matter of fact, the fame of SQL-based databases kept me away of noSQL. I was thinking of SQLite, but I have no idea how much lighter is MongoDB (comparing to mysql).

[gizmola]

Long thin tables are a mysql specialty.  Btree+ indexing guarantees that access to the last entered row will be lightning fast, and readers do not block writers.  Compare this to your file based approach and as the file grows in size reading it will become slower and slower.  SQLite is light, which is to say that it has no server.  This means that every process that does a sqllite connection needs to allocate memory for anything sql lite is going to do.  I would not use sql lite, however this does remind me that there are name/value pair db's like berkley db that are extremely fast.  So you could supplement this by having a key of "lastline" and just read that and only write to your log file when the value read does not match the 'lastline' key.    This is basically the same thing you would do with memcached or apc, but it might be easier for you to use a small berkley db.  Mongodb could be a great solution, but you would need to install the mongodb server and that seems like overkilll for this problem.

[etrader]

Thanks for useful information. I learned a lot.