Jump to content
qasimkhans

PHP read only new added entry in file

Recommended Posts

i have a log file which gets new data time by time. is there any way i can read only new added entries (block) in the file? instead of start reading file from start to the end of file. this file size will be in GBs soon. Thanks. currently i am using file() to read the file. following is the example of new added block in file.

 

----SMS_START----

recv_time:2014-10-09 18:32:39
Span: 1
From-Number: +1347XXXXXXX
Timestamp: 14/10/09 18:32:16 96
Type: PDU
SMS
-SMSC-Number: +12404492163
Content: Thanks,

 

----SMS_END----

Share this post


Link to post
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

Share this post


Link to post
Share on other sites

you can use fseek() to start reading from a file at a specific location, but you will need to know the location.

 

if all the blocks of data are the same length, you can just create an index (stored in its own file or a database table) of which numbered block corresponds to the date/time of the data, then just do a little math (block number * length of each block) to find the location to fseek() to.

 

if the blocks are variable length, you can create an index (stored in its own file or a database table) of the actual offset that corresponds to the data/time of the data. just get the filesize() of the log file before you write new data to the file. the filesize() value would be the fseek() location for the new block of data. store that in the index with the date/time of the data.

Share this post


Link to post
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>

Share this post


Link to post
Share on other sites

 

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>

 

that log file generated by system. so i have to read data from that log file.

Share this post


Link to post
Share on other sites

If you are reading the log file and copying the content to the database, then why not clear out the log file when you do that?

Share this post


Link to post
Share on other sites

If you are reading the log file and copying the content to the database, then why not clear out the log file when you do that?

A good idea, Thanks, how to clear the log file?

Share this post


Link to post
Share on other sites

A good idea, Thanks, how to clear the log file?

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

 

 

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

Share this post


Link to post
Share on other sites

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

Thanks, i will try it and let you know.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.