PHP read only new added entry in file

[qasimkhans]

i have a log file which gets new data time by time. is there any way i can read only new added entries (block) in the file? instead of start reading file from start to the end of file. this file size will be in GBs soon. Thanks. currently i am using file() to read the file. following is the example of new added block in file.

 

----SMS_START----

recv_time:2014-10-09 18:32:39
Span: 1
From-Number: +1347XXXXXXX
Timestamp: 14/10/09 18:32:16 96
Type: PDU
SMS-SMSC-Number: +12404492163
Content: Thanks,

 

----SMS_END----

[Psycho]

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

[mac_gyver]

you can use fseek() to start reading from a file at a specific location, but you will need to know the location.

 

if all the blocks of data are the same length, you can just create an index (stored in its own file or a database table) of which numbered block corresponds to the date/time of the data, then just do a little math (block number * length of each block) to find the location to fseek() to.

 

if the blocks are variable length, you can create an index (stored in its own file or a database table) of the actual offset that corresponds to the data/time of the data. just get the filesize() of the log file before you write new data to the file. the filesize() value would be the fseek() location for the new block of data. store that in the index with the date/time of the data.

[qasimkhans]

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

following is my script, i read the file and save in DB.

 

<?php

require_once('dbconnect.php');

 

$file = file("/var/log/asterisk/recvsms/recvsms_log");

foreach($file as $key => $value)

{

$filter_value = trim($value);

 

if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){

$new_array[] = $value;

$my_array = array_chunk($new_array, 7);

}

}

$count = count($my_array);

for($I=0;$I<$count;$I++){

foreach($my_array[$I] as $key => $value){

 

$ss = explode(":", $value);

$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);

}

mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)

     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");

}

?>

[qasimkhans]

 

following is my script, i read the file and save in DB.

 

<?php

require_once('dbconnect.php');

 

$file = file("/var/log/asterisk/recvsms/recvsms_log");

foreach($file as $key => $value)

{

$filter_value = trim($value);

 

if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){

$new_array[] = $value;

$my_array = array_chunk($new_array, 7);

}

}

$count = count($my_array);

for($I=0;$I<$count;$I++){

foreach($my_array[$I] as $key => $value){

 

$ss = explode(":", $value);

$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);

}

mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)

     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");

}

?>

 

that log file generated by system. so i have to read data from that log file.

[Psycho]

If you are reading the log file and copying the content to the database, then why not clear out the log file when you do that?

[qasimkhans]

If you are reading the log file and copying the content to the database, then why not clear out the log file when you do that?

A good idea, Thanks, how to clear the log file?

[Psycho]

A good idea, Thanks, how to clear the log file?

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

 

 

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

[qasimkhans]

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

Thanks, i will try it and let you know.