Jump to content

PHP read only new added entry in file


qasimkhans

Recommended Posts

i have a log file which gets new data time by time. is there any way i can read only new added entries (block) in the file? instead of start reading file from start to the end of file. this file size will be in GBs soon. Thanks. currently i am using file() to read the file. following is the example of new added block in file.

 

----SMS_START----

recv_time:2014-10-09 18:32:39
Span: 1
From-Number: +1347XXXXXXX
Timestamp: 14/10/09 18:32:16 96
Type: PDU
SMS
-SMSC-Number: +12404492163
Content: Thanks,

 

----SMS_END----

Link to post
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

Link to post
Share on other sites

you can use fseek() to start reading from a file at a specific location, but you will need to know the location.

 

if all the blocks of data are the same length, you can just create an index (stored in its own file or a database table) of which numbered block corresponds to the date/time of the data, then just do a little math (block number * length of each block) to find the location to fseek() to.

 

if the blocks are variable length, you can create an index (stored in its own file or a database table) of the actual offset that corresponds to the data/time of the data. just get the filesize() of the log file before you write new data to the file. the filesize() value would be the fseek() location for the new block of data. store that in the index with the date/time of the data.

Link to post
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>
Link to post
Share on other sites

 

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>

 

that log file generated by system. so i have to read data from that log file.

Link to post
Share on other sites

A good idea, Thanks, how to clear the log file?

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

 

 

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);
Link to post
Share on other sites

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

Thanks, i will try it and let you know.

Link to post
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.