Jump to content

PHP read only new added entry in file


qasimkhans

Recommended Posts

i have a log file which gets new data time by time. is there any way i can read only new added entries (block) in the file? instead of start reading file from start to the end of file. this file size will be in GBs soon. Thanks. currently i am using file() to read the file. following is the example of new added block in file.

 

----SMS_START----

recv_time:2014-10-09 18:32:39
Span: 1
From-Number: +1347XXXXXXX
Timestamp: 14/10/09 18:32:16 96
Type: PDU
SMS
-SMSC-Number: +12404492163
Content: Thanks,

 

----SMS_END----

Link to comment
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

Link to comment
Share on other sites

you can use fseek() to start reading from a file at a specific location, but you will need to know the location.

 

if all the blocks of data are the same length, you can just create an index (stored in its own file or a database table) of which numbered block corresponds to the date/time of the data, then just do a little math (block number * length of each block) to find the location to fseek() to.

 

if the blocks are variable length, you can create an index (stored in its own file or a database table) of the actual offset that corresponds to the data/time of the data. just get the filesize() of the log file before you write new data to the file. the filesize() value would be the fseek() location for the new block of data. store that in the index with the date/time of the data.

Link to comment
Share on other sites

A couple things come to mind:

 

1. Write the data to a database instead of (or in addition to) the log file. Getting dynamic entries would be much easier.

 

2. Create a new log file each week/month and archive older ones on a regular basis. Having log files n GB will become a problem even if you aren't wanting to dynamically read specific entries. Then you can just pull content from the current log file and possibly one back.

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>
Link to comment
Share on other sites

 

following is my script, i read the file and save in DB.
 
<?php
require_once('dbconnect.php');
 
$file = file("/var/log/asterisk/recvsms/recvsms_log");
foreach($file as $key => $value)
{
$filter_value = trim($value);
 
if($filter_value!="" && $filter_value!="----SMS_START----" && $filter_value!="----SMS_END----"){
$new_array[] = $value;
$my_array = array_chunk($new_array, 7);
}
}
$count = count($my_array);
for($I=0;$I<$count;$I++){
foreach($my_array[$I] as $key => $value){
 
$ss = explode(":", $value);
$mm_array[$I][$ss[0]] = str_replace($ss[0].":", "", $value);
}
mysql_query("insert into receive_sms (receive_time,span,sms_from_number,timestamp,type,sms_center_number,sms_content)
     values('".$mm_array[$I]['recv_time']."','".$mm_array[$I]['Span']."','".$mm_array[$I]['FromNumber']."','".$mm_array[$I]['Timestamp']."','".$mm_array[$I]['Type']."','".$mm_array[$I]['SMS-SMSC-Number']."','".$mm_array[$I]['Content']."')");
}
?>

 

that log file generated by system. so i have to read data from that log file.

Link to comment
Share on other sites

A good idea, Thanks, how to clear the log file?

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

 

 

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);
Link to comment
Share on other sites

 

Open the file for editing and use the "w" mode to truncate the file. Then close the file. Not sure, but you *may* have to write something to the file first. If so, just write an empty string. Do this AFTER you have read the contents and put into the DB.

$myFile = "path/to/logfile.log";
$fh = fopen($myFile, 'w');
fwrite($fh, ''); //May not be needed
fclose($fh);

Thanks, i will try it and let you know.

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.