Creating a custom feed parser.

[maexus]

What I'm trying to do is have a cron job run a php script that checks an XML feed every 30 minutes. It will check for feed entries that have been added or modified in the last 30 minutes and then sends  those feed entries through a function. I should mention the feed I'm dealing with is gmail's ATOM feed.

 

Now, I have a script running that does this by taking the current timestamp and subtracting 30 minutes then comparing it to each of the entries last modified times. This actually seems to work well except there are times where it misses an email or two, especially when it's a reply email. I'm assuming this is due to gmails conversation system. I'm also assuming this is not how other feed parsers work as mine is entirely dependant on the cron job running every 30 minutes on the nose. It just feels half assed. Then again, this may not be an issue if Google just realized a proper API for gmail. 

 

Here is my last working version of the code if it helps. BTW, This was written at work so it's a little sloppy.

 

<?php

//1800 seconds is 30 minutes. You need to match this with the number of seconds of your cron intervals
$currentTime = time() - 1800;

//The credentials to make this all work
define("TWITTER_USERNAME",NULL);
define("TWITTER_PASSWORD",NULL);
define("GMAIL_USERNAME",NULL);
define("GMAIL_PASSWORD",NULL);

//proxy settings, check with your host to see if these are required
define("REQUIRES_PROXY",1);
define("PROXY_URL", "http://proxy.shr.secureserver.net");
define("PROXY_PORT","3128");

//no need to touch these
define("TWITTER_API_URL","http://twitter.com/statuses/update.xml?status=");
define("GMAIL_API_URL","https://mail.google.com/mail/feed/atom");

//this is all the dirty work and doesn't need to be touched
function postNotification($message){
$c = curl_init();
curl_setopt($c, CURLOPT_URL, TWITTER_API_URL.urlencode(stripslashes(urldecode($message))));
curl_setopt($c, CURLOPT_USERPWD, TWITTER_USERNAME.":".TWITTER_PASSWORD);
curl_setopt($c, CURLOPT_VERBOSE, 1);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($c, CURLOPT_POST, 1);

$exec = curl_exec($c);
$exec_array = curl_getinfo($c);

curl_close($c);

if($exec_array['http_code'] == "200"){
	echo "Everything went to plan";
}else{
	echo "Something fucked up";
}
}

$gmailCurl = curl_init();
curl_setopt($gmailCurl, CURLOPT_URL, GMAIL_API_URL);
curl_setopt($gmailCurl, CURLOPT_USERPWD, GMAIL_USERNAME.":".GMAIL_PASSWORD);
curl_setopt($gmailCurl, CURLOPT_VERBOSE, 1);
curl_setopt($gmailCurl, CURLOPT_RETURNTRANSFER, 1);

if(REQUIRES_PROXY == TRUE){
curl_setopt ($gmailCurl, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt ($gmailCurl, CURLOPT_PROXY,PROXY_URL.":".PROXY_PORT);
curl_setopt ($gmailCurl, CURLOPT_SSL_VERIFYPEER, 0);
}

$gmailXml = curl_exec($gmailCurl);
curl_close($gmailCurl);

$emails = new SimpleXMLElement($gmailXml);

foreach($emails->entry as $email){
$emailTimestamp = strtotime($email->issued);
if($emailTimestamp > $currentTime){
	postNotification("S: ".$email->author->name." / T: ".$email->title." / M: ".$email->summary);
}
}

[maexus]

After looking at my gmail inbox feed, it appears that gmail has a 25-30 minute cache on the data. This may be causing the occasional missed emails being sent through the function. This also means I need a new solution other than just subtracting 30 minutes from the current time and then hoping it's perfect insync with gmails cache timer and the cron job running. I could store the ID's of the check entries in a database and flush it every once and a while but that seems like alot of bloat for such a small script. Suggestions?