Email Parsing

[rklockner]

I am building a script to input the contents of an email into a database.  More specifically, there will be a variable with a value on each line, with the variable name corresponding to a field in the database.

 

After creating the destination email account, I modified /var/qmail/mailnames/domain.com/emailacct/.qmail to include a link to the php file I would like to run.  The contents of the file:

 

[root@ip-ip~]# vi /var/qmail/mailnames/epartnersoftware.com/RELI/.qmail                                 

| true

./Maildir/

/var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php

 

 

I then restarted qmail (/etc/init.d/qmail restart)

 

Set the permissions to 755 (chmod 755 /var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php)

 

When I send an email to the account, I get the following error:

 

"Hi. This is the qmail-send program at ip-ipaddress.ip.secureserver.net.

I'm afraid I wasn't able to deliver your message to the following addresses.

This is a permanent error; I've given up. Sorry it didn't work out.

 

<[email protected]>:

Unable to open /var/www/vhosts/domain.com/httpdocs/email_parser1/email_parse.php: access denied. (#4.2.1)"

 

The strange part... early on in my testing I got the file to open, but never again.

 

NOTE: This idea came from an earlier post found here http://www.phpfreaks.com/forums/index.php?topic=250995.0

 

Thanks for the help.

[rklockner]

Right now the file is located in the web (httpdocs) directory on the server.  I posted on some forums last night, and people think that the email account may not have the ability to access the httpdocs directory and that I may need to put the file in the user's directory.

 

This would create problems for me as we will be adding new clients regularly to the software that will have their own email addresses that will be used to parse leads into their own access-area of the database.

 

I would like to keep the file in a location that all new accounts can access without having to upload a new file with each new account.

 

Some operating systems allow you to specifically remove the ability for people to read, write, etc. separate from the file system.  Are we able to do this? 

 

In other testing, I have resolved the permissions error... kind of.  I moved the file to the user's directory and had to set the permissions to 777, and I no longer get the error.  However, the script in my page does not run.

 

NOTE: The data will only be coming via email, so we must utilize a method that will parse an email.

 

[rklockner]

I know I saw this somewhere in another post, but I have a new issue.  I set the permissions of my scipt file to 777, and now the email is appending itself to the end of my script page.

[schilly]

First thing that sticks out

[root@ip-ip~]# vi /var/qmail/mailnames/epartnersoftware.com/RELI/.qmail                                  
| true
./Maildir/
/var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php

 

You need to have | in front of your script path.

 

[root@ip-ip~]# vi /var/qmail/mailnames/epartnersoftware.com/RELI/.qmail                                  
| true
./Maildir/
| /var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php

 

Start with my blog example script first to make sure the piping is working properly:

#!/usr/bin/php
<?php
// read in email from stdin
$fd = fopen("php://stdin", "r");
$email = "";
while (!feof($fd)) {
$email .= fread($fd, 1024);
}
fclose($fd);

//send us the email to make sure it worked
mail('[email protected]','someone sent us an email at [email protected]',"Here is the the full email:\n\n$email");
?>

 

I'm fairly certain the qmail *nix user cannot access your httpdocs directory so you need to put your script somewhere the qmail *nix user can access it. Make sure any includes you put in your script use the full path. Any includes must also be accessible by the qmail *nix user.

 

Change your .qmail first first then try my sample script and see what happens.

[rklockner]

I'm gettng much closer.  I don't have time right now to look into this further, but I thought I would post this anyway incase someone else might have insight before I get at it.

 

However, by adding the '|' appears to have me moving in the right direction!  This appears to me that it is attempting to run it as a command line file and not php.  I'll review my code later.

 

Thanks for all the help so far!

 

Usage: php [options] [-f] <file> [--] [args...]

      php [options] -r <code> [--] [args...]

      php [options] [-B <begin_code>] -R <code> [-E <end_code>] [--] [args...]

      php [options] [-B <begin_code>] -F <file> [-E <end_code>] [--] [args...]

      php [options] -- [args...]

/  -a              Run interactively

  -c <path>|<file> Look for php.ini file in this directory

  -n              No php.ini file will be used

  -d foo[=bar]    Define INI entry foo with value 'bar'

  -e              Generate extended information for debugger/profiler

  -f <file>        Parse <file>.

  -h              This help

  -i              PHP information

  -l              Syntax check only (lint)

  -m              Show compiled in modules

  -r <code>        Run PHP <code> without using script tags <?..?>

  -B <begin_code>  Run PHP <begin_code> before processing input lines

  -R <code>        Run PHP <code> for every input line

  -F <file>        Parse and execute <file> for every input line

  -E <end_code>    Run PHP <end_code> after processing all input lines

  -H              Hide any passed arguments from external tools.

  -s              Display colour syntax highlighted source.

  -v              Version number

  -w              Display source with stripped comments and whitespace.

  -z <file>        Load Zend extension <file>.

/  args...          Arguments passed to script. Use -- args when first argument

                  starts with - or script is read from stdin

 

[rklockner]

OK, I spent a few minutes, and this is what is getting bounced back to me now.

 

/bin/sh: /var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php: /usr/bin/php

: bad interpreter: No such file or directory

 

I am assuming that means php is not installed in that directory.  Anyone know how to find the appropriate directory?

[BlueSkyIS]

command-line:

 

which php

[rklockner]

Not sure...  The one that would be able to read a php file.

[BlueSkyIS]

the command to find where php is is:

 

which php

 

user@localhost~$ which php

/usr/bin/php

 

[rklockner]

OH... oops. 

 

That is bad news then.  PHP is installed in /usr/bin/php, so my error must be elsewhere.

 

Thanks

[rklockner]

Got it.

 

This error was generated as I was FTPing from a Windows machine to a Linux server.  I had to convert the file using:

 

dos2unix /var/www/vhosts/domain.com/httpdocs/emailparser/email_parse.php

 

Thanks for everyones help.

 

FYI - qmail can access the httpdocs folder using permissions 755 to run the php scipt.

[schilly]

Everything working?

 

[rklockner]

It is now accessing the file.  As I see it, that was the hard part.  Later tonight I will actually build the piece reviews the email and pulls out the new lead information and inputs it into the database.  This should be easy from here.

 

Thanks for getting me over the hump!

[schilly]

no prob. ya the piping part is definitely the hardest.

 

see my blog if you want to parse out the header info. ill have a post up this week on saving attachments and i have a post in the works on parsing bounce emails.

[rklockner]

If I will have multiple email accounts that will need to have emails parsed, is there a way to parse all emails to a particular domain?

[schilly]

not that I know of. there may be a global config file. you could still use the same script, you would just need to set up the .qmail file every time a new email account was created or depending how your system works, BCC a specific email account set up for piping.

[rklockner]

Everything is working perfectly.  However, now that it is operational, the client would like to maintain it himself.  Once I showed him the steps, his response was, "Can't it be done easier?" 

 

I have created a panel for him in his software to assign email addresses to his users that when an email arrives, it will look up the users id matched to the email address, and add the data in the email to that user.  However, when to set up an email address the following steps currently have to be done:

 

1. Create the email address in Plesk

2. Add the line of code to access the script

3. Restart qmail

4. Set up the email address in his software to link it to the specific user.

 

I don't think this is so bad for the service it is accomplishing, but he would like to log in to his software, create the email account, and have his software do everything on the backend.

 

Any thoughts?

[rklockner]

Playing with this, I came up with a great solution.  There is a first time for everything... 

 

Instead of creating each email account on the server, create a catchall email account that copies of your undeliverable emails are forwarded.  This will still retain the original "To:" email address. 

 

You place the code to run the script on only the catchall email address.  It will match the "To" address to the corresponding record in the table, and pull the id associated with that row.

 

In my preliminary tests, this seems to be working.

[schilly]

Yea. That was my idea with the BCC option.

 

Other option:

 

You may be able to create the email account via cmd line through qmail instead of via plesk.

 

http://www.whirlycott.com/phil/pop3.html

 

This relies on your PHP build having exec access. The apache user probably doesn't have access to do that so you may need to "su" to the root account. Other option would be to create some kind of nix script (bash, shell, etc) and exec that from your php file.