Preventing emails being sent from website with user's FROM email from initiating warnings

[NotionCommotion]

I have a website which allows users to send emails in their behalf.  I am using PHPMailer with smtp using the mailserver provided by my hosting company, and setting SetFrom as the user's email.

As the user is generating the email, I don't think I am doing anything unethical or illegal.  If you disagree, please advise why.

Recently I noticed Gmail displayed the following warning when receiving such an email:

 

Gmail couldn't verify that this message was sent by bigcorporation.com.  Learn more

 

 

 

Upon clicking "Learn more", multiple content was displayed, however, it appears that the following applied.
 

Spammers can forge a message to make it look like it's sent by a real website or company that you might trust. To help protect you from such messages, Google tries to verify the real sender using email authentication.

The authentication process tries to verify the real sender by looking at a message's authentication data. This data should be included in a message's "signed-by" or "mailed-by" headers (shown beneath the subject line when you look at a message's details). When the sender doesn't include this data, we can't be sure whether or not the message was forged. For example, a message might claim to be from a Gmail address, but we can't confirm that claim if the message doesn't have authentication data.

 

 

Is this what is causing this warning?  Is there anything that can be done to prevent it?  If need be, an acceptable solution is to have the user provide such authentication data, store it in a DB, and include it with all emails initiated by him or her.

[requinix]

As the user is generating the email, I don't think I am doing anything unethical or illegal.  If you disagree, please advise why.

I wouldn't think there's anything wrong with it. May be suspicious, to the receiver. Consider putting some kind of notice within the email indicating that it was sent by you on someone's behalf. "Report abuse" kind of thing. If that even makes sense.

 

Recently I noticed Gmail displayed the following warning when receiving such an email:

I assume bigcorporation.com is the domain of the email who you're claiming to send as? This is part of spam filtering, and can easily get your emails filtered out, so if that's going to be a problem then you need to re-think what you're doing and how.

Two things you can do to avoid it:

1. Send using the user's actual emailing service, which will require information they don't know and information they might not want to give you.

2. Set up domain verification. DomainKeys (DKIM) and SPF validation are the mechanisms for that, but to do verification the user needs to alter the DNS records for their domain to indicate that your email server is allowed to send emails on their behalf.

 

In other words, you can't really do anything about it.

 

With that said, there is a third option:

3. Don't claim to be sending from their email address. You'll avoid the verification problem because you're not attempting to act on behalf of anyone. Again, this may or may not make sense/be viable for your uses. A little notice at the bottom of the email, "This email was sent on behalf of from . Report abuse, etc." would be a good idea.

[ginerjm]

Using someone else's email address as the From address for your email IS a definite red flag to email servers as it should be.  Just because you think it is ok to forge a from address to send one simple email doesn't mean that a mail server will think like that.  The job of a mail server (IMHO) is to prevent such abuses!!!  People do this ALL THE TIME and that is why all of us legitimate email users have to deal with emails coming back from people that we didn't email to or why our IP addresses get blocked because of spam problems.  There is absolutely no good reason to use someone else's email address as the from address that is FROM YOU

[NotionCommotion]

There is absolutely no good reason to use someone else's email address as the from address that is FROM YOU

 

I can think of one good reason.  If I, as a user, use some application to send someone an email, I want them to know it is from me so they take the time to open it.  As a recipient, I often will not even look at an email if I do not know who it is coming from.

[ginerjm]

Then send the email from YOUR account where the From address will be properly set to your address.  Don't send it from the application which is already designed for a different purpose.

 

Frankly - apps that allow such behavior are quite possibly the source of many spam messages today.

[NotionCommotion]

Thanks requinix,

 

I currently have some notice at the bottom of the email stating that the email was sent by mysite.com.

 

Yes, bigcorporation.com is the domain I am claiming to send as.

 

In response to your three options:

1. Maybe it will be viable if I give the user the ability to either use their own email service, or if they don't know it or wish to share it, use mine at the risk of having emails to their intended recipients be blocked.  If I worked at bigcorporation.com, however, I would likely be wary of doing so as I might get in trouble.
2. For DKIM and SPF, will the IT manager of bigcorporation.com need to alter the companies email policies?  If so, probably not going to happen.
3. Maybe.  At a minimum, the most email clients would display the recipient's name, true?  Using their email as the reply email will never be blocked, right?  Maybe adding something to the effect of "Sent in behalf" as the send email address will make it more clear?

[NotionCommotion]

Then send the email from YOUR account where the From address will be properly set to your address.  Don't send it from the application which is already designed for a different purpose.

 

Frankly - apps that allow such behavior are quite possibly the source of many spam messages today.

 

Thanks ginerjm,  You mean send it from my email client (i.e. Outlook, etc)?  Or send it from my website using FROM notioncommotion@mysite.com, and using the email's author's REPLY email only?

[ginerjm]

No - I meant if you want to send an email that is truly from you, don't use a web app to do it - use your own email client.  Spoofing a part of an email to mis-represent it is one of the biggest problems on the net in this day.  That and viruses. And hacking of websites to make them do something not intended by the designer.  All of these are fraudulent activities and for you to even consider doing what you propose makes you (sorry) guilty of the same behavior.

 

If you need to make an email stand out and show who it's from when generating it from your appl., then put that into the Subject.

[NotionCommotion]

Spoofing a part of an email to mis-represent it is one of the biggest problems on the net in this day.  That and viruses. And hacking of websites to make them do something not intended by the designer.  All of these are fraudulent activities and for you to even consider doing what you propose makes you (sorry) guilty of the same behavior.

 

Wow, I didn't know I was guilty of all that.  Think they will let me into Heaven?

[gizmola]

Hi NotionCommotion,

 

Just to clarify what you are doing, albeit for ostensibly understandable reasons, is one or both of "spoofing" or "relaying" which are considered highly undesirable by spam classification systems.  

 

They also, depending on the implementation, open your site up to blacklisting, because nefarious individuals often exploit these features, which used to be common but are now considered a "really bad idea"  to send spam, using your systems.

 

So first relaying:

 

When you are the Mail transfer agent (MTA) for a domain or even several domains, that is something that is configured in DNS, in your MTA settings, and typically has SPF and DKIM settings wired in.  These are things the sysadmin configures, and there are also reverse DNS settings that come into play.

 

In summary, the only emails your MTA is supposed to send on your behalf are emails from user@yourdomain.xyz.   If your MTA sends out emails or relays emails from user@someotherdomain.com, then you are "relaying" emails for that domain, and that is not supposed to be possible, and is an exploitable problem that degrades email for everyone.  That will get you on blacklists.

 

I'm not sure how you are sending out emails, as it is possible to send them directly from an app server, but that is also a great way NOT to get your emails delivered.  Sure they will go out and even be received by some sites, but many more will blackhole your emails (receive and silently delete them) or grade them with high spam scores that are sent to the end user's spam box.  You can also again, get your site on blacklists for doing this.  

 

You should only send email out via a valid MTA for your domain!

 

Many sites also, often by necessity use remailing services, especially if they are hosted on cloud services, as many of these services (AWS for example) severely limit the number of emails that can be sent out of their network directly via SMTP protocol.

 

The one trick that people often try and use to get around the problem without entirely violating the rules and spoofing the from address, is to send emails out as a valid user for your domain, but add the "from user" as the reply to email header.  I know that's not what you're trying to do, but is about all you are typically allowed to do without raising your spam score to the "this is spam" level of most classification systems.

 

I know this is not what you wanted to do, but you simply can not spoof a from address and send it out of your domain and not suffer repercussions that will at very least have the majority of your emails going into the receivers spam box, or more often than not, simply rejected or silently deleted.

 

This is because spoofing from addresses is harmful to the email ecosystem at large, and is a huge red flag for spam classification systems.  It can also cause the receiver's system to erroneously spray error replies to domains that had nothing to do with the email, not to mention the fact, that inherently you are not able to prove to anyone that the email address you are claiming the email is coming from, is actually the person who owns the email in question.

 

This is one reason why the good old html mail tag causes the browser to invoke the user's configured mail client, so that they can send emails out through their own system as it is the only way to legitimately do what you're hoping to accomplish, and have it taken seriously by the receiving email systems.

 

Sorry to be the bearer of the bad news, but the bad people ruined email for the rest of the world a long time ago.

[NotionCommotion]

Hi gizmola,

 

Thank you for your comprehensive response.

 

I learned a new word today: “nefarious”.  Well used  

 

I have had almost 100% delivery success when using both gmail smtp servers as well as my hosting companies smtp servers.  The one issue I have witnessed is when sending an email where the FROM email belonged to a large company and the recipient belonged to that same domain, the email wasn’t delivered.

 

That being said, it appears that I have been relaying and you have me convinced that negative consequences will likely arise in the future.  A typical email header when sent to a gmail account looks like the following (if I send it to a corporate email, there is much more).  Relaying?  Do you mind explaining any other implications gleamed from these headers?

Delivered-To: jane.recipient@gmail.com
Received: by 10.27.231.134 with SMTP id e124asp2784405wlh;
        Mon, 30 Nov 2015 07:02:42 -0800 (PST)
X-Received: by 10.50.18.114 with SMTP id v18xr418453asd.91.1448895762159;
        Mon, 30 Nov 2015 07:02:42 -0800 (PST)
Return-Path: <john.sender@bigcorporation.com>
Received: from smtp1.mywebhosting.com (smtp1.mywebhosting.com. [987.654.32.109])
        by mx.google.com with SMTP id 88si11073401ioi.168.2015.11.30.07.02.41
        for <jane.recipient@gmail.com>;
        Mon, 30 Nov 2015 07:02:42 -0800 (PST)
Received-SPF: neutral (google.com: 987.654.32.109 is neither permitted nor denied by best guess record for domain of john.sender@bigcorporation.com) client-ip=987.654.32.109;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 987.654.32.109 is neither permitted nor denied by best guess record for domain of john.sender@bigcorporation.com) smtp.mailfrom=john.sender@bigcorporation.com
Received: (qmail 30248 invoked from network); 30 Nov 2015 15:05:52 -0000
Received: from unknown (HELO subdomain.mysite.com) (outgoing@mysite.com@123.456.789.01)
    by smtp1.mywebhosting.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Mon, 30 Nov 2015 10:05:52 -0500
Date: Mon, 30 Nov 2015 07:02:40 -0800
To: Jane Doe <jane.recipient@gmail.com>
From: John Doe <john.sender@bigcorporation.com>
Subject: Request for Bid - My Subject
Message-ID: <d8513dfb2385467256e7684c6913fef8@subdomain.mysite.com>
X-Mailer: PHPMailer 5.2.12 (https://github.com/PHPMailer/PHPMailer/)
MIME-Version: 1.0
Content-Type: multipart/alternative;
    boundary="b1_d8513dfb2385467256e7684c6913fef8"
Content-Transfer-Encoding: 8bit

The trick of using a valid FROM email address and the email author’s email address for the REPLY TO doesn’t seem that bad.  You stated “I know that's not what you're trying to do…”.  Are there many shortcomings of this approach?  If the email author’s email address is john.sender@bigcorporation.com, would it work to use a fictional FROM email of john.sender@mysite.com and a REPLY email of john.sender@bigcorporation.com.

 

Thanks again!

[ginerjm]

To quote your initial post:

 

 

I have a website which allows users to send emails in (sic) their behalf.  I am using PHPMailer with SMTP using the mailserver provided by my hosting company and setting SetFrom as the user's email.

That statement simply says "I'm hosting an application that people can use to send faked emails out on."  Is that what you want to do?  I doubt that your hosting company would allow you to continue doing this if you become a pariah among mailservers (which means they do if you are on a shared host.). 

 

I cannot fathom an application that I have ever used for the express purpose of sending someone an email.  Sure - sometimes an appl asks if I want to send a link or something similar to another person and those emails do go out with that apps/servers credentials on them, usually with some reference to my id in the message saying who it was sent on behalf of.

 

What you continue to march on to is in my opinion poor internet behavior.  Everything that Gizmola told you in her succinct and spot-on post should be listened to and used to guide you to make a change in your plan.

 

Emails should always indicate where they came from - NOT some false address that may or may not be real.  You say that the supposed client will be placing their own address in the From field, but how do you know this to be true?  Your host probably would frown on an appl design that plans on mis-using their mail services in this way and could very well terminate your account.

 

Please find another way to inform your recipients from whom these emails have been sent.  (I already suggested using the Subject line.)  Spam is not the way to go and one less spammer in the world is a good thing.  

[requinix]

I should probably read the rest of the thread, but

 

I could imagine a legitimate use case: a third-party service that acts on behalf of some company but itself stays mostly in the shadows. Salesforce does it. Various payment providers and gateways have something like it (like the one my health insurance company uses). It makes sense for those services to be sending email that looks like it's from the hosted company rather than the service's own domain.

With that said, those are the kinds of situations where the hosted company should set up emailing properly - by creating SPF/DKIM records to validate the service's mail servers (as providing email server information would be iffy).

 

It's a concept that's so frequently abused but still possible to use legitimately.

[gizmola]

Basically I agree with ginerjm's summary of my comments.

 

Is this a generic feature of your site, or do you have some special relationship with bigcorporation.com that we're unclear on?

 

With the header you provided there are some interesting things it is telling you.  One is that bigcorporation.com doesn't have an SPF entry in their DNS, so google is not marking you down for that.  Not having SPF is unusual these days.

 

With that said, gmail doesn't provide you insight into how they determine things to be spam.  They just stick stuff in spam using whatever internal criteria they choose.  Other businesses may be using spam assassin or some other system that will frequently add custom mail headers with scoring information that many clients use.

 

At the end of the day, unless you have a special relationship, and have had sysadmin cooperation taking place behind the scenes so that you are a valid MTA for the domain or have added an SPF or DKIM for that business into your MTA, then you're going to have issues.

 

It looks like you are using the smtp1.mywebhosting.com MTA to send your emails out, so this certainly qualifies as "spoofing" and I guarantee you that your email is going to be considered spam by many systems.  Just because you are receiving it on a test account within gmail doesn't mean that many other accounts even within gmail will have it considered spam.  The large hosted email providers like yahoo, google and microsoft have complicated proprietary systems that do all sorts of processing to make individualized determinations about what they classify as spam on a user by user basis.

 

If your box has previously received a lot of email from mywebhosting.com, for example, they may be de facto whitelisting other emails, when for some other user, they would stick it in the spam folder.

[NotionCommotion]

For the record, ginerjm, I don't want to allow people to send out fake emails and generate spam.  While I had been setting the SetFrom address as the user's email, I stated in my response to gizmola that I no longer think it is a good idea.  It has nothing to do with ethics as feel in an ideal world it is more appropriate to display identifying email of the individual who authored and sent the communication, and only to do with how to feasible deliver well intended emails.

 

I agree requinix there are many legitimate use cases.  To see how Evite does it, I just signed up.  They use the recipricant’s name and reply email, but an Evite from email.  I think this is what I will do.  My problem with the SPF/DKIM option is it would be too much work for some small individual in a big organization to get IT to make the changes (or am I missing something?).

 

Thanks gizmola.  Bigcorporation.com is only a possible recipricant.  The headers I showed you came from my site, and went to my gmail account.  I think I am benefiting from mywebhosting.com’s whitelisting, and while I am not sending out spam, I should not take it for granted.  Agree an email from my site as the FROM email, the user’s name, and the user’s email as reply only makes sense?

[gizmola]

DKIM and SPF are for entire organizations.  Unless you were hosting all mail activities for a domain, or acting as a forwarding agent, you wouldn't be interested in those technologies.  In other words, you can't for example, change the SPF information for each outbound email.  BigCo.com would have to set up an SPF entry in their DNS that would indicate that your mail system was authorized by them to send email on their behalf.  That is not going to happen.  DKIM is a similar and even more strict technology that uses public/private key setups in the MTA (mail servers) to sign emails so that other servers can verify that they actually came from your domain.  Again this would not be applicable, but these features of email security were brought up because we needed to clarify what you are doing.

 

 

It seems at this point, we all agree that the solution you want is stated by you in your most recent responses:

 

 

1. Send the email From:  account@yourdomain.tld

2. Add the reply to:  original username

 

You could experiment with setting the "text name" portion to be the same for both, as it just occurred to me:

 

 

Some Name <email@domain.tld>

 

From an end user standpoint this is much better to send the mail with the descriptive name that you're presumably getting from the users, and as this is coming from a form they fill in, you can do some checking there to make sure you have a semi-valid name, or don't allow the form to send the email, which will knock down some bots, or at least make the emails they send less likely to confuse the end users.  

 

You want to take whatever precautions you can on this type of form to make it hard for a bot to utilize it to spam a lot of people.

[NotionCommotion]

Thank you gizmola.  I appreciate your and requinix's and ginerjm's help.

 

Regarding setting the "text name" portion to be the same for both, do I understand you to mean:

 

From Email:   myaccount@mydomain.tld

From Name:  theirname <theiremail@theirdomain.tld>

[NotionCommotion]

I ended up going with the following for a while:

 

From Email:   myaccount@mydomain.tld

From Name:  theirname <theiremail@theirdomain.tld>

 

User's were confused as it appears it was sent by or would be returned to both addresses as typical email clients will show both emails in list view (both are shown in detail view, but I think people have been accustom to not noticing).

 

I've since gone to:

 

From Email:   myaccount@mydomain.tld

From Name:  theirname

 

While maybe not perfect, I think it is the best compromise to meet user expectations and pass through email servers.