email decryption

[Destramic]

hey guys im currently using libsodium to encrypt users data which is stored in a database...my concern is when a user registers an account on my website, i want to check that the email provided is not already registered to another account, but the problem is that the email address stored in the database is encrypted...so how do i check?

 

i have perviouslt been suggested to store the email as:

• a separate HMAC
• ECB mode
• no encryption as long as the e-mail addresses are kept away from the web frontend

 

but even when using HMAC the email can easily be viewed, MySQL's ECB mode i've read so many bad things about regarding it having so many security issue etc...and the email having no encrption could mean that if my database every got attacked its all there in black and white.

 

here is my encryption class:

<?php

namespace Encryption;

use Exception;

class Encryption
{
	private $private_key;
	
	public function __construct(sting $private_key)
	{
		if (!extension_loaded('libsodium'))
		{
		    throw new Exception('Encryption: PHP libsodium extension not loaded.');
		}
		
        $private_key = trim($private_key);
        
		if (!preg_match('/^[a-z\d+\/]{43}=$/i', $private_key))
		{
		    throw new Exception('Encryption: Unrecognized key.');
		}
		
		$this->private_key = base64_decode($private_key);
	}
	
	public function encrypt(string $data)
	{
		$nonce = \Sodium\randombytes_buf(\Sodium\CRYPTO_AEAD_CHACHA20POLY1305_NPUBBYTES);
	  
		$ciphertext = \Sodium\crypto_aead_chacha20poly1305_encrypt(
		    $data,
		    null,
		    $nonce,
		    $this->private_key
		);
		
		return base64_encode($nonce) . ':' . base64_encode($ciphertext);
	}
	
	public function decrypt(string $ciphertext)
	{
		$ciphertext = $this->parse_ciphertext($ciphertext);

		list($nonce, $ciphertext) = $ciphertext;

		$decrypted = \Sodium\crypto_aead_chacha20poly1305_decrypt(
		    $ciphertext,
		    null,
		    $nonce,
		    $this->private_key
		);

		if (!$decrypted)
		{
		    throw new Exception('Encryption: Decryption Failed.');
		}
		
		return $decrypted;
	}

	private function parse_ciphertext(string $ciphertext)
	{
	    $ciphertext = trim($ciphertext);
	    
	    if (!preg_match('/^(?:[a-z\d+\/]{11}=)?:[a-z\d+\/]+)(=|==)?$/i', $ciphertext))
	    {
	        throw new Exception('Encryption: Unrecognized ciphertext.');
	    }
	    
		$ciphertext = explode(':', $ciphertext);

		return array(
			base64_decode($ciphertext[0]),
			base64_decode($ciphertext[1])
		);
	}
}

it just seems like i've taken one step forward in being secure, but taking 2 steps back when it comes to processing simple scripts such as verifying email isn't registered. retrieving account by email address etc.

 

i know the answer isn't going to be a simple as

SELECT username FROM user WHERE email_address = 'whatever@gmail.com'

but there must be a logical way to check encrypted email address with a string.

 

any other thoughts on this please guys?

 

 

thank you for your time

[Jacques1]

The point of the HMAC is to act as an independent “checksum” so that you can keep your e-mail addresses encrypted in any way you want and perform the UNIQUE check on the HMAC instead of the (unknown) address.

 

This completely solves the problem.

[Destramic]

now that is smart...i wish i thought of it

 

i had a mess about with hmac over the weekend, as i've decided to use it with the users cookies...is what i made for hmac more than suitable?

 

 

here is a working example

<?php

class Encryption
{
    private $private_key;

    public function __construct(string $private_key)
    {
        if (!extension_loaded('libsodium'))
        {
            throw new Exception('Encryption: PHP libsodium extension not loaded.');
        }

        $private_key = trim($private_key);

        if (!preg_match('/^[a-z\d+\/]{43}=$/i', $private_key))
        {
            throw new Exception('Encryption: Unrecognized key.');
        }

        $this->private_key = base64_decode($private_key);
    }

    public function encrypt(string $data)
    {
        $nonce = \Sodium\randombytes_buf(\Sodium\CRYPTO_AEAD_CHACHA20POLY1305_NPUBBYTES);
         
        $ciphertext = \Sodium\crypto_aead_chacha20poly1305_encrypt(
            $data,
            null,
            $nonce,
            $this->private_key
        );

        return base64_encode($nonce) . ':' . base64_encode($ciphertext);
    }

    public function decrypt(string $ciphertext)
    {
        $ciphertext = $this->parse_ciphertext($ciphertext);

        list($nonce, $ciphertext) = $ciphertext;

        $decrypted = \Sodium\crypto_aead_chacha20poly1305_decrypt(
            $ciphertext,
            null,
            $nonce,
            $this->private_key
        );

        if ($decrypted === false)
        {
            throw new Exception('Encryption: Decryption Failed.');
        }

        return $decrypted;
    }

    private function parse_ciphertext(string $ciphertext)
    {
        if (!preg_match('/^(?:[a-z\d+\/]{11}=)?:[a-z\d+\/]+)(=|==)?$/i', $ciphertext))
        {
            throw new Exception('Encryption: Unrecognized ciphertext.');
        }
         
        $ciphertext = explode(':', $ciphertext);

        return array(
            base64_decode($ciphertext[0]),
            base64_decode($ciphertext[1])
        );
    }
}

class HMAC
{
    private $private_key;
    private $algo;
    
    public function __construct(string $private_key, string $algo = 'sha512')
    {
        $private_key = trim($private_key);
        
        if (!preg_match('/^[a-z\d+\/]{43}=$/i', $private_key))
        {
            throw new Exception('Encryption: Unrecognized key.');
        }
        else if (!in_array(strtolower($algo), hash_algos()))
        {
            throw new Exception(sprintf('HMAC: Algo %s unsupported.', $algo));
        }
        
        $this->private_key = base64_decode($private_key);
        $this->algo        = $algo;
    }
    
    public function seal(string $message, string $public_key)
    {
        $seal = base64_encode(hash_hmac($this->algo, $message, $this->private_key));
        
        return base64_encode($message) . ':'. $seal . ':' . base64_encode($public_key);
    }
    
    public function sign(string $seal, string $public_key)
    {
        if (!preg_match('/^((?:[a-z\d+\/]+)(=|==)?)?:[a-z\d+\/]+)(=|==)??:[a-z\d+\/]+)(=|==)?$/i', $seal))
        {
            throw new Exception('HMAC: Unrecognized seal.');
        }
        
        list($message, $seal, $key) = explode(':', $seal);
        
        $message = base64_decode($message);
        $signed  = base64_encode(hash_hmac($this->algo, $message, $this->private_key));
        
        if ($seal == $signed && base64_decode($key) == $public_key)
        {
            return $message;
        }
        
        throw new Exception('HMAC: Seal corrupted.');
    }
}

$public_key             = 'ZZtJVgUu2fRz+c4o6QHj6v/mAqGAgyowlUxs3xoMHuw=';
$hmac_private_key       = 'DxA58JcURnz891sVXowkF6VPyanis+GvwZXWcoxwE5M=';
$encryption_private_key = 'qB2fZkseI4ccJ45Y1/VzoHARA6Sft6IVkeS4r2Z+YYM=';

$encryption    = new Encryption($encryption_private_key);
$email_address = $encryption->encrypt('email@test.com');
// q101ZtOPjW8=:b9vrNQFhpC5wWhfWDmzu2XcjBly234AASKU11AiM

$hmac          = new HMAC($hmac_private_key);
$seal          = $hmac->seal($email_address, $public_key);
// TWZIaTVxdjdrd1E9OjRvNlE3b05UcFA5SVB1QkR4cEZTZGpUSElFMDd2ai9mRzhwYUd4VmE=:Nzk5NzhhMzgzYjQ0ODc0MjExNDcxMjg1OWVkMmNlY2EwMmE4ZDVlM2E3ZmM5NWJkZTFmZjMwMTkyOTZiOWNjZjZjMjk5NWQzOGJmZTE2MTRkMTAyMzg2NTZmYTg0OWQwYjBhNjAxYTZhYTg5YTI1ZTY2MWRiN2MzZDk4MzU3MTc=:Wlp0SlZnVXUyZlJ6K2M0bzZRSGo2di9tQXFHQWd5b3dsVXhzM3hvTUh1dz0=

$email_address = $hmac->sign($seal, $public_key);
// q101ZtOPjW8=:b9vrNQFhpC5wWhfWDmzu2XcjBly234AASKU11AiM

echo $encryption->decrypt($email_address);
// email@test.com

thank you jacques for your patience and help on this matter

[Destramic]

actually my example isn't going to work...email address is encrypted and placed inside the hmac but the seal will obviously be different everytime...so there would be no way for me to compare.

 

i'm completely lost here

[Jacques1]

Just don't include the message in the output. Only return the HMAC.

• Storing a “public key” together with the HMAC makes no sense either; I'm not even sure what you mean by that, because HMACs have nothing to do with public-key cryptography.
• HMAC validation must not use the standard string comparison algorithm; when you use the == or === operator, PHP stops at the first non-matching byte and allows an attacker to figure out the expected HMAC through time differences. You need a special comparison function which is immune to timing attacks like hash_equals().
• The expected key length is hard-coded to 256 bits, but it should be equal the output length of the hash algorithm (RFC 2104). In the case of SHA-512, that's 512 bits. So either create a lookup table for the output sizes of all supported hash algorithms, or use a hard-coded algorithm together with its specific length (SHA-256 is perfectly fine, no need for anything else).
• Use binary keys as arguments; expecting pre-encoded strings is a bit weird and makes validation more difficult.
• Currently, you double-encode the computed HMAC by letting PHP hex-encode the result and then adding your own Base64 encoding. This makes no sense and wastes space. If you want your own encoding, apply it to binary data (set the fourth parameter of hash_hmac() to true).

Edited October 11, 2016 by Jacques1

[Destramic]

i'm finding it really hard to keep up as most of this is going over my head...although i think i've made some progress after some reading about...also the reason i base64 encode is so thats easy to store in db as a blob

 

here is what i got as it stands, but i'm stuck now and i'm strugging to see how this is going to work.

<?php

class Encryption
{
    private $private_key;

    public function __construct(string $private_key)
    {
        if (!extension_loaded('libsodium'))
        {
            throw new Exception('Encryption: PHP libsodium extension not loaded.');
        }

        $private_key = trim($private_key);

        if (!preg_match('/^[a-z\d+\/]{43}=$/i', $private_key))
        {
            throw new Exception('Encryption: Unrecognized key.');
        }

        $this->private_key = base64_decode($private_key);
    }

    public function encrypt(string $data)
    {
        $nonce = \Sodium\randombytes_buf(\Sodium\CRYPTO_AEAD_CHACHA20POLY1305_NPUBBYTES);
         
        $ciphertext = \Sodium\crypto_aead_chacha20poly1305_encrypt(
            $data,
            null,
            $nonce,
            $this->private_key
        );

        return base64_encode($nonce) . ':' . base64_encode($ciphertext);
    }

    public function decrypt(string $ciphertext)
    {
        $ciphertext = $this->parse_ciphertext($ciphertext);

        list($nonce, $ciphertext) = $ciphertext;

        $decrypted = \Sodium\crypto_aead_chacha20poly1305_decrypt(
            $ciphertext,
            null,
            $nonce,
            $this->private_key
        );

        if ($decrypted === false)
        {
            throw new Exception('Encryption: Decryption Failed.');
        }

        return $decrypted;
    }

    private function parse_ciphertext(string $ciphertext)
    {
        if (!preg_match('/^(?:[a-z\d+\/]{11}=)?:[a-z\d+\/]+)(=|==)?$/i', $ciphertext))
        {
            throw new Exception('Encryption: Unrecognized ciphertext.');
        }
         
        $ciphertext = explode(':', $ciphertext);

        return array(
            base64_decode($ciphertext[0]),
            base64_decode($ciphertext[1])
        );
    }
}

class HMAC
{
    private $private_key;
    private $algo;
    
    public function __construct(string $private_key, string $algo = 'sha512')
    {
        $private_key = trim($private_key);
        
        if (!preg_match('/^[a-z\d+\/]{43}=$/i', $private_key))
        {
            throw new Exception('Encryption: Unrecognized key.');
        }
        else if (!in_array(strtolower($algo), hash_algos()))
        {
            throw new Exception(sprintf('HMAC: Algo %s unsupported.', $algo));
        }
        
        $this->private_key = bin2hex($private_key);
        $this->algo        = $algo;
        $this->length      = strlen(hash($algo, null, true));
    }
    
    public function seal(string $message)
    {
        $hmac = hash_hmac(
            $this->algo,
            $message,
            $this->private_key,
            true
        );
        
        return base64_encode($hmac . $message);
    }
    
    public function sign(string $seal)
    {
        if (!preg_match('/^(?:[a-z\d+\/]+)(=|==)?$/i', $seal))
        {
            throw new Exception('HMAC: Unrecognized seal.');
        }

        $seal    = base64_decode($seal);
        $message = mb_substr($seal, $this->length, null, '8bit');
        $seal    = mb_substr($seal, 0, $this->length, '8bit');
        
        $signed = hash_hmac(
            $this->algo,
            $message,
            $this->private_key,
            true
        );

        if (!hash_equals($seal, $signed))
        {
            throw new Exception('HMAC: Seal corrupted.');
        }
        
        return true;
    }
}

$hmac_private_key       = 'ZZtJVgUu2fRz+c4o6QHj6v/mAqGAgyowlUxs3xoMHuw=';
$encryption_private_key = 'qB2fZkseI4ccJ45Y1/VzoHARA6Sft6IVkeS4r2Z+YYM=';

$encryption    = new Encryption($encryption_private_key);
$email_address = $encryption->encrypt('email@test.com');

$hmac  = new HMAC($hmac_private_key, 'sha256');
echo $seal = $hmac->seal($email_address);

var_dump($hmac->sign($seal));

could i get some more help on this please?

 

thank you

[Destramic]

ok i think i may have made some progress here after a lot of hard thinking and detemination...

 

i read about mysql sha2() and had a little play about with it

SELECT  SHA2('abc', 256)
> '936a185caaa266bb9cbe981e9e05cb78cd732b0b3280eb944412bb6f8f8f07af'

i stored the hash into my hashed column and ran this:

SELECT * FROM development.hash_test WHERE hashed = SHA2('abc', 256);

which brings up the correct row....so i though if i create a hmac and save it in a row it should work also...but no

 

i used the following and turned removed true on the raw parameter

 public function seal(string $message)
    {
        $hmac = hash_hmac(
            $this->algo, 
            $message, 
            $this->private_key
        );
        
        return $hmac;
    }

$hmac  = new HMAC($hmac_private_key, 'sha256');
echo $seal = $hmac->seal('helloworld');

which gave me a string like so:

1b3e0c20a197aa3bd20460dedc81033cac47581e7d8e1c0ba18872a3c5bfc4de

but it returns 0 rows when executing the following:

SELECT * FROM development.hash_test WHERE hashed = SHA2('helloworld', 256);

please tell me i'm close to what i'm trying to achieve and what it is i'm doing wrong?

 

thank you

[Jacques1]

A SHA-2 hash is not a SHA-2 HMAC. As I already said above, you simply create an extra column for the HMAC of the e-mail address and make it UNIQUE. Whenever you want to add a new address, you calculate the HMAC with PHP and try to insert the address together with the HMAC. If the address already exists, the HMAC will be identitical to an existing value and violate the UNIQUE constraint. Otherwise the query will succeed, and you know that the address is indeed unique.

 

It's really very simple once you understand the idea.

<?php



class HMAC
{
    /**
     * @var the underlying algorithm for the HMAC
     */
    const ALGO = 'sha256';

    /**
     * @var the length of the HMAC key in bytes
     */
    const KEY_LENGTH = 32;

    /**
     * @var string the HMAC key
     */
    protected $key;

    /**
     * HMAC constructor.
     *
     * @param string $key the HMAC key
     */
    public function __construct($key)
    {
        if (!is_string($key))
        {
            throw new InvalidArgumentException('Invalid key type. Expected a string.');
        }

        if (strlen($key) != static::KEY_LENGTH)
        {
            throw new InvalidArgumentException('Invalid key length. Expected '.static::KEY_LENGTH.' bytes.');
        }

        $this->key = $key;
    }

    /**
     * Calculates the HMAC of a message
     *
     * @param string $message the message
     *
     * @return string the resulting HMAC
     */
    public function calculate($message)
    {
        if (!is_string($message))
        {
            throw new InvalidArgumentException('Invalid message type. Expected a string.');
        }

        return hash_hmac(static::ALGO, $message, $this->key);
    }

    /**
     * Verifies the integrity of a message with a known HMAC.
     *
     * @param string $message the message to be verified
     * @param string $hmac    the known HMAC
     *
     * @return bool whether the HMAC of the message matches the known HMAC
     */
    public function verify($message, $hmac)
    {
        if (!is_string($message))
        {
            throw new InvalidArgumentException('Invalid message type. Expected a string.');
        }

        if (!is_string($hmac))
        {
            throw new InvalidArgumentException('Invalid HMAC type. Expected a string.');
        }

        return hash_equals($this->calculate($message), $hmac);
    }
}

<?php

$key = hex2bin('8311f033e7235286f5d9b17f2e83366989c3f12b414dc15348dbed2aaa102b2d');    // note binary key
$hMAC = new HMAC($key);

$email = 'foo@example.com';
$emailHMAC = $hMAC->calculate($email);
// insert $email with $emailHMAC

[Destramic]

i so over thought the whole process and the answer was right infront of me! ...sorry jacques.

 

i can see clearly how this works now

 

thank you for your time, patience and help