Encryption for Messaging Exchange

Perhaps you're a regular reader here on my blog, but perhaps not. Those of you who have read some of my posts might remember that our application is a home-grown message bus system that is multi-threaded and runs as a Windows Service. It basically receives, processes and sends messages between instances of the application running on other servers. All this is usually done on a local network or domain, but the messages could also, in theory, be sent out over the Internet. In any case, Encryption is obviously a necessity.

I have already written two blogs that are not about encryption, per se, but *do* have to do with the encryption process we use in our application.

http://geek-goddess-bonnie.blogspot.com/2016/04/transport-rsa-key-containers.html
http://geek-goddess-bonnie.blogspot.com/2015/01/compress-decompress.html

For encryption, we use AES Encryption and utilize an RSA KeyContainer (stored at the machine-level rather than the user-level) for encrypting the AES symmetric keys (which we store, encrypted, in a database table).

Here are some links explaining the two types of encryption that are involved in my solution:

http://thebestvpn.com/advanced-encryption-standard-aes/#cipher
http://www.boxcryptor.com/en/encryption/

This is quoted from the first article:

Like nearly all encryption algorithms, AES relies on the use of keys during the encryption and decryption process. Since the AES algorithm is symmetric, the same key is used for both encryption and decryption.

... symmetric algorithms require you to find a secure method of transferring the key to the desired recipient.

This is quoted from the second article:

RSA works with two different keys: A public and a private one. Both work complementary to each other, which means that a message encrypted with one of them can only be decrypted by its counterpart. Since the private key cannot be calculated from the public key, the latter is generally available to the public.

AES is an excellent encryption algorithm, and is theoretically "un-crackable". With the only "drawback" being that it uses symmetric keys (as the first article above explains). But, symmetric key algorithms are significantly faster than asymmetric algorithms and can easily encrypt/decrypt large messages.  RSA encryption is best for encrypting smaller messages, because it is slower. So a combination of the two is the best of both worlds: Generate your AES symmetric keys, Encrypt them with RSA and then you're able to send the RSA public keys anywhere.

This is the concept I will attempt to explain in this blog post. I have a downloadable solution for showing how this works. It uses two Windows Forms (AgencyOneMessageForm and AgencyTwoMessageForm) that "talk to" each other (by reading/writing files). This is only for ease of demonstrating the encrypt/decrypt process. Normally, the keys are stored in databases and the messaging between the multiple systems utilizes web service calls to send the messages (which are also stored in databases).

Download the demo code from here (GeekGoddessEncryptionDemo.zip)

The very first thing these demo Forms do is set up the RSA encryption. Note that if the KeyContainer doesn't already exist in the machine store (which, in case you're interested, is stored here: C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys), it is created, then retrieved again every time the following code is run. In this demo, the two Forms use a different KeyContainerName, thus generating different RSA keys. In real-life these application would be on different physical machines and so would have different RSA keys even when they use the same KeyContainerName.

// Make sure you have this using
// using System.Security.Cryptography;

// Crypto methods
protected void SetupCrypto()
{
    CspParameters cp = new CspParameters();
    cp.KeyContainerName = this.KeyContainerName;
    cp.Flags |= CspProviderFlags.UseMachineKeyStore;
    this.KeyContainer = new RSACryptoServiceProvider(cp);
    this.PublicEncryptionKey = this.KeyContainer.ToXmlString(false);
}

Next, we set up the AES encryption. For this demo, we'll designate AgencyOne as the "Central Scrutinizer" (for those who don't "get" that, it's a reference to a Frank Zappa album called Joe's Garage). Anyway, what this means is that AgencyOne is the Controller, it controls the creation of the AES keys. Once it creates (and stores) the AES keys, the other Agencies in the system can join the group by sending their own Public RSA key to the Central Scrutinizer, which uses their Public key to encrypt the AES keys and send them back. The receiving Agency will save the encrypted keys, as is, in a file (in real life, in a database).  That way, the same AES keys are used between all the Agencies in the group. Each Agency uses it's own RSA Private key to decrypt the encrypted keys that were just sent to it.

Here is what AgencyOne uses to create the AES encryption keys and store/retrieve from the AgencyOneKeys.xml file:

private void SetEncryptionKeys()
{
    // Normally, I retrieve (and save) the encrypted asymmetric AES keys from a database table.
    // But, for the purpose of this example (to make it easier to demonstrate), they are just being stored in an XML file.
    this.dsKeys = new DataSet();
    string fileName = this.AgencyName + "Keys.xml";
    if (File.Exists(fileName))
        this.dsKeys.ReadXml(fileName);
    else
    {
        this.dsKeys.DataSetName = "EncryptionDataSet";
        DataTable dtKeys = new DataTable();
        this.dsKeys.Tables.Add(dtKeys);

        dtKeys.TableName = "Encryption";
        dtKeys.Columns.Add("Agency");
        dtKeys.Columns.Add("EncryptionKey");
        dtKeys.Columns.Add("EncryptionIV");
        dtKeys.Rows.Add(dtKeys.NewRow());
    }
 
    DataRow row = this.dsKeys.Tables[0].Rows[0];
    string Key = row["EncryptionKey"].ToString();
    string IV = row["EncryptionIV"].ToString();

    if (Key != "" && IV != "")
    {
        // Remember, the Key/IV are stored RSA encrypted ... we have to decrypt them in order to use them
        EncryptionKey = this.KeyContainer.Decrypt(Convert.FromBase64String(Key), false);
        EncryptionIV = this.KeyContainer.Decrypt(Convert.FromBase64String(IV), false);
    }

    if (EncryptionKey == null || EncryptionIV == null)
    {
        // First time this is run, there won't be any keys set up. Create and save them now.
        this.Encrypter = new EncryptionClass();
        this.Encrypter.HashGuidsIntoEncryptionKeys(ref EncryptionKey, ref EncryptionIV);

        // encrypt with RSA public key, save to the file
        RSACryptoServiceProvider PublicRSA = new RSACryptoServiceProvider();
        PublicRSA.FromXmlString(PublicEncryptionKey);
        Key = Convert.ToBase64String(PublicRSA.Encrypt(EncryptionKey, false));
        IV = Convert.ToBase64String(PublicRSA.Encrypt(EncryptionIV, false));

        row["Agency"] = this.AgencyName;
        row["EncryptionKey"] = Key;
        row["EncryptionIV"] = IV;

        this.dsKeys.WriteXml(fileName);
    }
}
public void HashGuidsIntoEncryptionKeys(ref byte[] key, ref byte[] iv)
{
    string KeyGuid = Guid.NewGuid().ToString();
    string IVGuid = Guid.NewGuid().ToString();
    KeyGuid = KeyGuid.Replace("-", "");
    IVGuid = IVGuid.Replace("-", "");

    // Hash the GUID into byte arrays - Key is 32 bytes and IV is 16
    SHA256Managed Hasher = new SHA256Managed();
    Hasher.ComputeHash(Encoding.ASCII.GetBytes(KeyGuid));
    key = Hasher.Hash;

    Hasher.ComputeHash(Encoding.ASCII.GetBytes(IVGuid));
    byte[] result = Hasher.Hash;
    iv = new byte[16];
    for (int i = 0; i < 16; i++) iv[i] = result[i];
}

AgencyTwo also calls its own SetEncryptionKeys() method, which is slightly different than AgencyOne's because it doesn't have to create any AES keys, just decrypt the keys it has saved when it received them from the Central Scrutinizer:

private void SetEncryptionKeys()
{
    // Normally, I retrieve (and save) the encrypted asymmetric AES keys from a database table.
    // But, for the purpose of this example (to make it easier to demonstrate), they are just being stored in an XML file.
    this.dsKeys = new DataSet();
    string fileName = this.AgencyName + "Keys.xml";
    if (File.Exists(fileName))
        this.dsKeys.ReadXml(fileName);
    else
        return; // wait to join the Group

    DataRow row = this.dsKeys.Tables[0].Rows[0];
    string Key = row["EncryptionKey"].ToString();
    string IV = row["EncryptionIV"].ToString();

    if (Key != "" && IV != "")
    {
        // Remember, the Key/IV are stored RSA encrypted ... we have to decrypt them in order to use them
        EncryptionKey = this.KeyContainer.Decrypt(Convert.FromBase64String(Key), false);
        EncryptionIV = this.KeyContainer.Decrypt(Convert.FromBase64String(IV), false);
    }

    if (EncryptionKey == null || EncryptionIV == null)
        MessageBox.Show("Problem with " + fileName + ". Contains empty Encryption keys");
}

And, the last bit of code I will show here, the EncryptionClass:

public class EncryptionClass
{
    #region Declarations

    public byte[] EncryptionKey { get; set; }
    public byte[] EncryptionIV { get; set; }

    #endregion

    #region Constructors

    public EncryptionClass()
    {
    }
    public EncryptionClass(byte[] encryptionKey, byte[] encryptionIV)
    {
        this.EncryptionKey = encryptionKey;
        this.EncryptionIV = encryptionIV;
    }

    #endregion

    #region Public Methods

    public string Encrypt(string data)
    {
        return this.Encrypt(data, this.EncryptionKey, this.EncryptionIV);
    }
    public string Decrypt(string data)
    {
        return Encoding.UTF8.GetString(this.Decrypt(data, this.EncryptionKey, this.EncryptionIV));
    }
    public void HashGuidsIntoEncryptionKeys(ref byte[] key, ref byte[] iv)
    {
        string KeyGuid = Guid.NewGuid().ToString();
        string IVGuid = Guid.NewGuid().ToString();
        KeyGuid = KeyGuid.Replace("-", "");
        IVGuid = IVGuid.Replace("-", "");

        // Hash the GUID into byte arrays - Key is 32 bytes and IV is 16
        SHA256Managed Hasher = new SHA256Managed();
        Hasher.ComputeHash(Encoding.ASCII.GetBytes(KeyGuid));
        key = Hasher.Hash;

        Hasher.ComputeHash(Encoding.ASCII.GetBytes(IVGuid));
        byte[] result = Hasher.Hash;
        iv = new byte[16];
        for (int i = 0; i < 16; i++) iv[i] = result[i];
    }

    #endregion

    #region Private Methods

    private string Encrypt(string Text, byte[] key, byte[] iv)
    {
        byte[] TextBytes = Encoding.UTF8.GetBytes(Text);
        return this.Encrypt(TextBytes, key, iv);
    }
    private string Encrypt(byte[] TextBytes, byte[] key, byte[] iv)
    {
        try
        {
            AesCryptoServiceProvider RM = new AesCryptoServiceProvider();
            RM.Mode = CipherMode.CBC;
            ICryptoTransform encryptor = RM.CreateEncryptor(key, iv);

            // Create memory and crypto streams
            MemoryStream MStream = new MemoryStream();
            CryptoStream CStream = new CryptoStream(MStream, encryptor, CryptoStreamMode.Write);

            // Encrypt and put in byte array
            CStream.Write(TextBytes, 0, TextBytes.Length);
            CStream.FlushFinalBlock();
            byte[] CipherTextBytes = MStream.ToArray();

            // Close streams
            MStream.Close();
            CStream.Close();

            // Convert to Base64 string
            string CipherText = Convert.ToBase64String(CipherTextBytes);
            return CipherText;
        }
        catch (Exception ex)
        {
            throw new Exception("Error in Encrypt" + ex.Message);
        }
    }
    private byte[] Decrypt(string Text, byte[] key, byte[] iv)
    {
        try
        {
            if (string.IsNullOrEmpty(Text))
                Text = this.Encrypt("");

            AesCryptoServiceProvider RM = new AesCryptoServiceProvider();
            RM.Mode = CipherMode.CBC;
            ICryptoTransform decryptor = RM.CreateDecryptor(key, iv);

            // Convert string to byte array
            byte[] TextBytes = Convert.FromBase64String(Text);

            // Create memory and crypto streams
            MemoryStream MStream = new MemoryStream(TextBytes);
            CryptoStream CStream = new CryptoStream(MStream, decryptor, CryptoStreamMode.Read);

            // Decrypt to a byte array and close the streams
            byte[] DecryptedBytes = new byte[TextBytes.Length];
            int count = CStream.Read(DecryptedBytes, 0, DecryptedBytes.Length);
            MStream.Close();
            CStream.Close();

            // Create an array for return value and copy in the correct number of bytes
            byte[] returnval = new byte[count];
            Array.Copy(DecryptedBytes, returnval, count);
            return returnval;
        }
        catch (Exception ex)
        {
            throw new Exception("Error in Decrypt" + ex.Message);
        }
    }

    #endregion

}

When the app first starts, AgencyOne will create all its keys and store them in the AgencyOneKeys.xml. It will instantiate and Show() the AgencyTwo form, which will create its RSA keys and send the Public key to AgencyOne (which simply consists of saving a file called PublicKey.txt). You'll have to click on AgencyOne's "Receive Public Key" button in order to for it to get the file and send encrypted keys back to AgencyTwo (which simply consists of saving a file called EncryptedKeys.txt). After that, you'll have to click on AgencyTwo's "Receive Encrypted Keys" button in order to finish setting up AgencyTwo's keys (at which time, it will save a file AgencyTwoKeys.xml).  Those two buttons will disappear from the Forms, as they are not needed once this initial "hand-shaking" has been done. You'll notice that if you run the application again.

Messaging is done much the same way. If you type into the top box and click "Send To" (One or Two), it will encrypt and send the encrypted message to the other Form (saving a file called MessageSent.txt). The other Form gets the message by clicking on "Receive From" (One or Two) reading the MessageSent.txt file, decrypting it and displaying it in the bottom box. There are default messages in both Forms in case you've not entered anything in the top box.

That's it, in a nutshell. Download the Demo to see it in action, plus any other little details I may not have included in this post. Again, the important part of this blog post is the encryption concepts and how (and why) to use them in a real world scenario, such as our messaging system. The Demo code is just to illustrate those concepts and to provide you with the encryption code necessary to implement your own encryption. But, you'll primarily use this messaging exchange kind of scenario in a back end server, you’d never use Windows Forms for communication in such a manner in a real world application.

But, I'm sure you'll find other uses for the encryption methodology I've shared with you today!

Happy coding! =0)