Sunday, April 28, 2019

Passing Data Using Static Classes

Typically, when I see questions on the Forums about passing data between Forms, the questions are usually about passing data in, what I will call, the "classic" sense: two Forms are open, one is used for gathering data and the other is used for manipulating and/or displaying data. For those types of questions, I refer the person to my blog posts from many years ago:

http://geek-goddess-bonnie.blogspot.com/2011/01/passing-data-between-forms.html
http://geek-goddess-bonnie.blogspot.com/2012/12/passing-data-between-forms-redux_31.html

And this one is often relevant too:

https://geek-goddess-bonnie.blogspot.com/2010/06/program-to-interface.html

I came across a question yesterday that at first seemed like it was going to be about passing data in the "classic" sense. Until I thought some more about what the person was actually asking about:  the "data-gathering" Form was a Login Form and it would collect information from the user, an Account Number for example. They then wanted to display that information on all other Forms in the application.  Since Login Forms are displayed at application start-up and then they "go away", we no longer have the ability to pass the data in the "classic" sense (two Forms are not going to both be open). Some other mechanism is needed.

This particular scenario with a Login Form, in my opinion, only requires a static class with static properties. For this application, I think that a singleton class would be overkill (because the class is only ever going to be populated at application startup and nowhere else or at any other time).

So, let's start with a static Account class:

public static class AccountClass
{
public static string AccountNumber { get; set; }
// plus, any other static info you'd like to share with your other Forms
}

In your LoginForm, perhaps after the user had clicked OK to login, you'd want to set the static members of the AccountClass:


private void OKButton_Click(object sender, EventArgs e)
{
AccountClass.AccountNumber = this.txtAccountNumber.Text;
// plus, whatever else you need to do in this button click
}

Then, in any other Forms where you need this information, get it from the static class. For example, to display the AccountNumber in a label on your Form, take care of that in the Form's constructor:

public CustomerForm()
{
InitializeComponent();
this.lblAccountNumber.Text = AccountClass.AccountNumber;
}

Happy Coding! =0)

Wednesday, February 27, 2019

Debugging Lambda / LINQ

Have you ever had a LINQ query or lambda expression and you couldn't for the life of you figure out why it wasn't working? To make debugging easier, you can actually set breakpoints on individual parts of the lambda expression. The query below is an example. Say that you're getting the error: System.FormatException: "Input string was not in a correct format."  Judging from that, you think it is probably the int.Parse that is causing the problem, but what is the value of the row that's having an issue? So, set a breakpoint on the .Parse (just select Parse and hit F9), and if you don't have a lot of rows you could easily find the row where the "amount" column contains a non-numeric value.

private void TestLambdaHowtoSetBreakpoint()
{
// Highlight one (or several) of the lambda parts of this LINQ query (such as the .Sum or the .Parse)
// and hit F9 to set breakpoints only on that part of the query! Awesome!
var sum = this.dtGlobal.AsEnumerable().Select(row => row)
.GroupBy(row => row.Field<string>("label"))
.ToDictionary(grp => grp.Key, intField => intField.Sum(row => int.Parse(row["amount"].ToString())));
}

Some people may be confused about the difference between lambda and LINQ. There is a good explanation in this StackOverflow thread (the second reply): https://stackoverflow.com/questions/34980144/difference-between-lambda-and-linq

I'll include some of that StackOveflow reply here:
----------------------------------------------------------------
This is LINQ:
var a = from b in someList
where b.Value == something
select b;

But it can be written with a lambda expression:
var a = someList.Where(b => b.Value == something);

The lambda part is b => b.Value == something.

Whereas:
mock.Setup(m => m.SomeOp()).Returns(new Thing()); 

uses a lambda expression (the m => m.SomeOp()), but has nothing to do with LINQ.

To make it easy, just remember that  => is the lambda operator.
----------------------------------------------------------------

That’s it! I hope you’ve learned something about debugging lambda expressions! Happy coding!  =0)

Friday, January 25, 2019

Encryption for Messaging Exchange

Perhaps you're a regular reader here on my blog, but perhaps not. Those of you who have read some of my posts might remember that our application is a home-grown message bus system that is multi-threaded and runs as a Windows Service. It basically receives, processes and sends messages between instances of the application running on other servers. All this is usually done on a local network or domain, but the messages could also, in theory, be sent out over the Internet. In any case, Encryption is obviously a necessity.

I have already written two blogs that are not about encryption, per se, but *do* have to do with the encryption process we use in our application.

http://geek-goddess-bonnie.blogspot.com/2016/04/transport-rsa-key-containers.html
http://geek-goddess-bonnie.blogspot.com/2015/01/compress-decompress.html

For encryption, we use AES Encryption and utilize an RSA KeyContainer (stored at the machine-level rather than the user-level) for encrypting the AES symmetric keys (which we store, encrypted, in a database table).

Here are some links explaining the two types of encryption that are involved in my solution:

http://thebestvpn.com/advanced-encryption-standard-aes/#cipher
http://www.boxcryptor.com/en/encryption/

This is quoted from the first article:

Like nearly all encryption algorithms, AES relies on the use of keys during the encryption and decryption process. Since the AES algorithm is symmetric, the same key is used for both encryption and decryption.

... symmetric algorithms require you to find a secure method of transferring the key to the desired recipient.

This is quoted from the second article:

RSA works with two different keys: A public and a private one. Both work complementary to each other, which means that a message encrypted with one of them can only be decrypted by its counterpart. Since the private key cannot be calculated from the public key, the latter is generally available to the public.

AES is an excellent encryption algorithm, and is theoretically "un-crackable". With the only "drawback" being that it uses symmetric keys (as the first article above explains). But, symmetric key algorithms are significantly faster than asymmetric algorithms and can easily encrypt/decrypt large messages.  RSA encryption is best for encrypting smaller messages, because it is slower. So a combination of the two is the best of both worlds: Generate your AES symmetric keys, Encrypt them with RSA and then you're able to send the RSA public keys anywhere.

This is the concept I will attempt to explain in this blog post. I have a downloadable solution for showing how this works. It uses two Windows Forms (AgencyOneMessageForm and AgencyTwoMessageForm) that "talk to" each other (by reading/writing files). This is only for ease of demonstrating the encrypt/decrypt process. Normally, the keys are stored in databases and the messaging between the multiple systems utilizes web service calls to send the messages (which are also stored in databases).

Download the demo code from here (GeekGoddessEncryptionDemo.zip)

The very first thing these demo Forms do is set up the RSA encryption. Note that if the KeyContainer doesn't already exist in the machine store (which, in case you're interested, is stored here: C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys), it is created, then retrieved again every time the following code is run. In this demo, the two Forms use a different KeyContainerName, thus generating different RSA keys. In real-life these application would be on different physical machines and so would have different RSA keys even when they use the same KeyContainerName.

// Make sure you have this using
// using System.Security.Cryptography;

// Crypto methods
protected void SetupCrypto()
{
    CspParameters cp = new CspParameters();
    cp.KeyContainerName = this.KeyContainerName;
    cp.Flags |= CspProviderFlags.UseMachineKeyStore;
    this.KeyContainer = new RSACryptoServiceProvider(cp);
    this.PublicEncryptionKey = this.KeyContainer.ToXmlString(false);
}

Next, we set up the AES encryption. For this demo, we'll designate AgencyOne as the "Central Scrutinizer" (for those who don't "get" that, it's a reference to a Frank Zappa album called Joe's Garage). Anyway, what this means is that AgencyOne is the Controller, it controls the creation of the AES keys. Once it creates (and stores) the AES keys, the other Agencies in the system can join the group by sending their own Public RSA key to the Central Scrutinizer, which uses their Public key to encrypt the AES keys and send them back. The receiving Agency will save the encrypted keys, as is, in a file (in real life, in a database).  That way, the same AES keys are used between all the Agencies in the group. Each Agency uses it's own RSA Private key to decrypt the encrypted keys that were just sent to it.

Here is what AgencyOne uses to create the AES encryption keys and store/retrieve from the AgencyOneKeys.xml file:

private void SetEncryptionKeys()
{
    // Normally, I retrieve (and save) the encrypted asymmetric AES keys from a database table.
    // But, for the purpose of this example (to make it easier to demonstrate), they are just being stored in an XML file.
    this.dsKeys = new DataSet();
    string fileName = this.AgencyName + "Keys.xml";
    if (File.Exists(fileName))
        this.dsKeys.ReadXml(fileName);
    else
    {
        this.dsKeys.DataSetName = "EncryptionDataSet";
        DataTable dtKeys = new DataTable();
        this.dsKeys.Tables.Add(dtKeys);

        dtKeys.TableName = "Encryption";
        dtKeys.Columns.Add("Agency");
        dtKeys.Columns.Add("EncryptionKey");
        dtKeys.Columns.Add("EncryptionIV");
        dtKeys.Rows.Add(dtKeys.NewRow());
    }
 
    DataRow row = this.dsKeys.Tables[0].Rows[0];
    string Key = row["EncryptionKey"].ToString();
    string IV = row["EncryptionIV"].ToString();

    if (Key != "" && IV != "")
    {
        // Remember, the Key/IV are stored RSA encrypted ... we have to decrypt them in order to use them
        EncryptionKey = this.KeyContainer.Decrypt(Convert.FromBase64String(Key), false);
        EncryptionIV = this.KeyContainer.Decrypt(Convert.FromBase64String(IV), false);
    }

    if (EncryptionKey == null || EncryptionIV == null)
    {
        // First time this is run, there won't be any keys set up. Create and save them now.
        this.Encrypter = new EncryptionClass();
        this.Encrypter.HashGuidsIntoEncryptionKeys(ref EncryptionKey, ref EncryptionIV);

        // encrypt with RSA public key, save to the file
        RSACryptoServiceProvider PublicRSA = new RSACryptoServiceProvider();
        PublicRSA.FromXmlString(PublicEncryptionKey);
        Key = Convert.ToBase64String(PublicRSA.Encrypt(EncryptionKey, false));
        IV = Convert.ToBase64String(PublicRSA.Encrypt(EncryptionIV, false));

        row["Agency"] = this.AgencyName;
        row["EncryptionKey"] = Key;
        row["EncryptionIV"] = IV;

        this.dsKeys.WriteXml(fileName);
    }
}
public void HashGuidsIntoEncryptionKeys(ref byte[] key, ref byte[] iv)
{
    string KeyGuid = Guid.NewGuid().ToString();
    string IVGuid = Guid.NewGuid().ToString();
    KeyGuid = KeyGuid.Replace("-", "");
    IVGuid = IVGuid.Replace("-", "");

    // Hash the GUID into byte arrays - Key is 32 bytes and IV is 16
    SHA256Managed Hasher = new SHA256Managed();
    Hasher.ComputeHash(Encoding.ASCII.GetBytes(KeyGuid));
    key = Hasher.Hash;

    Hasher.ComputeHash(Encoding.ASCII.GetBytes(IVGuid));
    byte[] result = Hasher.Hash;
    iv = new byte[16];
    for (int i = 0; i < 16; i++) iv[i] = result[i];
}

AgencyTwo also calls its own SetEncryptionKeys() method, which is slightly different than AgencyOne's because it doesn't have to create any AES keys, just decrypt the keys it has saved when it received them from the Central Scrutinizer:

private void SetEncryptionKeys()
{
    // Normally, I retrieve (and save) the encrypted asymmetric AES keys from a database table.
    // But, for the purpose of this example (to make it easier to demonstrate), they are just being stored in an XML file.
    this.dsKeys = new DataSet();
    string fileName = this.AgencyName + "Keys.xml";
    if (File.Exists(fileName))
        this.dsKeys.ReadXml(fileName);
    else
        return; // wait to join the Group

    DataRow row = this.dsKeys.Tables[0].Rows[0];
    string Key = row["EncryptionKey"].ToString();
    string IV = row["EncryptionIV"].ToString();

    if (Key != "" && IV != "")
    {
        // Remember, the Key/IV are stored RSA encrypted ... we have to decrypt them in order to use them
        EncryptionKey = this.KeyContainer.Decrypt(Convert.FromBase64String(Key), false);
        EncryptionIV = this.KeyContainer.Decrypt(Convert.FromBase64String(IV), false);
    }

    if (EncryptionKey == null || EncryptionIV == null)
        MessageBox.Show("Problem with " + fileName + ". Contains empty Encryption keys");
}

And, the last bit of code I will show here, the EncryptionClass:

public class EncryptionClass
{
    #region Declarations

    public byte[] EncryptionKey { get; set; }
    public byte[] EncryptionIV { get; set; }

    #endregion

    #region Constructors

    public EncryptionClass()
    {
    }
    public EncryptionClass(byte[] encryptionKey, byte[] encryptionIV)
    {
        this.EncryptionKey = encryptionKey;
        this.EncryptionIV = encryptionIV;
    }

    #endregion

    #region Public Methods

    public string Encrypt(string data)
    {
        return this.Encrypt(data, this.EncryptionKey, this.EncryptionIV);
    }
    public string Decrypt(string data)
    {
        return Encoding.UTF8.GetString(this.Decrypt(data, this.EncryptionKey, this.EncryptionIV));
    }
    public void HashGuidsIntoEncryptionKeys(ref byte[] key, ref byte[] iv)
    {
        string KeyGuid = Guid.NewGuid().ToString();
        string IVGuid = Guid.NewGuid().ToString();
        KeyGuid = KeyGuid.Replace("-", "");
        IVGuid = IVGuid.Replace("-", "");

        // Hash the GUID into byte arrays - Key is 32 bytes and IV is 16
        SHA256Managed Hasher = new SHA256Managed();
        Hasher.ComputeHash(Encoding.ASCII.GetBytes(KeyGuid));
        key = Hasher.Hash;

        Hasher.ComputeHash(Encoding.ASCII.GetBytes(IVGuid));
        byte[] result = Hasher.Hash;
        iv = new byte[16];
        for (int i = 0; i < 16; i++) iv[i] = result[i];
    }

    #endregion

    #region Private Methods

    private string Encrypt(string Text, byte[] key, byte[] iv)
    {
        byte[] TextBytes = Encoding.UTF8.GetBytes(Text);
        return this.Encrypt(TextBytes, key, iv);
    }
    private string Encrypt(byte[] TextBytes, byte[] key, byte[] iv)
    {
        try
        {
            AesCryptoServiceProvider RM = new AesCryptoServiceProvider();
            RM.Mode = CipherMode.CBC;
            ICryptoTransform encryptor = RM.CreateEncryptor(key, iv);

            // Create memory and crypto streams
            MemoryStream MStream = new MemoryStream();
            CryptoStream CStream = new CryptoStream(MStream, encryptor, CryptoStreamMode.Write);

            // Encrypt and put in byte array
            CStream.Write(TextBytes, 0, TextBytes.Length);
            CStream.FlushFinalBlock();
            byte[] CipherTextBytes = MStream.ToArray();

            // Close streams
            MStream.Close();
            CStream.Close();

            // Convert to Base64 string
            string CipherText = Convert.ToBase64String(CipherTextBytes);
            return CipherText;
        }
        catch (Exception ex)
        {
            throw new Exception("Error in Encrypt" + ex.Message);
        }
    }
    private byte[] Decrypt(string Text, byte[] key, byte[] iv)
    {
        try
        {
            if (string.IsNullOrEmpty(Text))
                Text = this.Encrypt("");

            AesCryptoServiceProvider RM = new AesCryptoServiceProvider();
            RM.Mode = CipherMode.CBC;
            ICryptoTransform decryptor = RM.CreateDecryptor(key, iv);

            // Convert string to byte array
            byte[] TextBytes = Convert.FromBase64String(Text);

            // Create memory and crypto streams
            MemoryStream MStream = new MemoryStream(TextBytes);
            CryptoStream CStream = new CryptoStream(MStream, decryptor, CryptoStreamMode.Read);

            // Decrypt to a byte array and close the streams
            byte[] DecryptedBytes = new byte[TextBytes.Length];
            int count = CStream.Read(DecryptedBytes, 0, DecryptedBytes.Length);
            MStream.Close();
            CStream.Close();

            // Create an array for return value and copy in the correct number of bytes
            byte[] returnval = new byte[count];
            Array.Copy(DecryptedBytes, returnval, count);
            return returnval;
        }
        catch (Exception ex)
        {
            throw new Exception("Error in Decrypt" + ex.Message);
        }
    }

    #endregion

}

When the app first starts, AgencyOne will create all its keys and store them in the AgencyOneKeys.xml. It will instantiate and Show() the AgencyTwo form, which will create its RSA keys and send the Public key to AgencyOne (which simply consists of saving a file called PublicKey.txt). You'll have to click on AgencyOne's "Receive Public Key" button in order to for it to get the file and send encrypted keys back to AgencyTwo (which simply consists of saving a file called EncryptedKeys.txt). After that, you'll have to click on AgencyTwo's "Receive Encrypted Keys" button in order to finish setting up AgencyTwo's keys (at which time, it will save a file AgencyTwoKeys.xml).  Those two buttons will disappear from the Forms, as they are not needed once this initial "hand-shaking" has been done. You'll notice that if you run the application again.

Messaging is done much the same way. If you type into the top box and click "Send To" (One or Two), it will encrypt and send the encrypted message to the other Form (saving a file called MessageSent.txt). The other Form gets the message by clicking on "Receive From" (One or Two) reading the MessageSent.txt file, decrypting it and displaying it in the bottom box. There are default messages in both Forms in case you've not entered anything in the top box.

That's it, in a nutshell. Download the Demo to see it in action, plus any other little details I may not have included in this post. Again, the important part of this blog post is the encryption concepts and how (and why) to use them in a real world scenario, such as our messaging system. The Demo code is just to illustrate those concepts and to provide you with the encryption code necessary to implement your own encryption. But, you'll primarily use this messaging exchange kind of scenario in a back end server, you’d never use Windows Forms for communication in such a manner in a real world application.

But, I'm sure you'll find other uses for the encryption methodology I've shared with you today!

Happy coding! =0)

Friday, December 28, 2018

Event-Driven TCP

Merry Christmas and/or Happy Holidays! Happy New Year too! Here's a present for you ... the long-awaited, easy-to-use set of TCP Client classes that I use in my own applications (maybe abbreviated a bit, but not by much)!

First, let me start off mentioning the best site that I've found for an "almost perfect" class to use for easily writing event-driven TCP clients. The site is called DaniWeb: http://www.daniweb.com/software-development/csharp/code/422291/user-friendly-asynchronous-event-driven-tcp-client I say "almost perfect", because it needed some tweaking for my purposes.

I had initially made an addition of one Timer, and called it tmrTestConnection. It turns out that it was all that was ever needed, none of the other Timers in the original DaniWeb EventDrivenTCPClient class are necessary and can actually cause problems (or at least the way I needed to use this class). I also added several variables: m_LastReported, m_LastReceived (DateTime) and TCPFailureInterval (configurable), all dealing with connectivity.

Here's how my test Client application uses TCP:

  • The Client connects to a TCP Server (I will provide a bare-bones Server example for your testing, in case you don't have a TCP Server handy).
  • The Server can either respond to the Client (once a Connection has been established):
    • by sending data (the Server retrieves some data from a File, which is part of the solution)
    • or do nothing until the Client sends a "Handshake" message, and then respond to the Client by sending the data.
  • That will, of course, depend on your application and the requirements of the Server that you are connecting.
    • In my test example here, I originally wrote it so that the Server would send data as soon as the Client connects.
    • But then decided to change it so that the Server waits for the "Handshake" message before sending any data.

And, there's connectivity error handling:

  • If the Server is not available when the Client first starts, the Client will continue to try to Connect, periodically logging Connect attempts at a configurable interval.
  • If the Server goes down, the Client continually tries to Connect again, periodically logging Connect attempts at a configurable interval (in the tmrTestConnection_Elapsed event handler).
  • While the Client is connected to the Server, if data is not received on a regular basis, it is logged periodically at a configurable interval (also in the tmrTestConnection_Elapsed).

There's too much code to post here in my blog. I'll post bits and pieces of it to illustrate the above points, but I've zipped up the two solutions and they are on my Google Drive: here’s the link for the Client and here’s the link for the Server . The Client zip file also includes the original code from DaniWeb (not added to the project, but the file DaniWeb.cs is with all the other files), but you should still take a look at the DaniWeb link that I posted at the start of this blog. It might be a good idea to download my zip files now and follow along:

The EventDrivenTCPClient is not supposed to be used directly. There is another class, TcpListener, that does the job of starting it up and hooking up a few events and then calling its Connect() method to get it all started. Here is the code for the StartListener() method in the TcpListener class:


protected virtual void StartListener()
{
    if (this.TestFromFiles)
    {
        this.GetDataFromFile();
        return;
    }
    this.TcpEvent = new EventDrivenTCPClient(IPAddress.Parse(this.ipAddress), this.Port);
    if (this.ReconnectInterval != 0)
        this.TcpEvent.ReconnectInterval = this.ReconnectInterval;
    this.TcpEvent.ConnectionStatusChanged += new EventDrivenTCPClient.delConnectionStatusChanged(TcpEvent_ConnectionStatusChanged);
    this.TcpEvent.DataReceived += new EventDrivenTCPClient.delDataReceived(TcpEvent_DataReceived);
    this.TcpEvent.Connect();
}

I'll explain the purpose of the TestFromFiles in a minute, because as you notice from the above code, it is not starting up anything! There is a method to my madness, however, and I will get to that soon.

So, if you're not testing from files, you'll receive the data from the Server in the TcpEvent_DataReceived() event handler. The TcpEvent (aka the EventDrivenTCPClient) fires the event whenever data is received from the TCP port. Now, here's where more work needs to be done, here in the TcpListener class ... because the data that is received from the TcpEvent, is in chunks based on the buffer size. We need to parse out that data into meaningful complete pieces of data.

In other words, the data received can be a partial set of data,  in which case we have to continue waiting to receive the rest of the set ...
Or it could contain the rest of the set and part of the next one, in which case, we'll do something with the complete set and wait to receive the rest of the next one ...
Or it could contain the rest of the set and complete multiple sets, in which case, we'll do something with the multiple complete sets.

I just so happen to have a BufferedDataParser class that will do just that (see the DataParser.cs file). The TcpEvent_DataReceived() event handler does only one thing: it calls the ProcessDataReceived() method, which makes use of the DataParser, like so:


protected void ProcessDataReceived(object data)
{
    if (this.DataParser != null)
    {
        List<string> DataList = this.DataParser.ProcessDataReceived(ref buffer, data.ToString());
        foreach (string Xml in DataList)
        {
            this.SendMessage(Xml);
            if (this.IsRunningFromConsole)
            {
                Console.WriteLine("******************* Begin Raw Data ***********************");
                Console.WriteLine(Xml);
                Console.WriteLine("*********************** End Raw  *************************");
                Console.WriteLine("Length: {0}", Xml.Length);
            }
        }
    }
    else
    {
        LogOutput.WriteLine("No DataParser!");
        if (this.IsRunningFromConsole)
            Console.WriteLine("Length: {0}", data.ToString().Length);
        this.SendMessage(data.ToString());
    }
}

Notice in the code above, the parameter "ref buffer" in the call to the DataParser.ProcessDataReceived(ref buffer, data.ToString()) method. That gives the continuity necessary for the DataParser to contain the entire partial set of data with every chunk of data received from the TCP port, until the Parser determines it has at least a full set of data (the current chunk of data may contain a partial set or multiple sets of data, hence the need for a List<string> returned from the Parser).

The BufferedDataParser is an abstract class, that is meant to be inherited by different kinds of Parsers that rely on receiving buffered data. I've included two different kinds of Parsers (one of which is demonstrated in the application). So, here's the abstract class:


public abstract class BufferedDataParser : IBufferedDataParser
{
    #region Declarations

    protected string HeartBeat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><ENDOFXML/>";
    protected string ValidStart = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>";
    protected string ValidEnd = "</AXmlTargets>";
    protected bool IncludeDelimiters = true;

    #endregion

    #region Methods

    public List<string> ProcessDataReceived(ref string buffer, string dataReceived)
    {
        List<string> DataList = new List<string>();
        try
        {
            //add received data to the buffer
            buffer += dataReceived;

            this.RemoveHeartbeatData(ref buffer);

            int whileLoopWatchdog = 0;
            while (buffer.Length > 0)
            {
                this.RemoveDataPrecedingStart(ref buffer);

                //verify that a complete message is present, if so process just that
                string Complete = this.ProcessCompleteMessage(ref buffer);
                if (Complete.IsNotNullOrEmpty())
                {
                    string Xml;
                    if (this.IncludeDelimiters)
                        Xml = Complete;
                    else
                        Xml = this.ParseOutDelimiters(Complete);

                    DataList.Add(Xml);

                    //remove the data that was pulled out for processing
                    //this will replace multiple identical occurrences but that
                    //is fine
                    buffer = buffer.Replace(Complete, "");
                }

                //HACK: prevent endless while loop
                //(although 20+ complete sets of data at once probably wouldn't
                //happen, and if it did the remainder would be caught on
                //next data receive)
                whileLoopWatchdog++;
                if (whileLoopWatchdog > 20)
                {
                    whileLoopWatchdog = 0;
                    break;
                }
            }
        }
        catch (Exception ex)
        {
            LogOutput.WriteLine(ex);
        }
        return DataList;
    }

    protected virtual void RemoveHeartbeatData(ref string buffer)
    {
        int whileLoopWatchdog = 0;

        //remove any "heartbeat" data
        while (buffer.Contains(HeartBeat))
        {
            //this will replace multiple identical occurrences but that
            //is fine
            buffer = buffer.Replace(HeartBeat, "");

            //HACK: prevent endless while loop
            //(although 100+ heartbeats should never happen)
            whileLoopWatchdog++;
            if (whileLoopWatchdog > 100)
            {
                break;
            }
        }
    }
    protected virtual void RemoveDataPrecedingStart(ref string buffer)
    {
        //discard any text that precedes the valid start
        if (buffer.IndexOf(ValidStart) > 0)
        {
            buffer = buffer.Substring(
            buffer.IndexOf(ValidStart));
        }
    }
    protected virtual string ProcessCompleteMessage(ref string buffer)
    {
        string Complete = "";

        if ((buffer.StartsWith(ValidStart)) && (buffer.Contains(ValidEnd)) && (buffer.IndexOf(ValidStart) < buffer.IndexOf(ValidEnd)))
        {
            Complete = buffer.Substring(buffer.IndexOf(ValidStart),
            buffer.IndexOf(ValidEnd) + ValidEnd.Length - buffer.IndexOf(ValidStart));
        }

        return Complete;
    }
    protected virtual string ParseOutDelimiters(string Complete)
    {
        return Complete.Substring(Complete.IndexOf(ValidStart) + ValidStart.Length, Complete.IndexOf(ValidEnd) - ValidStart.Length);
    }

    #endregion
}

Some DataParsers, simply need to use different delimiters and let the base class handle everything else, like the StxEtxDataParser:


public class StxEtxDataParser :BufferedDataParser
    {
    public StxEtxDataParser()
    {
        this.HeartBeat = "no heartbeat";
        this.ValidStart = new string(new char[] { (char)2 });
        this.ValidEnd = new string(new char[] { (char)3 });
        this.IncludeDelimiters = false;
    }
}

STX/ETX are common delimiters for TCP transmissions. STX is a (char)2 and ETX is a (char)3.

The DataParsers.cs file also contains a CountDelimiterDataProcessor. It inherits from the BufferedDataParser also, but has a totally different way of parsing the received data. I won't post it here, you can check it out when you look at the downloaded zip file.

Now, as promised, I'll get back to the purpose of TestFromFiles: You should have sample data from the Server you're trying to connect to saved in a file. You obviously have to have this, or how would you ever be able to develop an application without knowing something about the data!?! By using this saved file to test from, you can quickly determine if your DataParser is parsing the data correctly without worrying about connecting to a Server. If there's a problem, keep refactoring your DataParser (not the abstract class! That one should not be touched!) until it's perfect.

I think that's about it for now. Download the two zip files and give it a test yourself. It's good to go "right out of the box" ... you only have to change the ipaddress (and port if you want to) in the app.configs of both Client and Server.

Happy Coding!  =0)

Saturday, September 29, 2018

VS2017 - A Few Idiosyncrasies

I recently found some time to install the most recent version of Visual Studio 2017 and start using it for real work. I had actually had a few of the early pre-Release versions installed, but hadn't yet used them for anything other than "playing" (didn't want to use a pre-Release version for real work ... I'm not *that* much into "cutting edge"). I'd been so busy that I just continued using VS 2015 for work, even after the official release of VS2017.

What actually prompted me to install and start using VS2017 was that my laptop died. When I got the replacement, of course I had to re-install my entire development environment (not to mention a bunch of personal stuff not related to work). So, what the heck, might as well go with the latest and greatest!

I quickly found out that there were a few idiosyncrasies where things differed from my earlier VS 2015 version. I wanted to put some of it back to "my way":

  • Some shortcut keys aren't what they used to be. But, short-cut keys are easy to change:
    • Go to Tools > Options > Environment > Keyboard
    • Find the command shortcut you want to change and in the "Press shortcut keys" textbox, just do as it says ... press the keys.
    • Click OK when you're done
    • For example, Building a Solution has been shortcut F6 for so many, many years (even before Visual Studio was designed) that it's ingrained in my finger muscle-memory. F6 is a "complement" to F5  (for Build and Execute Debugging). However, it seems to have changed at some point (I don't know if it changed in this VS version or some previous one) to Ctrl+Shift+B. I just could *not* get used to that, so I changed it back to F6 and now I'm happy.
  • Outlining :
    • The Outlining shortcut keys work a little differently than they used to in previous versions of VS.
    • They all used to be specified as Ctrl+M, SecondKey ... but now it is Ctrl+M, Ctrl+SecondKey
      • Just keep holding the Ctrl down during both M and SecondKey, rather than making it two different key combinations as shown.
      • It works slightly different than in previous versions of VS, where either one of these worked:
        • continuously hold down the Ctrl key
        • or do Ctrl+M, let go of the Ctrl and type SecondKey
        • Now, if you try that second option, you get an error and it does nothing.
    • Collapse to Definitions
      • By default now, it does *NOT* Collapse the Regions! Arrgghh!! This particular "feature" is what prompted me to write this blog post, because it *really* irritated me!!!  But, not to worry, it can be fixed. Yay!
        • Tools > Options > Text Editors > C# > Advanced > Outlining > check "Collapse #regions when collapsing to definitions"
  • CodeLens is a great feature and it's been around since at least VS2015 (although I wasn't using it then). Be aware that it's not available on the VS2017 Community edition (only Professional and higher). You can read about it here:  https://docs.microsoft.com/en-us/visualstudio/ide/find-code-changes-and-other-history-with-codelens?view=vs-2017
    • But, here's the downside ... it puts its cool little goodies just where I DO NOT WANT THEM!!
    • Here's what my code looks like without CodeLens, nice and compact!
    • Here's what it looks like with CodeLens, the look of my code is now quite annoying (to me anyway):

    • But, you can disable the whole thing with an Options setting:
      • Tools > Options > Text Editors > All Languages > CodeLens > uncheck "Enable CodeLens"
    • I would really have preferred an option to change the position of where the CodeLens information is displayed, because as I said I believe some of what it does can be quite helpful. But, unfortunately, I don't think that there is anyway to do it, at least not that I know of.

Well, that's about it for now. So far, I'm glad it's a short list!!  Happy Coding!!  =0)

Sunday, August 19, 2018

ComboBox Gotcha #2

Recently, I participated in an MSDN Forum thread about problems with ComboBoxes, SelectedValue and SelectedIndex. It reminded me of a "gotcha" that I knew about a long time ago, but had forgotten about. I haven't been doing any development on Windows Forms apps in a long time (I have been doing only back-end server side stuff for more than a few years now), although many years ago I was responsible for a pretty extensive Windows Forms "Framework" for our developers to use for all of our company's applications. So, I knew about this then (and I'm sure that I built it into the Framework's base classes DataBinding methods). But, I digress ... let's get to a description of the problem and how to make sure it doesn't happen to you!

Say that in your application, you need to do something with the SelectedValue of a ComboBox. So, you create an event handler for either the .SelectedIndexChanged event or the .SelectedValueChanged event. But it doesn't work the way you expect it to and you're getting exceptions sometimes. What could be the problem? Let's look at the result of some debugging code placed in each of those event handlers to try and troubleshoot the problem:

Here's the debugging code:

private void cboCustomer_SelectedValueChanged(object sender, EventArgs e)
{
Console.WriteLine($"Value Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
Console.WriteLine($"Value Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
private void cboCustomer_SelectedIndexChanged(object sender, EventArgs e)
{
Console.WriteLine($"Index Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
Console.WriteLine($"Index Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}

And here are the results:

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is 01, is a System.String

This is obviously, not a good thing ... the events are firing way too many times. And why is the SelectedValue a DataRowView at first (in my example I am using a DataTable as the Combo's DataSource)?  In fact, these events should not be firing at all at this time ... we have simply set up the databinding for the ComboBox.  Wow! What did we do wrong?

Here's the databinding code (remember that this is BAD CODE!)

this.bsCustomers = new BindingSource();
this.bsCustomers.DataSource = this.oDataFromXML.Tables["Customer"];

this.cboCustomer.SelectedIndexChanged += cboCustomer_SelectedIndexChanged;
this.cboCustomer.SelectedValueChanged += cboCustomer_SelectedValueChanged;

this.cboCustomer.DataSource = this.bsCustomers;
this.cboCustomer.DisplayMember = "Last Name";
this.cboCustomer.ValueMember = "CustomerID";

The problem exists because the DataSource is set first. The subsequent statements setting the DisplayMember and ValueMember each fire the events (because now the SelectedValue Properties are changed by setting those Members).

There is an easy way to fix this, and perhaps you've already guessed it!  Set the DataSource *after* setting the DisplayMember and ValueMember.

this.bsCustomers = new BindingSource();
this.bsCustomers.DataSource = this.oDataFromXML.Tables["Customer"];

this.cboCustomer.SelectedIndexChanged += cboCustomer_SelectedIndexChanged;
this.cboCustomer.SelectedValueChanged += cboCustomer_SelectedValueChanged;

this.cboCustomer.DisplayMember = "Last Name";
this.cboCustomer.ValueMember = "CustomerID";
this.cboCustomer.DataSource = this.bsCustomers;

But wait ... what's going on? Now the code in the event handlers throws an exception!

Value Changed: SelectedIndex is -1
Exception thrown: 'System.NullReferenceException' in WindowsApplication1.exe

That's because, now that we've done the databinding *correctly*, initially nothing is selected (this only happens once) and consequently the SelectedIndex is -1 and the SelectedValue will be null.  You *must* have code to check that SelectedIndex is not negative! Especially if you databind the ComboBox to a DataTable that initially has no data in it! When there *is* data, the first item will then be automatically selected and the SelectedIndex will be 0 ... but if there is *no* data, obviously nothing can be selected. The SelectedIndex will be -1 to indicate that.

private void cboCustomer_SelectedValueChanged(object sender, EventArgs e)
{
Console.WriteLine($"Value Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
if (cboCustomer.SelectedIndex > -1)
{
Console.WriteLine($"Value Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
}
private void cboCustomer_SelectedIndexChanged(object sender, EventArgs e)
{
Console.WriteLine($"Index Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
if (cboCustomer.SelectedIndex > -1)
{
Console.WriteLine($"Index Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
}

And the results now:

Value Changed: SelectedIndex is -1
Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is 01, is a System.String

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is 01, is a System.String

In a similar vein, I wrote a blog post back in 2012 with the title "ComboBox Gotchas". But, it was about a totally different problem ... a ComboBox that is in sync with other controls on the Form, when it's not supposed to be. See it here, if you're interested: https://geek-goddess-bonnie.blogspot.com/2012/12/combobox-gotchas.html


Sunday, May 27, 2018

(A)Synchronicity

Back in the day, "The Police" sang a song about Synchronicity. Today, I'm writing a blog about Asynchronicity.  ;0)

To be more precise, this blog post will explore whether or not there are performance differences between the various ways to write asynchronous code. I will be looking specifically at the code needed for a multi-threaded application, where the threads just do their thing, and we don't care what they're doing (in other words, kind of a Fire-And-Forget type of thing). I will explore these three methodologies:

  • System.Threading.Thread.Start()
  • System.Threading.ThreadPool.QueueUserWorkItem()
  • System.Threading.Tasks.Task.Run()

I'm betting that the latter two methods are simply syntactic sugar on top of the original Thread.Start() ... let's see if we can get some numerical evidence of that. And, just for the heck of it, I'm going to throw in a freebie ... I'm going to compare the Task.Run() both without and with a .ConfigureAwait(false) ... (not that it matters for this discussion). So, here's the code I used for testing this (called from a Button Click so I could easily run it again):

private System.Diagnostics.Stopwatch stopWatchThread;
private System.Diagnostics.Stopwatch stopWatchQueueUserWork;
private System.Diagnostics.Stopwatch stopWatchTask;
private System.Diagnostics.Stopwatch stopWatchTaskAwaitFalse; private void CompareThreeAsyncMethods()
{
this.stopWatchThread = new Stopwatch();
Console.WriteLine("Start Thread at {0}", DateTime.Now);
this.stopWatchThread.Start();
ThreadStart threadStart = new ThreadStart(RunThread);
Thread myThread = new Thread(threadStart);
myThread.Start(); this.stopWatchQueueUserWork = new Stopwatch();
Console.WriteLine("Start QueueUserWork at {0}", DateTime.Now);
stopWatchQueueUserWork.Start();
ThreadPool.QueueUserWorkItem(WorkToDo, new WorkSettings() { stopWatch = stopWatchQueueUserWork, WorkName = "QueueUserWork" }); this.stopWatchTask = new Stopwatch();
Console.WriteLine("Start Task at {0}", DateTime.Now);
stopWatchTask.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() {stopWatch = stopWatchTask, WorkName = "Task" });
}); this.stopWatchTaskAwaitFalse = new Stopwatch();
Console.WriteLine("Start Task AwaitFalse at {0}", DateTime.Now);
stopWatchTaskAwaitFalse.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTaskAwaitFalse, WorkName = "TaskAwaitFalse" });
}).ConfigureAwait(false); Console.WriteLine("Show results as threads complete ... {0}\r\n", DateTime.Now);
}
public class WorkSettings
{
public System.Diagnostics.Stopwatch stopWatch { get; set; }
public string WorkName { get; set; }
}
// Need this only for the Thread option
private void RunThread()
{
this.WorkToDo(new WorkSettings() { stopWatch = this.stopWatchThread, WorkName = "Thread" });
} private void WorkToDo(object obj)
{
WorkSettings work = (WorkSettings)obj;
for (int i = 0; i < 2147483647; i++) // largest int 2,147,483,647
{
//Thread.Sleep(500);
}

work.stopWatch.Stop();
Console.WriteLine("{0} Stopped at {1}. Elapsed Time {2} milliseconds\r\n", work.WorkName, DateTime.Now, work.stopWatch.ElapsedMilliseconds);
}

By starting a StopWatch before I start running its Asynchronous method, I take into account any "prep" work that might be involved in starting the call to the method. If there is any, it should be reflected in the overall time to complete a method. Here are the results:

Start Thread at 5/26/2018 5:13:38 PM
Start QueueUserWork at 5/26/2018 5:13:38 PM
Start Task at 5/26/2018 5:13:38 PM
Start Task AwaitFalse at 5/26/2018 5:13:38 PM
Show results as threads complete ... 5/26/2018 5:13:38 PM Thread Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5669 milliseconds
Task Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5775 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5836 milliseconds
QueueUserWork Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5859 milliseconds // last (10th) time
Thread Stopped at 5/26/2018 5:18:14 PM. Elapsed Time 5890 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 5978 milliseconds
QueueUserWork Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 5982 milliseconds
Task Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 6008 milliseconds

// Average
Average for Thread: 5986 milliseconds
Average for Task: 5964 milliseconds
Average for TaskAwaitFalse: 5973 milliseconds
Average for QueueUserWork: 6048 milliseconds

Judging by those results, I think that the minuscule difference might be attributed to them all running at once & taking up CPU resources. But, we can test that as well. I'm going to run them individually. Run one. Let it complete. Run another. Let it complete. I wonder if the times will be *exact* then! Let's re-work the code to look like this:

private CompareAsync methodIs = CompareAsync.Begin;
private CompareAsync currentMethodRunning = CompareAsync.Begin;
private void CompareThreeAsyncMethodsIndividually()
{
while (methodIs != CompareAsync.Done)
{
if (methodIs == CompareAsync.Begin)
methodIs = CompareAsync.Thread;
if (methodIs == currentMethodRunning)
continue;
switch (methodIs)
{
case CompareAsync.Thread:
currentMethodRunning = methodIs;
this.stopWatchThread = new Stopwatch();
Console.WriteLine("Start Thread at {0}", DateTime.Now);
this.stopWatchThread.Start();
ThreadStart threadStart = new ThreadStart(RunThread);
Thread myThread = new Thread(threadStart);
myThread.Start();
break;
case CompareAsync.QueueUserWork:
currentMethodRunning = methodIs;
this.stopWatchQueueUserWork = new Stopwatch();
Console.WriteLine("Start QueueUserWork at {0}", DateTime.Now);
stopWatchQueueUserWork.Start();
ThreadPool.QueueUserWorkItem(WorkToDo, new WorkSettings() { stopWatch = stopWatchQueueUserWork, WorkName = "QueueUserWork" });
break;
case CompareAsync.Task:
currentMethodRunning = methodIs;
this.stopWatchTask = new Stopwatch();
Console.WriteLine("Start Task at {0}", DateTime.Now);
stopWatchTask.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTask, WorkName = "Task" });
});
break;
case CompareAsync.TaskAwaitFalse:
currentMethodRunning = methodIs;
this.stopWatchTaskAwaitFalse = new Stopwatch();
Console.WriteLine("Start Task AwaitFalse at {0}", DateTime.Now);
stopWatchTaskAwaitFalse.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTaskAwaitFalse, WorkName = "TaskAwaitFalse" });
}).ConfigureAwait(false);
break;
default:
break;
}
}
}


And added an extra bit at the end of the WorkToDo() method such that it would be able to determine which method would run next:

private void WorkToDo(object obj)
{
WorkSettings work = (WorkSettings)obj;
for (int i = 0; i < 2147483647; i++) // largest int 2,147,483,647
{
//Thread.Sleep(500);
}
work.stopWatch.Stop();
Console.WriteLine("{0} Stopped at {1}. Elapsed Time {2} milliseconds\r\n", work.WorkName, DateTime.Now, work.stopWatch.ElapsedMilliseconds);

// ADDED THIS
// Set it to the next one
switch (methodIs)
{
case CompareAsync.Thread:
methodIs = CompareAsync.QueueUserWork;
break;
case CompareAsync.QueueUserWork:
methodIs = CompareAsync.Task;
break;
case CompareAsync.Task:
methodIs = CompareAsync.TaskAwaitFalse;
break;
case CompareAsync.TaskAwaitFalse:
methodIs = CompareAsync.Done;
break;
}
}

And now the results are this:

Start Thread at 5/26/2018 5:29:32 PM
Thread Stopped at 5/26/2018 5:29:37 PM. Elapsed Time 4877 milliseconds

Start QueueUserWork at 5/26/2018 5:29:37 PM
QueueUserWork Stopped at 5/26/2018 5:29:42 PM. Elapsed Time 4850 milliseconds

Start Task at 5/26/2018 5:29:42 PM
Task Stopped at 5/26/2018 5:29:47 PM. Elapsed Time 4918 milliseconds

Start Task AwaitFalse at 5/26/2018 5:29:47 PM
TaskAwaitFalse Stopped at 5/26/2018 5:29:52 PM. Elapsed Time 4978 milliseconds

// last time (10th)
Thread Stopped at 5/26/2018 5:32:37 PM. Elapsed Time 4910 milliseconds
QueueUserWork Stopped at 5/26/2018 5:32:42 PM. Elapsed Time 4821 milliseconds
Task Stopped at 5/26/2018 5:32:46 PM. Elapsed Time 4894 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:32:51 PM. Elapsed Time 4924 milliseconds

// Average
Average for Thread: 4986 milliseconds
Average for QueueUserWork: 4975 milliseconds
Average for Task: 4966 milliseconds
Average for TaskAwaitFalse: 4962 milliseconds

Well now ... there's still a difference between the 4 methods, isn't there? However, it is still teeny. Running each of these individually (instead of starting all 4 methods at the same time, as I did in the first test) produces the same results pretty much, the difference being that each is about 1000 milliseconds faster than the first test.

The difference between the fastest & slowest averages for the second test is only about .2% and that's in milliseconds! I'd say that these various methodologies for running asynchronous threads, aren't really any different (statistically speaking). Hey, I just like to mess around with numbers.  ;0)

Happy coding!  =0)