Friday, December 28, 2018

Event-Driven TCP

Merry Christmas and/or Happy Holidays! Happy New Year too! Here's a present for you ... the long-awaited, easy-to-use set of TCP Client classes that I use in my own applications (maybe abbreviated a bit, but not by much)!

First, let me start off mentioning the best site that I've found for an "almost perfect" class to use for easily writing event-driven TCP clients. The site is called DaniWeb: http://www.daniweb.com/software-development/csharp/code/422291/user-friendly-asynchronous-event-driven-tcp-client I say "almost perfect", because it needed some tweaking for my purposes.

I had initially made an addition of one Timer, and called it tmrTestConnection. It turns out that it was all that was ever needed, none of the other Timers in the original DaniWeb EventDrivenTCPClient class are necessary and can actually cause problems (or at least the way I needed to use this class). I also added several variables: m_LastReported, m_LastReceived (DateTime) and TCPFailureInterval (configurable), all dealing with connectivity.

Here's how my test Client application uses TCP:

  • The Client connects to a TCP Server (I will provide a bare-bones Server example for your testing, in case you don't have a TCP Server handy).
  • The Server can either respond to the Client (once a Connection has been established):
    • by sending data (the Server retrieves some data from a File, which is part of the solution)
    • or do nothing until the Client sends a "Handshake" message, and then respond to the Client by sending the data.
  • That will, of course, depend on your application and the requirements of the Server that you are connecting.
    • In my test example here, I originally wrote it so that the Server would send data as soon as the Client connects.
    • But then decided to change it so that the Server waits for the "Handshake" message before sending any data.

And, there's connectivity error handling:

  • If the Server is not available when the Client first starts, the Client will continue to try to Connect, periodically logging Connect attempts at a configurable interval.
  • If the Server goes down, the Client continually tries to Connect again, periodically logging Connect attempts at a configurable interval (in the tmrTestConnection_Elapsed event handler).
  • While the Client is connected to the Server, if data is not received on a regular basis, it is logged periodically at a configurable interval (also in the tmrTestConnection_Elapsed).

There's too much code to post here in my blog. I'll post bits and pieces of it to illustrate the above points, but I've zipped up the two solutions and they are on my Google Drive: here’s the link for the Client and here’s the link for the Server . The Client zip file also includes the original code from DaniWeb (not added to the project, but the file DaniWeb.cs is with all the other files), but you should still take a look at the DaniWeb link that I posted at the start of this blog. It might be a good idea to download my zip files now and follow along:

The EventDrivenTCPClient is not supposed to be used directly. There is another class, TcpListener, that does the job of starting it up and hooking up a few events and then calling its Connect() method to get it all started. Here is the code for the StartListener() method in the TcpListener class:


protected virtual void StartListener()
{
    if (this.TestFromFiles)
    {
        this.GetDataFromFile();
        return;
    }
    this.TcpEvent = new EventDrivenTCPClient(IPAddress.Parse(this.ipAddress), this.Port);
    if (this.ReconnectInterval != 0)
        this.TcpEvent.ReconnectInterval = this.ReconnectInterval;
    this.TcpEvent.ConnectionStatusChanged += new EventDrivenTCPClient.delConnectionStatusChanged(TcpEvent_ConnectionStatusChanged);
    this.TcpEvent.DataReceived += new EventDrivenTCPClient.delDataReceived(TcpEvent_DataReceived);
    this.TcpEvent.Connect();
}

I'll explain the purpose of the TestFromFiles in a minute, because as you notice from the above code, it is not starting up anything! There is a method to my madness, however, and I will get to that soon.

So, if you're not testing from files, you'll receive the data from the Server in the TcpEvent_DataReceived() event handler. The TcpEvent (aka the EventDrivenTCPClient) fires the event whenever data is received from the TCP port. Now, here's where more work needs to be done, here in the TcpListener class ... because the data that is received from the TcpEvent, is in chunks based on the buffer size. We need to parse out that data into meaningful complete pieces of data.

In other words, the data received can be a partial set of data,  in which case we have to continue waiting to receive the rest of the set ...
Or it could contain the rest of the set and part of the next one, in which case, we'll do something with the complete set and wait to receive the rest of the next one ...
Or it could contain the rest of the set and complete multiple sets, in which case, we'll do something with the multiple complete sets.

I just so happen to have a BufferedDataParser class that will do just that (see the DataParser.cs file). The TcpEvent_DataReceived() event handler does only one thing: it calls the ProcessDataReceived() method, which makes use of the DataParser, like so:


protected void ProcessDataReceived(object data)
{
    if (this.DataParser != null)
    {
        List<string> DataList = this.DataParser.ProcessDataReceived(ref buffer, data.ToString());
        foreach (string Xml in DataList)
        {
            this.SendMessage(Xml);
            if (this.IsRunningFromConsole)
            {
                Console.WriteLine("******************* Begin Raw Data ***********************");
                Console.WriteLine(Xml);
                Console.WriteLine("*********************** End Raw  *************************");
                Console.WriteLine("Length: {0}", Xml.Length);
            }
        }
    }
    else
    {
        LogOutput.WriteLine("No DataParser!");
        if (this.IsRunningFromConsole)
            Console.WriteLine("Length: {0}", data.ToString().Length);
        this.SendMessage(data.ToString());
    }
}

Notice in the code above, the parameter "ref buffer" in the call to the DataParser.ProcessDataReceived(ref buffer, data.ToString()) method. That gives the continuity necessary for the DataParser to contain the entire partial set of data with every chunk of data received from the TCP port, until the Parser determines it has at least a full set of data (the current chunk of data may contain a partial set or multiple sets of data, hence the need for a List<string> returned from the Parser).

The BufferedDataParser is an abstract class, that is meant to be inherited by different kinds of Parsers that rely on receiving buffered data. I've included two different kinds of Parsers (one of which is demonstrated in the application). So, here's the abstract class:


public abstract class BufferedDataParser : IBufferedDataParser
{
    #region Declarations

    protected string HeartBeat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><ENDOFXML/>";
    protected string ValidStart = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>";
    protected string ValidEnd = "</AXmlTargets>";
    protected bool IncludeDelimiters = true;

    #endregion

    #region Methods

    public List<string> ProcessDataReceived(ref string buffer, string dataReceived)
    {
        List<string> DataList = new List<string>();
        try
        {
            //add received data to the buffer
            buffer += dataReceived;

            this.RemoveHeartbeatData(ref buffer);

            int whileLoopWatchdog = 0;
            while (buffer.Length > 0)
            {
                this.RemoveDataPrecedingStart(ref buffer);

                //verify that a complete message is present, if so process just that
                string Complete = this.ProcessCompleteMessage(ref buffer);
                if (Complete.IsNotNullOrEmpty())
                {
                    string Xml;
                    if (this.IncludeDelimiters)
                        Xml = Complete;
                    else
                        Xml = this.ParseOutDelimiters(Complete);

                    DataList.Add(Xml);

                    //remove the data that was pulled out for processing
                    //this will replace multiple identical occurrences but that
                    //is fine
                    buffer = buffer.Replace(Complete, "");
                }

                //HACK: prevent endless while loop
                //(although 20+ complete sets of data at once probably wouldn't
                //happen, and if it did the remainder would be caught on
                //next data receive)
                whileLoopWatchdog++;
                if (whileLoopWatchdog > 20)
                {
                    whileLoopWatchdog = 0;
                    break;
                }
            }
        }
        catch (Exception ex)
        {
            LogOutput.WriteLine(ex);
        }
        return DataList;
    }

    protected virtual void RemoveHeartbeatData(ref string buffer)
    {
        int whileLoopWatchdog = 0;

        //remove any "heartbeat" data
        while (buffer.Contains(HeartBeat))
        {
            //this will replace multiple identical occurrences but that
            //is fine
            buffer = buffer.Replace(HeartBeat, "");

            //HACK: prevent endless while loop
            //(although 100+ heartbeats should never happen)
            whileLoopWatchdog++;
            if (whileLoopWatchdog > 100)
            {
                break;
            }
        }
    }
    protected virtual void RemoveDataPrecedingStart(ref string buffer)
    {
        //discard any text that precedes the valid start
        if (buffer.IndexOf(ValidStart) > 0)
        {
            buffer = buffer.Substring(
            buffer.IndexOf(ValidStart));
        }
    }
    protected virtual string ProcessCompleteMessage(ref string buffer)
    {
        string Complete = "";

        if ((buffer.StartsWith(ValidStart)) && (buffer.Contains(ValidEnd)) && (buffer.IndexOf(ValidStart) < buffer.IndexOf(ValidEnd)))
        {
            Complete = buffer.Substring(buffer.IndexOf(ValidStart),
            buffer.IndexOf(ValidEnd) + ValidEnd.Length - buffer.IndexOf(ValidStart));
        }

        return Complete;
    }
    protected virtual string ParseOutDelimiters(string Complete)
    {
        return Complete.Substring(Complete.IndexOf(ValidStart) + ValidStart.Length, Complete.IndexOf(ValidEnd) - ValidStart.Length);
    }

    #endregion
}

Some DataParsers, simply need to use different delimiters and let the base class handle everything else, like the StxEtxDataParser:


public class StxEtxDataParser :BufferedDataParser
    {
    public StxEtxDataParser()
    {
        this.HeartBeat = "no heartbeat";
        this.ValidStart = new string(new char[] { (char)2 });
        this.ValidEnd = new string(new char[] { (char)3 });
        this.IncludeDelimiters = false;
    }
}

STX/ETX are common delimiters for TCP transmissions. STX is a (char)2 and ETX is a (char)3.

The DataParsers.cs file also contains a CountDelimiterDataProcessor. It inherits from the BufferedDataParser also, but has a totally different way of parsing the received data. I won't post it here, you can check it out when you look at the downloaded zip file.

Now, as promised, I'll get back to the purpose of TestFromFiles: You should have sample data from the Server you're trying to connect to saved in a file. You obviously have to have this, or how would you ever be able to develop an application without knowing something about the data!?! By using this saved file to test from, you can quickly determine if your DataParser is parsing the data correctly without worrying about connecting to a Server. If there's a problem, keep refactoring your DataParser (not the abstract class! That one should not be touched!) until it's perfect.

I think that's about it for now. Download the two zip files and give it a test yourself. It's good to go "right out of the box" ... you only have to change the ipaddress (and port if you want to) in the app.configs of both Client and Server.

Happy Coding!  =0)

Saturday, September 29, 2018

VS2017 - A Few Idiosyncrasies

I recently found some time to install the most recent version of Visual Studio 2017 and start using it for real work. I had actually had a few of the early pre-Release versions installed, but hadn't yet used them for anything other than "playing" (didn't want to use a pre-Release version for real work ... I'm not *that* much into "cutting edge"). I'd been so busy that I just continued using VS 2015 for work, even after the official release of VS2017.

What actually prompted me to install and start using VS2017 was that my laptop died. When I got the replacement, of course I had to re-install my entire development environment (not to mention a bunch of personal stuff not related to work). So, what the heck, might as well go with the latest and greatest!

I quickly found out that there were a few idiosyncrasies where things differed from my earlier VS 2015 version. I wanted to put some of it back to "my way":

  • Some shortcut keys aren't what they used to be. But, short-cut keys are easy to change:
    • Go to Tools > Options > Environment > Keyboard
    • Find the command shortcut you want to change and in the "Press shortcut keys" textbox, just do as it says ... press the keys.
    • Click OK when you're done
    • For example, Building a Solution has been shortcut F6 for so many, many years (even before Visual Studio was designed) that it's ingrained in my finger muscle-memory. F6 is a "complement" to F5  (for Build and Execute Debugging). However, it seems to have changed at some point (I don't know if it changed in this VS version or some previous one) to Ctrl+Shift+B. I just could *not* get used to that, so I changed it back to F6 and now I'm happy.
  • Outlining :
    • The Outlining shortcut keys work a little differently than they used to in previous versions of VS.
    • They all used to be specified as Ctrl+M, SecondKey ... but now it is Ctrl+M, Ctrl+SecondKey
      • Just keep holding the Ctrl down during both M and SecondKey, rather than making it two different key combinations as shown.
      • It works slightly different than in previous versions of VS, where either one of these worked:
        • continuously hold down the Ctrl key
        • or do Ctrl+M, let go of the Ctrl and type SecondKey
        • Now, if you try that second option, you get an error and it does nothing.
    • Collapse to Definitions
      • By default now, it does *NOT* Collapse the Regions! Arrgghh!! This particular "feature" is what prompted me to write this blog post, because it *really* irritated me!!!  But, not to worry, it can be fixed. Yay!
        • Tools > Options > Text Editors > C# > Advanced > Outlining > check "Collapse #regions when collapsing to definitions"
  • CodeLens is a great feature and it's been around since at least VS2015 (although I wasn't using it then). Be aware that it's not available on the VS2017 Community edition (only Professional and higher). You can read about it here:  https://docs.microsoft.com/en-us/visualstudio/ide/find-code-changes-and-other-history-with-codelens?view=vs-2017
    • But, here's the downside ... it puts its cool little goodies just where I DO NOT WANT THEM!!
    • Here's what my code looks like without CodeLens, nice and compact!
    • Here's what it looks like with CodeLens, the look of my code is now quite annoying (to me anyway):

    • But, you can disable the whole thing with an Options setting:
      • Tools > Options > Text Editors > All Languages > CodeLens > uncheck "Enable CodeLens"
    • I would really have preferred an option to change the position of where the CodeLens information is displayed, because as I said I believe some of what it does can be quite helpful. But, unfortunately, I don't think that there is anyway to do it, at least not that I know of.

Well, that's about it for now. So far, I'm glad it's a short list!!  Happy Coding!!  =0)

Sunday, August 19, 2018

ComboBox Gotcha #2

Recently, I participated in an MSDN Forum thread about problems with ComboBoxes, SelectedValue and SelectedIndex. It reminded me of a "gotcha" that I knew about a long time ago, but had forgotten about. I haven't been doing any development on Windows Forms apps in a long time (I have been doing only back-end server side stuff for more than a few years now), although many years ago I was responsible for a pretty extensive Windows Forms "Framework" for our developers to use for all of our company's applications. So, I knew about this then (and I'm sure that I built it into the Framework's base classes DataBinding methods). But, I digress ... let's get to a description of the problem and how to make sure it doesn't happen to you!

Say that in your application, you need to do something with the SelectedValue of a ComboBox. So, you create an event handler for either the .SelectedIndexChanged event or the .SelectedValueChanged event. But it doesn't work the way you expect it to and you're getting exceptions sometimes. What could be the problem? Let's look at the result of some debugging code placed in each of those event handlers to try and troubleshoot the problem:

Here's the debugging code:

private void cboCustomer_SelectedValueChanged(object sender, EventArgs e)
{
Console.WriteLine($"Value Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
Console.WriteLine($"Value Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
private void cboCustomer_SelectedIndexChanged(object sender, EventArgs e)
{
Console.WriteLine($"Index Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
Console.WriteLine($"Index Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}

And here are the results:

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is System.Data.DataRowView, is a System.Data.DataRowView

Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is 01, is a System.String

This is obviously, not a good thing ... the events are firing way too many times. And why is the SelectedValue a DataRowView at first (in my example I am using a DataTable as the Combo's DataSource)?  In fact, these events should not be firing at all at this time ... we have simply set up the databinding for the ComboBox.  Wow! What did we do wrong?

Here's the databinding code (remember that this is BAD CODE!)

this.bsCustomers = new BindingSource();
this.bsCustomers.DataSource = this.oDataFromXML.Tables["Customer"];

this.cboCustomer.SelectedIndexChanged += cboCustomer_SelectedIndexChanged;
this.cboCustomer.SelectedValueChanged += cboCustomer_SelectedValueChanged;

this.cboCustomer.DataSource = this.bsCustomers;
this.cboCustomer.DisplayMember = "Last Name";
this.cboCustomer.ValueMember = "CustomerID";

The problem exists because the DataSource is set first. The subsequent statements setting the DisplayMember and ValueMember each fire the events (because now the SelectedValue Properties are changed by setting those Members).

There is an easy way to fix this, and perhaps you've already guessed it!  Set the DataSource *after* setting the DisplayMember and ValueMember.

this.bsCustomers = new BindingSource();
this.bsCustomers.DataSource = this.oDataFromXML.Tables["Customer"];

this.cboCustomer.SelectedIndexChanged += cboCustomer_SelectedIndexChanged;
this.cboCustomer.SelectedValueChanged += cboCustomer_SelectedValueChanged;

this.cboCustomer.DisplayMember = "Last Name";
this.cboCustomer.ValueMember = "CustomerID";
this.cboCustomer.DataSource = this.bsCustomers;

But wait ... what's going on? Now the code in the event handlers throws an exception!

Value Changed: SelectedIndex is -1
Exception thrown: 'System.NullReferenceException' in WindowsApplication1.exe

That's because, now that we've done the databinding *correctly*, initially nothing is selected (this only happens once) and consequently the SelectedIndex is -1 and the SelectedValue will be null.  You *must* have code to check that SelectedIndex is not negative! Especially if you databind the ComboBox to a DataTable that initially has no data in it! When there *is* data, the first item will then be automatically selected and the SelectedIndex will be 0 ... but if there is *no* data, obviously nothing can be selected. The SelectedIndex will be -1 to indicate that.

private void cboCustomer_SelectedValueChanged(object sender, EventArgs e)
{
Console.WriteLine($"Value Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
if (cboCustomer.SelectedIndex > -1)
{
Console.WriteLine($"Value Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
}
private void cboCustomer_SelectedIndexChanged(object sender, EventArgs e)
{
Console.WriteLine($"Index Changed: SelectedIndex is {cboCustomer.SelectedIndex}");
if (cboCustomer.SelectedIndex > -1)
{
Console.WriteLine($"Index Changed: SelectedValue is {cboCustomer.SelectedValue}, is a {cboCustomer.SelectedValue.GetType()}");
Console.WriteLine("");
// the rest of your code is here
}
}

And the results now:

Value Changed: SelectedIndex is -1
Value Changed: SelectedIndex is 0
Value Changed: SelectedValue is 01, is a System.String

Index Changed: SelectedIndex is 0
Index Changed: SelectedValue is 01, is a System.String

In a similar vein, I wrote a blog post back in 2012 with the title "ComboBox Gotchas". But, it was about a totally different problem ... a ComboBox that is in sync with other controls on the Form, when it's not supposed to be. See it here, if you're interested: https://geek-goddess-bonnie.blogspot.com/2012/12/combobox-gotchas.html


Sunday, May 27, 2018

(A)Synchronicity

Back in the day, "The Police" sang a song about Synchronicity. Today, I'm writing a blog about Asynchronicity.  ;0)

To be more precise, this blog post will explore whether or not there are performance differences between the various ways to write asynchronous code. I will be looking specifically at the code needed for a multi-threaded application, where the threads just do their thing, and we don't care what they're doing (in other words, kind of a Fire-And-Forget type of thing). I will explore these three methodologies:

  • System.Threading.Thread.Start()
  • System.Threading.ThreadPool.QueueUserWorkItem()
  • System.Threading.Tasks.Task.Run()

I'm betting that the latter two methods are simply syntactic sugar on top of the original Thread.Start() ... let's see if we can get some numerical evidence of that. And, just for the heck of it, I'm going to throw in a freebie ... I'm going to compare the Task.Run() both without and with a .ConfigureAwait(false) ... (not that it matters for this discussion). So, here's the code I used for testing this (called from a Button Click so I could easily run it again):

private System.Diagnostics.Stopwatch stopWatchThread;
private System.Diagnostics.Stopwatch stopWatchQueueUserWork;
private System.Diagnostics.Stopwatch stopWatchTask;
private System.Diagnostics.Stopwatch stopWatchTaskAwaitFalse; private void CompareThreeAsyncMethods()
{
this.stopWatchThread = new Stopwatch();
Console.WriteLine("Start Thread at {0}", DateTime.Now);
this.stopWatchThread.Start();
ThreadStart threadStart = new ThreadStart(RunThread);
Thread myThread = new Thread(threadStart);
myThread.Start(); this.stopWatchQueueUserWork = new Stopwatch();
Console.WriteLine("Start QueueUserWork at {0}", DateTime.Now);
stopWatchQueueUserWork.Start();
ThreadPool.QueueUserWorkItem(WorkToDo, new WorkSettings() { stopWatch = stopWatchQueueUserWork, WorkName = "QueueUserWork" }); this.stopWatchTask = new Stopwatch();
Console.WriteLine("Start Task at {0}", DateTime.Now);
stopWatchTask.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() {stopWatch = stopWatchTask, WorkName = "Task" });
}); this.stopWatchTaskAwaitFalse = new Stopwatch();
Console.WriteLine("Start Task AwaitFalse at {0}", DateTime.Now);
stopWatchTaskAwaitFalse.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTaskAwaitFalse, WorkName = "TaskAwaitFalse" });
}).ConfigureAwait(false); Console.WriteLine("Show results as threads complete ... {0}\r\n", DateTime.Now);
}
public class WorkSettings
{
public System.Diagnostics.Stopwatch stopWatch { get; set; }
public string WorkName { get; set; }
}
// Need this only for the Thread option
private void RunThread()
{
this.WorkToDo(new WorkSettings() { stopWatch = this.stopWatchThread, WorkName = "Thread" });
} private void WorkToDo(object obj)
{
WorkSettings work = (WorkSettings)obj;
for (int i = 0; i < 2147483647; i++) // largest int 2,147,483,647
{
//Thread.Sleep(500);
}

work.stopWatch.Stop();
Console.WriteLine("{0} Stopped at {1}. Elapsed Time {2} milliseconds\r\n", work.WorkName, DateTime.Now, work.stopWatch.ElapsedMilliseconds);
}

By starting a StopWatch before I start running its Asynchronous method, I take into account any "prep" work that might be involved in starting the call to the method. If there is any, it should be reflected in the overall time to complete a method. Here are the results:

Start Thread at 5/26/2018 5:13:38 PM
Start QueueUserWork at 5/26/2018 5:13:38 PM
Start Task at 5/26/2018 5:13:38 PM
Start Task AwaitFalse at 5/26/2018 5:13:38 PM
Show results as threads complete ... 5/26/2018 5:13:38 PM Thread Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5669 milliseconds
Task Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5775 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5836 milliseconds
QueueUserWork Stopped at 5/26/2018 5:13:44 PM. Elapsed Time 5859 milliseconds // last (10th) time
Thread Stopped at 5/26/2018 5:18:14 PM. Elapsed Time 5890 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 5978 milliseconds
QueueUserWork Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 5982 milliseconds
Task Stopped at 5/26/2018 5:18:15 PM. Elapsed Time 6008 milliseconds

// Average
Average for Thread: 5986 milliseconds
Average for Task: 5964 milliseconds
Average for TaskAwaitFalse: 5973 milliseconds
Average for QueueUserWork: 6048 milliseconds

Judging by those results, I think that the minuscule difference might be attributed to them all running at once & taking up CPU resources. But, we can test that as well. I'm going to run them individually. Run one. Let it complete. Run another. Let it complete. I wonder if the times will be *exact* then! Let's re-work the code to look like this:

private CompareAsync methodIs = CompareAsync.Begin;
private CompareAsync currentMethodRunning = CompareAsync.Begin;
private void CompareThreeAsyncMethodsIndividually()
{
while (methodIs != CompareAsync.Done)
{
if (methodIs == CompareAsync.Begin)
methodIs = CompareAsync.Thread;
if (methodIs == currentMethodRunning)
continue;
switch (methodIs)
{
case CompareAsync.Thread:
currentMethodRunning = methodIs;
this.stopWatchThread = new Stopwatch();
Console.WriteLine("Start Thread at {0}", DateTime.Now);
this.stopWatchThread.Start();
ThreadStart threadStart = new ThreadStart(RunThread);
Thread myThread = new Thread(threadStart);
myThread.Start();
break;
case CompareAsync.QueueUserWork:
currentMethodRunning = methodIs;
this.stopWatchQueueUserWork = new Stopwatch();
Console.WriteLine("Start QueueUserWork at {0}", DateTime.Now);
stopWatchQueueUserWork.Start();
ThreadPool.QueueUserWorkItem(WorkToDo, new WorkSettings() { stopWatch = stopWatchQueueUserWork, WorkName = "QueueUserWork" });
break;
case CompareAsync.Task:
currentMethodRunning = methodIs;
this.stopWatchTask = new Stopwatch();
Console.WriteLine("Start Task at {0}", DateTime.Now);
stopWatchTask.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTask, WorkName = "Task" });
});
break;
case CompareAsync.TaskAwaitFalse:
currentMethodRunning = methodIs;
this.stopWatchTaskAwaitFalse = new Stopwatch();
Console.WriteLine("Start Task AwaitFalse at {0}", DateTime.Now);
stopWatchTaskAwaitFalse.Start();
Task.Run(() =>
{
WorkToDo(new WorkSettings() { stopWatch = stopWatchTaskAwaitFalse, WorkName = "TaskAwaitFalse" });
}).ConfigureAwait(false);
break;
default:
break;
}
}
}


And added an extra bit at the end of the WorkToDo() method such that it would be able to determine which method would run next:

private void WorkToDo(object obj)
{
WorkSettings work = (WorkSettings)obj;
for (int i = 0; i < 2147483647; i++) // largest int 2,147,483,647
{
//Thread.Sleep(500);
}
work.stopWatch.Stop();
Console.WriteLine("{0} Stopped at {1}. Elapsed Time {2} milliseconds\r\n", work.WorkName, DateTime.Now, work.stopWatch.ElapsedMilliseconds);

// ADDED THIS
// Set it to the next one
switch (methodIs)
{
case CompareAsync.Thread:
methodIs = CompareAsync.QueueUserWork;
break;
case CompareAsync.QueueUserWork:
methodIs = CompareAsync.Task;
break;
case CompareAsync.Task:
methodIs = CompareAsync.TaskAwaitFalse;
break;
case CompareAsync.TaskAwaitFalse:
methodIs = CompareAsync.Done;
break;
}
}

And now the results are this:

Start Thread at 5/26/2018 5:29:32 PM
Thread Stopped at 5/26/2018 5:29:37 PM. Elapsed Time 4877 milliseconds

Start QueueUserWork at 5/26/2018 5:29:37 PM
QueueUserWork Stopped at 5/26/2018 5:29:42 PM. Elapsed Time 4850 milliseconds

Start Task at 5/26/2018 5:29:42 PM
Task Stopped at 5/26/2018 5:29:47 PM. Elapsed Time 4918 milliseconds

Start Task AwaitFalse at 5/26/2018 5:29:47 PM
TaskAwaitFalse Stopped at 5/26/2018 5:29:52 PM. Elapsed Time 4978 milliseconds

// last time (10th)
Thread Stopped at 5/26/2018 5:32:37 PM. Elapsed Time 4910 milliseconds
QueueUserWork Stopped at 5/26/2018 5:32:42 PM. Elapsed Time 4821 milliseconds
Task Stopped at 5/26/2018 5:32:46 PM. Elapsed Time 4894 milliseconds
TaskAwaitFalse Stopped at 5/26/2018 5:32:51 PM. Elapsed Time 4924 milliseconds

// Average
Average for Thread: 4986 milliseconds
Average for QueueUserWork: 4975 milliseconds
Average for Task: 4966 milliseconds
Average for TaskAwaitFalse: 4962 milliseconds

Well now ... there's still a difference between the 4 methods, isn't there? However, it is still teeny. Running each of these individually (instead of starting all 4 methods at the same time, as I did in the first test) produces the same results pretty much, the difference being that each is about 1000 milliseconds faster than the first test.

The difference between the fastest & slowest averages for the second test is only about .2% and that's in milliseconds! I'd say that these various methodologies for running asynchronous threads, aren't really any different (statistically speaking). Hey, I just like to mess around with numbers.  ;0)

Happy coding!  =0)

Sunday, February 25, 2018

Parent/Child DataBinding

I have seen a few questions lately on the MSDN Forums about synchronizing parent/child data on a Windows Form. I thought that I had written a blog post about this years ago, but apparently I'm mis-remembering (probably remembering all the Forum posts that I've answered in the past)! This particular post will deal with parent/child using DataSets, and we can deal with this in two ways: using DataRelations or using the Filter property of the child BindingSource.

First, let's take a look at using DataRelations ... this is a pretty straightforward process (especially if you are already using DataRelations).

// First, you'll need two BindingSources:
private BindingSource bsParent;
private BindingSource bsChild;

You obviously need to have a DataRelation between two DataTables in your DataSet. If you don't already know how to set that up, it's pretty straightforward:

this.dsCust.Relations.Add("CustomerOrdersRelation",
    this.dsCust.Tables["Customer"].Columns["CustID"],
    this.dsCust.Tables["Orders"].Columns["CustID"]);

I usually recommend that DataBinding be set up in the Form's Load event handler. You'd have code like this (this would be after you've already set up a DataRelation):

this.bsParent = new BindingSource();
this.bsChild = new BindingSource();
 
this.bsParent.DataSource = this.dsCust;
this.bsParent.DataMember = "Customer";
this.bsChild.DataSource = this.bsParent;
this.bsChild.DataMember = "CustomerOrdersRelation";

There are two key things to note here:
  1. The DataSource for the bsChild is the bsParent.
  2. The DataMember for the bsChild is the Relationship, rather than a DataTable. This is what makes the whole thing work.
Now, we can use the BindingSources to DataBind parent/child DataGridViews and/or TextBoxes or other Controls, depending on the functionality you need:

this.oGridCustomer.DataSource = this.bsParent;
this.oGridOrders.DataSource = this.bsChild;
this.txtOrderDescription.DataBindings.Add("Text", this.bsChild, "description");

There *is* one little "gotcha" that you sometimes need to watch out for. That's the subject of bsChild.EndEdit(). The potential problem and solution is described in my blog post: https://geek-goddess-bonnie.blogspot.com/2010/03/uncommitted-child-table-changes.html

However, if you *can't* (or don't want to) use a DataRelation, then you do this with a Filter on the child BindingSource. It's only slightly different, because we have to utilize the bsParent.PositionChanged event handler to reset the Filter.

The bsParent is set up the same way as it was above. It's the bsChild that will be different:

bsChild.DataSource = this.dsCust.Tables["Orders"];
bsChild.Filter = string.Format("CustID = '{0}'", this.dsCust.Tables["Customer"].Rows[bsParent.Position]["CustID"]);
bsParent.PositionChanged += bsParent_PositionChanged;

And, for the Parent PositionChanged event handler, we just reset the Child Filter:

private void bsParent_PositionChanged(object sender, EventArgs e)
{
    bsChild.Filter = string.Format("CustID = '{0}'", this.dsCust.Tables["Customer"].Rows[bsParent.Position]["CustID"]);
}

And, obviously, since the two grids and the TextBox are set up using the bsParent and bsChild, nothing needs to change there.

In either scenario, as the user moves through the rows in the Customer grid, the corresponding child rows will be all that's displayed in the Order grid and the corresponding Order description in the textbox.


Tuesday, January 30, 2018

Use and Generate PUT Stored Procedures

To me, the "best practice" for DataAccess to a database is to always utilize Stored Procedures, if your database server supports them. Microsoft SQL Server, of course, supports Stored Procs as does Microsoft Access. I use SQL Server exclusively, so this blog post will be limited to that database server. It's easy enough to take the concepts shown here and apply the same methodology to an Access database, or to any other database that allows Stored Procs.

However, if you Google for information about the best practices involving Stored Procedures, you will, of course, find a gazillion conflicting opinions ... some agreeing with my opinion and others not. But still, I think it's an important best practice.

OK, now I'll get down off my soap box and on to the topic of my post!

I have a base DataAccess class, than can be used directly, or sub-classed to a domain use (for example, a large inventory system may have an OrdersDataAccess, an InventoryDataAccess and a CustomerDataAccess ... all subclassed from the BaseDataAccess). I have a series of DataAccess posts here in my blog, and I will show bits and pieces of it here in this post to illustrate my point. If you'd like to read these additional posts, you can use this search: http://geek-goddess-bonnie.blogspot.com/search?q=DataAccess Some of the earlier posts are incomplete, code-wise, but get "fleshed out" in later posts.

I see people on the Forums struggling with simple CRUD activity, trying to utilize the SqlCommandBuilder (usually incorrectly) and having all kinds of problems. SqlCommandBuilder is not the answer, in fact, I never use it ... there is an easier way to Insert and/or Update from a DataTable and do it by using the same Stored Proc (one for each table).  It's what I refer to as PUT procs, and the PUT Stored Proc will either INSERT or UPDATE depending on whether or not the Primary Key (PK) is included in the Parameters.

One caveat here ... I use the DataSet/DataTable paradigm for my DTOs (Data Transfer Objects). The same concepts can apply to POCO, with just different methods in the DataAccess class needed. (POCO means "Plain Old CLR Object which is defined here in Wikipedia: https://en.wikipedia.org/wiki/Plain_old_CLR_object as "a POCO does not have any dependency on an external framework and generally does not have any attached behaviour." ... but go ahead and read WikiPedia if you are unfamiliar with the term).

Here's a simple example of a PUT Stored Proc. I will show some methods you can use to create a Utility for generating such Procs automatically at the end of this blog post:

CREATE PROCEDURE [dbo].[bsp_StatusPut]
    @statuskey             bigint = NULL OUTPUT,
    @statusdatetime        datetime = NULL,
    @status                varchar(max) = NULL,
    @agency                varchar(100) = NULL,
    @messagetype           varchar(128) = NULL,
    @id                    uniqueidentifier = NULL
AS
    SET NOCOUNT ON
    SET XACT_ABORT ON

    BEGIN TRANSACTION
    IF (@statuskey <= 0 OR @statuskey IS NULL)
    BEGIN
        -- Insert Values into the Status table
        INSERT Status
            (
            statusdatetime,
            status,
            agency,
            messagetype,
            id)
        SELECT 
            @statusdatetime,
            @status,
            @agency,
            @messagetype,
            @id
 
        SELECT @statuskey = SCOPE_IDENTITY()
    END
    ELSE
    BEGIN
        -- Update the Status table
        UPDATE Status
        SET 
            statusdatetime = ISNULL(@statusdatetime, statusdatetime),
            status = ISNULL(@status, status),
            agency = ISNULL(@agency, agency),
            messagetype = ISNULL(@messagetype, messagetype),
            id = ISNULL(@id, id)
        WHERE statuskey = @statuskey
    END
 
    COMMIT TRANSACTION
    RETURN 0
GO


Now, as you can see, this SP doesn't have many parameters, and it would be easy to add them one-by-one. However, there's an easier way when you have a lot of columns in your table.  In my DataAccess class, I  have a set  of methods for dealing with Parameters. Here are the relevant bits of code from a BaseDataAccess class to go along with my example:

// This is not the entire class, see blog posts for more of the class
public class BaseDataAccess : IDisposable
{
    private IDbConnection oConnection;
    private IDbCommand oCommand;
    private IDbDataAdapter oAdapter;
    private IDataParameterCollection oParms;
    
    public string ConnectionString = "";
 
    public BaseDataAccess(string connectionString)
    {
        this.ConnectionString = connectionString;
        this.oConnection = new SqlConnection(connectionString);
        this.oCommand = new SqlCommand();
        this.oAdapter = new SqlDataAdapter((SqlCommand)this.oCommand);
        this.oCommand.Connection = this.oConnection;
        this.oCommand.CommandType = CommandType.StoredProcedure;
    }
 
    protected void SetAllParameters(DataRow Row)
    {
        this.ClearParameters();
        for (int i = 0; i < Row.Table.Columns.Count; i++)
        {
            this.AddParms(Row.Table.Columns[i].ColumnName, Row[i]);
        }
    }
    public void AddParms(string ParmName, object ParmValue)
    {
        if (ParmName.StartsWith("@") == false)
            ParmName = "@" + ParmName;
        
        if (ParmValue != DBNull.Value)
        {
            if (this.oCommand.Parameters.IndexOf(ParmName) > -1)
                this.oCommand.Parameters[ParmName].Value = ParmValue;
            else
                this.oCommand.Parameters.AddWithValue(ParmName, ParmValue);
        }
        else
            this.RemoveParm(ParmName);
    }
    protected void ClearParameters()
    {
        this.oCommand.Parameters.Clear();
    }
    public void RemoveParm(string ParmName)
    {
        if (ParmName.StartsWith("@") == false)
            ParmName = "@" + ParmName;
        if (this.oCommand.Parameters.IndexOf(ParmName) > -1)
            this.oCommand.Parameters.RemoveAt(ParmName);
    }
    public void SetParmDirection(string ParmName, ParameterDirection direction)
    {
        if (ParmName.StartsWith("@") == false)
            ParmName = "@" + ParmName;
        if (this.oCommand.Parameters.IndexOf(ParmName) > -1)
            this.oCommand.Parameters[ParmName].Direction = direction;
    }
    
    public void ExecuteCommand()
    {
        this.ExecuteCommand(this.oCommand.CommandText);
    }
    public void ExecuteCommand(string StoredProcName)
    {
        this.OpenConnection();
        this.oCommand.CommandText = StoredProcName;
        try
        {
            this.oCommand.ExecuteNonQuery();
        }
        catch (SqlException ex)
        {
            // Process the SqlException with your normal logging methodology
        }
    }
 
    public void OpenConnection()
    {
        try
        {
            if (this.oConnection.State != ConnectionState.Open)
            {
                this.oConnection.Open();
            }
        }
        catch (Exception ex)
        {
            // Process the Exception with your normal logging methodology
        }
    }
 
    public void Dispose()
    {
        if (this.oConnection != null)
        {
            if (this.oConnection.State != ConnectionState.Closed)
                this.oConnection.Close();
            this.oConnection.Dispose();
        }
    }
}


Here is how you might use these in your own DataAccess classes, for example, if you've created a StatusDataAccess class, sub-classed from BaseDataAccess. My example uses a Typed DataSet, but it could just as easily be used with a plain old DataSet (this is based on my SP example above):

public void UpdateStatus(StatusDataSet ds)
{
    using (DataAccess da = new DataAccess(this.MyConnectionString))
    {
        da.SetCommand("bsp_StatusPut");
        foreach (StatusDataSet.StatusRow call in ds.Status)
        {
            da.SetAllParameters(call);
            da.SetParmDirection("statuskey", ParameterDirection.InputOutput);
            da.ExecuteCommand();
        }
    }
}


OK, as promised earlier, here is a set of methods you can use to automatically generate the SQL Script to create such PUT Stored Procedures for every table in your database. Write yourself a database utility application, if you don't already have one, and include these methods in it:

#region Generating PUT Stored Procs
 
StringBuilder ProcBuilder;
public string GeneratePutStoredProc(string DatabaseName, string TableName)
{
    DataTable dtColumns = new DataTable(); ;
    string ErrorMsg = "";
    this.ProcBuilder = new StringBuilder();
 
    // First, let's fill a DataTable with the information for each column in the database Table
    using (SqlConnection conn = new SqlConnection(this.ConnectionStringStatus))
    {
        try
        {
            conn.Open();
            dtColumns = conn.GetSchema("Columns", new[] { DatabaseName, null, TableName });
        }
        catch (Exception ex)
        {
            ErrorMsg = ex.Message + "\r\n";
            if (ex.InnerException != null && ex.InnerException.Message != ex.Message)
                ErrorMsg += ex.InnerException.Message + "\r\n";
        }
    }
    // Get error-handling out of the way
    if (dtColumns.Rows.Count == 0 && ErrorMsg != "")
        ErrorMsg += string.Format("No columns retrieved for {0}/{1}.\r\n", DatabaseName, TableName);
    if (ErrorMsg != "")
    {
        ErrorMsg += "Check your database and/or connection";
        MessageBox.Show(ErrorMsg);
        return "";
    }
 
    // One of our database conventions is that the name of the PK column is always TableName + "Key"
        string KeyName = TableName + "Key";
 
    // Then start building the SQL script for the creation of the Stored Proc,
    // calling a few methods to do the heavy lifting
    this.BuildHeader(TableName, "PUT");
    this.BuildParameterList(dtColumns, KeyName);
 
    this.ProcBuilder.Append("\r\nAS\r\n");
    this.ProcBuilder.Append("\tSET NOCOUNT ON\r\n");
    this.ProcBuilder.Append("\tSET XACT_ABORT ON\r\n\r\n");
    this.ProcBuilder.Append("\tBEGIN TRANSACTION\r\n");
 
    this.ProcBuilder.AppendFormat("\tIF (@{0} <= 0 OR @{0} IS NULL)\r\n", KeyName);
    this.ProcBuilder.Append("\tBEGIN\r\n");
    this.BuildInsert(dtColumns, KeyName, TableName);
    this.ProcBuilder.Append("\tEND\r\n\tELSE\r\n\tBEGIN\r\n");
    this.BuildUpdate(dtColumns, KeyName, TableName);
    this.ProcBuilder.Append("\tEND\r\n\r\n\tCOMMIT TRANSACTION\r\n\tRETURN 0\r\n");
 
    this.BuildFooter();
 
    return this.ProcBuilder.ToString();
}
 
protected void BuildHeader(string TableName, string ProcType)
{
    string prefix = "bsp";
    string fullProcName = String.Format("{0}_{1}{2}", prefix, TableName, ProcType);
 
    this.ProcBuilder.Append("\r\n");
    this.ProcBuilder.Append("---------------------------------------------------------------\r\n");
    this.ProcBuilder.AppendFormat("-- Stored Procedure {0}\r\n", fullProcName);
    this.ProcBuilder.AppendFormat("-- Basic {1} Procedure for {0} \r\n", TableName, ProcType);
    this.ProcBuilder.AppendFormat("-- Automatically generated {0}\r\n", DateTime.Now);
    this.ProcBuilder.Append("---------------------------------------------------------------\r\n");
    this.ProcBuilder.AppendFormat("IF EXISTS(SELECT * FROM sys.objects WHERE name LIKE '{0}' and type = 'P')\r\n", fullProcName);
    this.ProcBuilder.AppendFormat("\tDROP PROCEDURE {0}\r\n", fullProcName);
    this.ProcBuilder.Append("GO\r\n");
    this.ProcBuilder.Append("SET QUOTED_IDENTIFIER OFF \r\n");
    this.ProcBuilder.Append("GO\r\n");
    this.ProcBuilder.Append("SET ANSI_NULLS ON \r\n");
    this.ProcBuilder.Append("GO\r\n\r\n");
    this.ProcBuilder.AppendFormat("CREATE PROCEDURE {0}\r\n", fullProcName);
}
protected void BuildParameterList(DataTable dtColumns, string KeyName)
{
    StringBuilder sb = new StringBuilder();
    foreach (DataRow row in dtColumns.Rows)
    {
        if (sb.Length > 0)
            sb.Append(",\r\n");
        sb.AppendFormat("\t@{0}\t\t{1} = NULL", row["COLUMN_NAME"], row["DATA_TYPE"]);
        if (row["COLUMN_NAME"].ToString().ToUpper() == KeyName.ToUpper())
            sb.Append(" OUTPUT");
    }
    this.ProcBuilder.Append(sb.ToString());
}
protected void BuildInsert(DataTable dtColumns, string KeyName, string TableName)
{
    this.ProcBuilder.AppendFormat("\t\t-- Insert Values into the {0} table\r\n", TableName);
    this.ProcBuilder.AppendFormat("\t\tINSERT {0}\r\n", TableName);
 
    StringBuilder sbColumns = new StringBuilder();
    StringBuilder sbParms = new StringBuilder();
    foreach (DataRow row in dtColumns.Rows)
    {
        if (row["COLUMN_NAME"].ToString().ToUpper() == KeyName.ToUpper())
            continue;
        if (sbColumns.Length > 0)
            sbColumns.Append(",\r\n");
        if (sbParms.Length > 0)
            sbParms.Append(",\r\n");
        sbColumns.AppendFormat("\t\t\t{0}", row["COLUMN_NAME"]);
        sbParms.AppendFormat("\t\t\t@{0}", row["COLUMN_NAME"]);
    }
    this.ProcBuilder.Append("\t\t\t(\r\n");
    this.ProcBuilder.AppendFormat("{0})\r\n", sbColumns.ToString());
    this.ProcBuilder.Append("\t\tSELECT\r\n");
    this.ProcBuilder.AppendFormat("{0}\r\n\r\n", sbParms.ToString());
    this.ProcBuilder.AppendFormat("\t\tSELECT @{0} = SCOPE_IDENTITY()\r\n", KeyName);
}
protected void BuildUpdate(DataTable dtColumns, string KeyName, string TableName)
{
    this.ProcBuilder.AppendFormat("\t\t-- Update the {0} table\r\n", TableName);
    this.ProcBuilder.AppendFormat("\t\tUPDATE {0}\r\n", TableName);
    this.ProcBuilder.Append("\t\tSET\r\n");
 
    StringBuilder sb = new StringBuilder();
    foreach (DataRow row in dtColumns.Rows)
    {
        if (row["COLUMN_NAME"].ToString().ToUpper() == KeyName.ToUpper())
            continue;
        if (sb.Length > 0)
            sb.Append(",\r\n");
        sb.AppendFormat("\t\t\t{0} = ISNULL(@{0}, {0})", row["COLUMN_NAME"]);
    }
    this.ProcBuilder.Append(sb.ToString());
    this.ProcBuilder.AppendFormat("\r\n\t\tWHERE {0} = @{0}\r\n", KeyName);
}
protected void BuildFooter()
{
    this.ProcBuilder.Append("\r\n");
    this.ProcBuilder.Append("GO\r\n");
    this.ProcBuilder.Append("SET QUOTED_IDENTIFIER OFF \r\n");
    this.ProcBuilder.Append("GO\r\n");
    this.ProcBuilder.Append("SET ANSI_NULLS ON \r\n");
    this.ProcBuilder.Append("GO\r\n");
}
 
#endregion


That's it. Happy coding!!  =0)