Saturday, July 23, 2016

PEMSTATUS & More With Reflection In .NET

Many, many, *many* years ago, when I was a Visual FoxPro developer, we could make use of a VFP function called PEMSTATUS. One of the things that function could tell you was whether or not a particular class/object contained a particular Property or Event or Method (hence the “PEM”) and other information about that P/E/M.

I’ve repurposed some of that functionality in .NET. Many of you may remember a post I wrote about 7 years ago, Reflection in .NET. That blog post was about using Reflection to dynamically instantiate a class using a string containing the name of the class. Well, we can also use Reflection for determining if a class/object contains a P/E/M, and we can then use Reflection for getting the value of a property or executing a method … all dynamically, just with a string containing the name.

Let’s say that we add some static methods to that MyReflectionClass from that 7-year-old post.

First, how about a PemStatus() method that will just tell us if a P/E/M exists? Here it is, along with two supporting methods:

// PemStatus checks for existence of a Property, Event or Method
public static bool PemStatus(object o, string name, string PEM)
{
    switch (PEM.ToUpper())
    {
        case "P":
            PropertyInfo pi = o.GetType().GetProperty(name);
            if (pi == null)
            {
                // GetProperty will only work on "real" properties ... those with get/set
                // Consequently, we want to look at Fields if we don't get a hit with GetProperty
                FieldInfo fi = GetFieldInfo(o, name);
                if (fi == null)
                    return false;
            }
            return true;
        case "E":
            EventInfo ei = o.GetType().GetEvent(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);
            if (ei == null)
            {
                // GetEvent doesn't always find the Event, so we'll check the Fields for it in that case
                FieldInfo fi = GetFieldInfo(o, name);
                return fi != null;
            }
            return ei != null;
        case "M":
            MethodInfo mi = GetMethodInfo(o, name);
            return mi != null;
        default:
            return false;
    }
}
private static FieldInfo GetFieldInfo(object o, string name)
{
    FieldInfo fi = null;
    if (o is System.Windows.Forms.Control && !(o is System.Windows.Forms.Form))
        fi = typeof(Control).GetField("Event" + name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);
    if (fi == null)
        fi = o.GetType().GetField(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);
 
    // Base class fields are not always searched, so we need to look in each BaseType if fi is still null
    if (fi == null)
    {
        Type baseType = o.GetType().BaseType;
        while (baseType != null && fi == null)
        {
            fi = baseType.GetField(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);
            baseType = baseType.BaseType;
        }
    }
 
    return fi;
}
private static MethodInfo GetMethodInfo(object o, string name)
{
    MethodInfo mi = null;
 
    mi = o.GetType().GetMethod(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static) as MethodInfo;
 
    // Base class methods are not always searched, so we need to look in each BaseType if mi is still null
    if (mi == null)
    {
        Type baseType = o.GetType().BaseType;
        while (baseType != null && mi == null)
        {
            mi = baseType.GetMethod(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);
            baseType = baseType.BaseType;
        }
    }
 
    return mi;
}

Now, what is this useful for? How about this: you might want to simply test for the existence of a P/E/M before deciding to do something else in your logic.  Let’s think of something that kind of makes sense. Let’s say that you’ve got an instance of a class (whether or not it’s been dynamically instantiated doesn’t matter) … let’s call it MyTest. You want to see if it has a Property called “TestIt”. I use the term “Property” loosely here, because it could also be a field (public, protected or even private).

Now, say your requirements are such that if MyTest has a TestIt property, then you want to get the value of that property and then see if MyTest contains a Method with the name contained in the TestIt property … and if so, execute that Method!  It’s actually pretty easy to do, but we’ll need some more static methods added to MyReflectionClass. Here they are:

// The next 3 methods, do stuff with the P/E/M:
// You can get the Property's value, execute the Event's handler, or execute the Method
public static object GetPropertyValue(object o, string name)
{
    PropertyInfo pi = o.GetType().GetProperty(name);
    if (pi != null)
        return pi.GetAccessors()[0].Invoke(o, null);
 
    FieldInfo fi = GetFieldInfo(o, name);
    if (fi != null)
        return fi.GetValue(o);
 
    return null;
}
public static void ExecuteEventHandler(object o, string name, EventArgs args = null)
{
    object eventKey = null;
    FieldInfo fi = GetFieldInfo(o, name);
 
    if (fi != null)
        eventKey = fi.GetValue(o);
 
    if (eventKey != null)
    {
        // This is for Events on non-Control objects
        if (eventKey is EventHandler)
            ((EventHandler)eventKey)(o, args);
        else
        {
            // This is for Events on Controls (such as a Button Click)
            EventHandlerList eventHandlerList = null;
            PropertyInfo pi = o.GetType().GetProperty("Events", BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Instance);
            if (pi != null)
                eventHandlerList = pi.GetValue(o, new object[] { }) as EventHandlerList;
 
            if (eventHandlerList != null)
            {
                var eventHandler = eventHandlerList[eventKey] as Delegate;
                Delegate[] invocationList = eventHandler.GetInvocationList();
                foreach (EventHandler item in invocationList)
                {
                    item(o, args);
                }
            }
        }
    }
}
public static object ExecuteMethod(object o, string name, object[] parms = null)
{
    object result = null;
 
    try
    {
        MethodInfo mi = GetMethodInfo(o, name);
        if (mi != null)
        {
            // Let's add checks for parameters while we're at it
            ParameterInfo[] pi = mi.GetParameters();
            if ((pi.Length == 0 && parms == null) || (parms != null && pi.Length == parms.Length))
                result = mi.Invoke(o, parms);
            else
                result = string.Format("Parameter mismatch in method '{0}'. Method expected {1} parameter{2}, but received {3}.",
                    name, pi.Length, pi.Length > 1 ? "s" : "", parms == null ? "none" : parms.Length.ToString());
        }
        else
            result = string.Format("Method '{0}' does not exist");
    }
    catch (Exception ex)
    {
        result = ex.Message;
    }
 
    return result;
}

Here is the class we’ll be instantiating and using in our testing:

public class TestForBlog
{
    protected string TestIt = "DoIt";

    protected void DoIt()
    {
        Console.WriteLine("Did It!!");
    }
}

The class *does* contain a field called TestIt. It defaults to containing the string “DoIt”. And there *is* a method called DoIt defined in this class!! Here’s how we implement this test:

// As mentioned, this could have been instantiated via Reflection, but that's not the point of this particular blog post. 
// Read the older post, for which I provided the link, to see how that would work ...
TestForBlog MyTest = new TestForBlog();
 
// First see if the Property "TestIt" exists:
if (MyReflectionClass.PemStatus(MyTest, "TestIt", "P"))
{
    // It does, let's grab the value, see if there's a method with that name and execute it if there is
    string MethodToTest = MyReflectionClass.GetPropertyValue(MyTest, "TestIt").ToString();
    if (MyReflectionClass.PemStatus(MyTest, MethodToTest, "M"))
        MyReflectionClass.ExecuteMethod(MyTest, MethodToTest);
}

That’s all there is to it. Hopefully you may find this useful!!

Happy coding!  =0)

Wednesday, April 27, 2016

Transport RSA Key Containers

We have a Message Bus application where messages are passed all around, from one application instance to another. This message traffic can be intranet, inter-domain or internet. We keep the messages safe from prying eyes by using RSA encryption (public keys are stored in databases).

Now, the whole RSA encryption thing is a topic for another day (or just Google for more info, in case I never get around to writing that blog post). Today, I’m going to talk about what to do if you need to move one of your existing Message Bus deployments to another computer for some reason (like the computer is old or dying and you need to replace it with new hardware).

This new computer will not have your RSA KeyContainer. Creating a new KeyContainer with the same name doesn’t get you the same keys, obviously. All your messaging traffic comes to a screeching halt, because now the messages can’t be decrypted with these different keys.

The solution is to copy your RSA KeyContainer to the new computer. I have a little utility application that I wrote that will read the RSA KeyContainer and write it to an XML file. The utility will also read from an XML file and create an RSA KeyContainer from it. In the UI of the utility, simply click one button to save the KeyContainer to the XML file. Click another button to create a KeyContainer from the XML file.

Here’s the basic code you need to do that (you can write the UI part of it yourself):

private string KeyContainerName = "Your.KeyContainer.Name";
private string DefaultFileName = "YourKeyContainer.xml";
private string SavedFileName = "";
private RSACryptoServiceProvider KeyContainer;

// call these from one of the two button clicks:
private void GetKeyContainerSaveToFile()
{
if (this.GetRSAContainer())
{
// save key container to a file
this.WriteDataToFile(this.KeyContainer.ToXmlString(true), this.DefaultFileName);
if (this.SavedFileName != "")
// Display to UI: "RSA KeyContainer saved to " + this.SavedFileName;
else
// Display to UI: "CANCELLED! RSA KeyContainer NOT saved to file";
}
}
private void GetXmlCreateKeyContainer()
{
if (this.GetRSAContainer())
{
// read key container XML from file and save it to the container
string key = this.ReadDataFromFile(this.DefaultFileName);
if (key != "")
{
this.KeyContainer.FromXmlString(key);
// Display to UI: "RSA KeyContainer info retrieved from file " + this.SavedFileName + " and saved to machine KeyContainer";
}
else
// Display to UI: "CANCELLED! RSA NOT retreived from file and NOT saved to machine";
}
}

// this will get an existing Container, or create a new Container if none exists
private bool GetRSAContainer()
{
try
{
// retrieve the key container
CspParameters cp = new CspParameters();
cp.KeyContainerName = this.KeyContainerName;
cp.Flags |= CspProviderFlags.UseMachineKeyStore;
this.KeyContainer = new RSACryptoServiceProvider(cp);
return true;
}
catch (Exception ex)
{
// Display the Exception to UI
return false;
}
}

// Write to an XML file:
private void WriteDataToFile(string Data, string FileName)
{
// Configure file dialog box
Microsoft.Win32.SaveFileDialog dlg = new Microsoft.Win32.SaveFileDialog();
dlg.FileName = FileName;
dlg.DefaultExt = ".xml"; // Default file extension
dlg.Filter = "XML documents (.xml)|*.xml"; // Filter files by extension

// Show save file dialog box
Nullable<bool> result = dlg.ShowDialog();

if (result == true)
{
this.SavedFileName = dlg.FileName;
using (StringReader sr = new StringReader(Data))
{
using (TextWriter tw = new StreamWriter(dlg.FileName))
{
string s = null;
while (true)
{
s = sr.ReadLine();
if (s != null)
tw.WriteLine(s);
else
break;
}
}
}
}
else
this.SavedFileName = "";
}

// Retrieve from the XML file
private string ReadDataFromFile(string FileName)
{
// Configure file dialog box
Microsoft.Win32.OpenFileDialog dlg = new Microsoft.Win32.OpenFileDialog();
dlg.FileName = FileName;
dlg.DefaultExt = ".xml"; // Default file extension
dlg.Filter = "XML documents (.xml)|*.xml"; // Filter files by extension

// Show open file dialog box
Nullable<bool> result = dlg.ShowDialog();

string xml = "";
if (result == true)
{
this.SavedFileName = dlg.FileName;
using (StreamReader file = new StreamReader(dlg.FileName))
{
xml = file.ReadToEnd();
}
}
else
this.SavedFileName = "";

return xml;
}


Saturday, March 26, 2016

DataBinding DataSets: Directly vs BindingSource

I’ve noticed some confusion over a blog post that I wrote many years ago. It can be found here: Fun With DataSets The post is about the various versions of DataRows (Proposed vs Current) and how a DataTable databound to a Control can sometimes not correctly report that DataSet.HasChanges() is true. The post was about how to get around that problem.

I was originally just going to add an Update to that post, for clarification, but decided that a new post would be better. (I will Update that old post, but only to add a link to this new one).

The clarification I want to make is that this appears to be an issue only if you DataBind directly to your DataTable, like this:

myDataGridView.DataSource = ds.Tables[0];  

However, I have not yet seen this happen when using a BindingSource, like this:

BindingSource bs = new BindingSource();
bs.DataSource = ds.Tables[0];  
myDataGridView.DataSource = bs;

To still play it safe, just in case, you could do the following (instead of the more “expensive” solution I suggested in my previous Fun With DataSets post):

// assuming the BindingSource is called "bs", and the DataSet is "ds"
bs.EndEdit();
if (ds.HasChanges())
{
    // process your DataSet changes
}

Why did I write that old post (back in 2009), when BindingSource had been introduced more than 3 years earlier? It had been new to .NET 2.0 and Visual Studio 2005, released in January 2006. One would assume that by 2009, developers would be using BindingSource all the time and would never have a reason to have to worry about this.  A little history of my background will explain this …

I started working with .NET 1.0 in April 2002 (it had been released only a month earlier). We had just begun a new startup company with plans to develop a new state-of-the-art .NET application. Along the way, we started developing our own “Framework” that would be used throughout our application. I was heavily involved with the “Framework” development. Unfortunately, 1.0 was a little buggy, so we were all glad when 1.1 came out a year later (with Visual Studio 2003). But, even 1.1 had its issues, one being the subject of these two posts. The CommitProposedChanges() method in the first post was born out of these issues.

Now, cut to 3 years later and the release of .NET 2.0 … well, by then I had written a full-blown “Framework” that had DataBinding “built-in” to a lot of the base control classes, not to mention a lot of new developers working on various applications who were used to using the “Framework” the way it was. I probably could have refactored the “Framework” classes to incorporate the use of BindingSource, but I’d have to be damn sure that it was backward compatible so as not to break existing code.  Nope … if it ain’t broke, don’t fix it! 

Now, judging from a lot of recent questions I’ve seen on various Forums, there are still plenty of people doing DataBinding the “old” way. So, even my old posts are still relevant! Gotta keep ‘em all happy!!  =0)

Saturday, February 13, 2016

Compare Speed of For vs Foreach

A bit more than 6 years ago, I posted Fun With DataSets, in which I wrote about the Proposed and Current versions of a DataRow. A code snippet in the post was iterating through each DataSet’s collection of DataTables and each DataTable’s collection of DataRows. I was using for loops. Years later, I have 30 comments on that post, but in the most recent thread of comments, we debated the merits of using for loops or foreach loops … which methodology is faster?

I had been under the impression, since way back in 2002 and the .NET 1.1 days, that the for was faster. It may have been true for 1.1 and I, unfortunately, held on to that belief all these years!! But my recent tests prove that wrong for all subsequent versions of .NET … foreach is much faster, roughly 7 to 8 times faster! I was curious if it was always so, and I changed the TargetFramework in my test application to be able to test all the way back to the 2.0 Framework (couldn’t get back as far as 1.1). And yes, it was the same with the 2.0 Framework … foreach wins each time!

Here is the code I used for testing this:

// First, I retreived a few thousand rows out of a database. 
// Then merged them together in a loop about 7 times,
// resulting in a DataSet called dsGlobal, containing about 2 million rows
this.GetALotOfData();

// Then, in a button click:
private void button1_Click(object sender, EventArgs e)
{
// Be sure to add a using System.Diagnostics;
Stopwatch oWatch = new Stopwatch();
string msg = "";

if (this.Toggle)
{
// Time the for loop
this.Toggle = false;
oWatch.Start();
msg = this.WithFor();
oWatch.Stop();
}
else
{
// Time the foreach loop
this.Toggle = true;
oWatch.Start();
msg = this.WithForEach();
oWatch.Stop();
}
this.txtResults.Text += string.Format("{0} in {1} milliseconds\r\n", msg, oWatch.ElapsedMilliseconds;
}
private string WithFor()
{
// In my original blog post that I referenced above, I looped like this:
for (int nTable = 0; nTable < this.dsGlobal.Tables.Count; nTable++)
{
for (int nRow = 0; nRow < this.dsGlobal.Tables[nTable].Rows.Count; nRow++)
{
if (this.dsGlobal.Tables[nTable].Rows[nRow].HasVersion(DataRowVersion.Proposed))
{
this.dsGlobal.Tables[nTable].Rows[nRow].EndEdit();
}
}
}
return "For";
}
private string WithForEach()
{
foreach (DataTable dt in this.dsGlobal.Tables)
foreach (DataRow row in dt.Rows)
{
if (row.HasVersion(DataRowVersion.Proposed))
row.EndEdit();
}
return "Foreach";
}

Each iteration I did was almost identical, time-wise, with for averaging right around 800 milliseconds and foreach averaging around 100 milliseconds, for almost 2 million rows (the DataSet contained only one DataTable). Here are the results:

 
Time using 1,943,936 Rows
For in 832 milliseconds
Foreach in 101 milliseconds
For in 768 milliseconds
Foreach in 102 milliseconds
For in 772 milliseconds
Foreach in 102 milliseconds
For in 858 milliseconds
Foreach in 103 milliseconds
For in 757 milliseconds
Foreach in 132 milliseconds
For in 750 milliseconds
Foreach in 102 milliseconds

Now, in the comment thread from that first post, my commenter was concerned about the way I was doing the for loop, thinking that there might be some overhead in having to check this.dsGlobal.Tables[nTable].Rows[nRow]  everytime through the loop. So I added another method to modify the for loop code, and tested it again:

private string WithForRowsCollection()
{
DataRowCollection rows;
for (int nTable = 0; nTable < this.dsGlobal.Tables.Count; nTable++)
{
rows = this.dsGlobal.Tables[nTable].Rows;
for (int nRow = 0; nRow < rows.Count; nRow++)
{
if (rows[nRow].HasVersion(DataRowVersion.Proposed))
{
rows[nRow].EndEdit();
}
}
}
return "For (rows)";
}

It didn't really make much difference. My original for loop tended toward 8 times slower, and the test with the modified for loop tends towards 7 times slower. Both sets of for loops (the original and the one using a row collection variable to iterate through), averaged between 700 - 800 milliseconds, with the first in the high 700's and the second in the low 700's.
 
Time using 1,943,936 Rows
For (rows) in 687 milliseconds
Foreach in 102 milliseconds
For (rows) in 670 milliseconds
Foreach in 112 milliseconds
For (rows) in 775 milliseconds
Foreach in 131 milliseconds
For (rows) in 675 milliseconds
Foreach in 102 milliseconds
For (rows) in 757 milliseconds
Foreach in 140 milliseconds
For (rows) in 763 milliseconds
Foreach in 103 milliseconds

Now, granted, we’re only talking about 700 to 800 hundred milliseconds here … but in any time-critical application, this could make a huge difference. And I’m not really talking about applications with User Interfaces … it’s not anything a user would notice at all. But, any kind of service application that runs standalone or in collaboration with other service applications (such as Windows Services types of applications that run server-side), it could be very important!

And now, I think I’ll go update my old Fun With DataSets post to include a link here and to post the new code!

Happy coding!  =0)

Wednesday, December 09, 2015

StackTrace Does The Detective Work

I had a really weird bug that I just could not track down. I have 3 Windows Service applications that “talk” to each other, passing data back and forth and logging bits of that data in each of their databases. In this particular case, the 3 apps have all the same code, just different config files (in real life there would be more differences between them, additional DLLs, but for testing there isn’t). The bug wasn’t causing any exceptions to be thrown, there was just an empty string being stored in the database for one particular “transaction” and that bit of data should have had some content,  not an empty string.

This is a rather tough situation to debug in Visual Studio, due to the fact that I’ve only got one of the apps running from VS and the other two are running from their EXEs (all as Console apps for testing, rather than as Windows Services). The one I have running from VS is not the one causing the issue, so I can’t easily debug without re-arranging all the config files so that I can run the problem one in VS. So, instead of doing that, I write the data out to XML files here and there throughout the process and I can look at the XML files to help find the problem. Unfortunately, this wasn’t working this time because all the data looked fine!

Consequently, I had no idea where this was going wrong. I only knew that right before the data was saved to the database (from a common Save method that is called throughout the app), that the Agency column was empty. Since this method is called from many places, it was going to be difficult to find which calling code caused the problem without debugging. Then I hit on an idea … maybe I could use StackTrace while the code was running! I didn’t know if that was possible, but it turns out that it is and it helped solve my perplexing bug. Here’s what I did:

In the DataAccess class method that saves the data, I check if the Agency string is null or empty and if so, I then call the following TroubleShootEmptyAgency() method:

private void TroubleShootEmptyAgency(string Agency, string Xml)
{
StringBuilder sb = new StringBuilder();
if (Agency == null)
sb.AppendLine("Null Agency");
else
sb.AppendLine("Empty Agency");

sb.AppendFormat("*** DataSet Content ***\r\n{0}\r\n***\r\n", Xml);

// Now, the fun part! The StackTrace!
System.Diagnostics.StackTrace stacktrace = new System.Diagnostics.StackTrace(true);
if (stacktrace != null)
{
sb.AppendLine("StackTrace:");
foreach (var frame in stacktrace.GetFrames())
{
sb.AppendFormat("{0}\n", frame.ToString());
}
}
// Write it to a File
System.IO.File.WriteAllText("EmptyAgency.xml", sb.ToString());
}

The StackTrace enabled me to see exactly what the call stack was that led up to the data trying to be saved with an empty Agency string. Consequently, I knew exactly which class and method was causing the problem (and then I clearly saw why that Agency string was empty there).

We all know that StackTrace can be very useful, but perhaps not everyone knows that it can be obtained from a running application. I didn’t know it until I tried it. Keep this in your bag of tricks, it just might come in handy!

Happy coding!

Wednesday, November 25, 2015

Information Exchange With Transformations

In the industry that I work in, Public Safety, there is a standard called NIEM, which stands for National Information Exchange Model (https://en.wikipedia.org/wiki/National_Information_Exchange_Model). It is an XML-based standard for sharing information between law enforcement agencies, first responders and other organizations. The various exchange scenarios are documented/described as XML schemas (.xsd’s), which are very complex schemas.

I have been using simple Typed DataSets since the early days of .NET (1.0 was buggy, 1.1 was much better). They are easy to fill from a database and easy to pass around between the layers/tiers of any application that I have written. However, to pass data from a law enforcement application to a fire department application, for example, there needs to be a common schema between them. Clearly, the simple Typed DataSet I use for a Police application will look totally different than the Typed DataSet I use for a Fire application. The way to solve this dilemma is to transmit the data with a common schema … NIEM to the rescue!

Transformations are used to Transform the Police DataSet to the common NIEM schema, then transmit the data as XML in the NIEM format to the Fire application, which then Transforms the NIEM XML data it has received into the schema for the Fire DataSet. That’s the process, in a nutshell.

In this blog post, I will show how to utilize Transformations to accomplish this. I am not going to show any of our actual Transformation schemas (.xslt) because that’s proprietary to our company. The NIEM schemas aren’t proprietary, but our own schemas are.

Also, I have to say that creating the .xslt’s without a tool is impossibly hard to do. I do not recommend it at all. I have used Altova’s MapForce and it is incredibly easy to use its graphic interface. I highly recommend it. Here is a link to information about MapForce, http://www.altova.com/mapforce.html, along with a link to what they call their Mission Kit, a whole toolbox of XML tools: http://www.altova.com/xml_tools.html (the Mission Kit includes MapForce and other goodies). They have free trial downloads if you want to try before you buy or just to experiment with. MapForce Basic Edition is $249. I do not work for Altova, I just use their MapForce product. There are probably other products on the market that will do the same thing, but you’d have to look for them yourself if you’re interested.

OK, now that that’s out of the way, let me show you some code. This is a simplified structure, just to show you the basic concepts of how to do this. In reality, I have many more Transformations and store them in a Dictionary. For the example I’m showing you here, I have two Typed DataSets, (PoliceDataSet and FireDataSet), and the common schema that I’m calling FakeCommonInformation (which would be something like the NIEM schema I mentioned above, if this were a real application).

Here are the two Transform variables and two methods to call to do the Transformations that I’ll be using in this sample code:

// be sure you have a using System.Xml.Xsl;
private XslCompiledTransform txToCommon;
private XslCompiledTransform txToLocal;

public string TransformToCommon(string Xml)
{
StringReader sr = new StringReader(Xml);
XmlReader xr = new XmlTextReader(sr);
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
xw.Formatting = Formatting.Indented;

this.txToCommon.Transform(xr, xw);
return sw.ToString();
}
public string TransformToLocal(string Xml)
{
StringReader sr = new StringReader(Xml);
XmlReader xr = new XmlTextReader(sr);
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
xw.Formatting = Formatting.Indented;

this.txToLocal.Transform(xr, xw);
return sw.ToString();
}

Here’s the code to initialize those Transform variables. Notice that they are loaded from individual .xslt files.

// Instantiate and Load Transformations
this.txToCommon = new XslCompiledTransform();
this.txToCommon.Load(@"..\..\PoliceDataSet-to-PartsCommon.xslt");

this.txToLocal = new XslCompiledTransform();
this.txToLocal.Load(@"..\..\PartsCommon-to-FireDataSet.xslt"); // path and filename

There are ways to actually compile all your .xslt files into one DLL, but I think that I’ll leave that for the next post I write.

All these schemas are not really Police, Fire or NIEM … it’s just that I had these test files lying around and re-used them for this post. Let’s just pretend that the columns and data contained in the DataSets actually have something to do with Public Safety. And here’s the scenario we’ll be mimicking with this sample code:

  • A Police application gets data from it’s database and fills the PoliceDataSet.
  • The application now needs to share this information with the Fire application.
  • The txToCommon Transformation is used to transform the data in the PoliceDataSet into an XML string that represents the Common format.
  • That Common XML string is transmitted over the wire (via a Web Service or other mechanisms) to the Fire application.
  • The Fire application uses the txToLocal Transformation to transform the Common XML to an XML string that represents the FireDataSet and the new XML is used to fill the FireDataSet with data it can use.

Now, I’m going to show you the schemas I used to do all this, so that you can take the pieces and put them in your own test application to see how well it works.

First, here’s the PoliceDataSet.xsd :

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="PoliceDataSet">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="code" type="xs:string"/>
<xs:element name="name" type="xs:string" minOccurs="0"/>
<xs:element name="type" type="xs:integer" minOccurs="0"/>
<xs:element name="deleted" type="xs:integer" minOccurs="0"/>
<xs:element name="cost" type="xs:decimal" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>

Next, the FireDataSet.xsd:

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="FireDataSet">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="FireCode" type="xs:string"/>
<xs:element name="FireName" type="xs:string" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>

Notice the difference between the two DataSets. The Police has 5 columns, whereas the Fire only has 2 columns. In the Police, two of the columns are “code” and “name”, whereas in the Fire, those same two entities are called “FireCode” and “FireName”. The Transformations take care of matching those up.

And, of course, we can’t forget the Common schema (the actual file is called PartsCommon.xsd):

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="FakeCommonInformation">
<xs:annotation>
<xs:documentation> Hello World</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="Parts" minOccurs="0">
<xs:annotation>
<xs:documentation>Parts Needed</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="code" type="xs:string"/>
<xs:element name="name" type="xs:string" minOccurs="0"/>
<xs:element name="type" type="xs:integer" minOccurs="0">
<xs:annotation>
<xs:documentation>0 (Zero) for single ingredient and 1 (One) for recipe</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="deleted" type="xs:integer" minOccurs="0"/>
<xs:element name="cost" type="xs:decimal" minOccurs="0"/>
<xs:element name="messageID" type="xs:string" minOccurs="0"/>
<xs:element name="quantityUnitsId" type="xs:integer" minOccurs="0"/>
<xs:element name="stockUnit" type="xs:string" minOccurs="0"/>
<xs:element name="UnitofMeasure" type="xs:string" minOccurs="0"/>
<xs:element name="tolerance" minOccurs="0" maxOccurs="1">
<xs:complexType>
<xs:sequence>
<xs:element name="toleranceType" type="xs:integer"/>
<xs:sequence>
<xs:element name="banding" minOccurs="1" maxOccurs="unbounded">
<xs:complexType>
<xs:choice>
<xs:sequence>
<xs:element name="lowerTolerance" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
<xs:element name="upperTolerance" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
<xs:element name="band" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
</xs:sequence>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Combinations" minOccurs="0">
<xs:annotation>
<xs:documentation>Mix combinations of Parts.</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence maxOccurs="unbounded">
<xs:element name="Combination">
<xs:complexType>
<xs:sequence>
<xs:element name="partCode" type="xs:string">
<xs:annotation>
<xs:documentation>Validated against a Part entry that represents a Combination</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="locationCode" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>Must previously have been entered using</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="yield" type="xs:integer" minOccurs="0"/>
<xs:element name="minProducts" type="xs:integer" minOccurs="0"/>
<xs:element name="maxProducts" type="xs:integer" minOccurs="0"/>
<xs:element name="addToStock" type="xs:integer" minOccurs="0"/>
<xs:element name="expiryDays" type="xs:double" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="version" type="xs:decimal" use="required" fixed="2.1"/>
</xs:complexType>
</xs:element>
</xs:schema>

As you can see, this schema looks absolutely nothing like typical DataSet schema. It’s much more complex and more hierarchical than simple tables and rows of columns in each table.

So, what do the Transformation .xslt’s look like?

Here’s the PoliceDataSet-to-PartsCommon.xslt:

<?xml version="1.0" encoding="UTF-8"?>
<!--
This file was generated by Altova MapForce 2012r2sp1

YOU SHOULD NOT MODIFY THIS FILE, BECAUSE IT WILL BE
OVERWRITTEN WHEN YOU RE-RUN CODE GENERATION.

Refer to the Altova MapForce Documentation for further details.
http://www.altova.com/mapforce
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match="/">
<FakeCommonInformation>
<xsl:attribute name="xsi:noNamespaceSchemaLocation" namespace="http://www.w3.org/2001/XMLSchema-instance">PartsCommon.xsd</xsl:attribute>
<Parts>
<xsl:for-each select="PoliceDataSet/Part">
<Part>
<code>
<xsl:value-of select="string(code)"/>
</code>
<xsl:for-each select="name">
<name>
<xsl:value-of select="string(.)"/>
</name>
</xsl:for-each>
<xsl:for-each select="type">
<type>
<xsl:value-of select="string(number(string(.)))"/>
</type>
</xsl:for-each>
<xsl:for-each select="deleted">
<deleted>
<xsl:value-of select="string(number(string(.)))"/>
</deleted>
</xsl:for-each>
<xsl:for-each select="cost">
<cost>
<xsl:value-of select="string(number(string(.)))"/>
</cost>
</xsl:for-each>
</Part>
</xsl:for-each>
</Parts>
</FakeCommonInformation>
</xsl:template>
</xsl:stylesheet>

And the PartsCommon-to-FireDataSet.xslt:

<?xml version="1.0" encoding="UTF-8"?>
<!--
This file was generated by Altova MapForce 2012r2sp1

YOU SHOULD NOT MODIFY THIS FILE, BECAUSE IT WILL BE
OVERWRITTEN WHEN YOU RE-RUN CODE GENERATION.

Refer to the Altova MapForce Documentation for further details.
http://www.altova.com/mapforce
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match="/">
<FireDataSet>
<xsl:attribute name="xsi:noNamespaceSchemaLocation" namespace="http://www.w3.org/2001/XMLSchema-instance">FireDataSet.xsd</xsl:attribute>
<xsl:for-each select="FakeCommonInformation/Parts/Part">
<Part>
<FireCode>
<xsl:value-of select="string(code)"/>
</FireCode>
<xsl:for-each select="name">
<FireName>
<xsl:value-of select="string(.)"/>
</FireName>
</xsl:for-each>
</Part>
</xsl:for-each>
</FireDataSet>
</xsl:template>
</xsl:stylesheet>

And now, the code to run through the above scenario:

1) A Police application gets data from it’s database and fills the PoliceDataSet (I’m just going to add 2 rows manually):

// Fill the DataSet (from database or wherever);
PoliceDataSet dsPolice = new PoliceDataSet();
var row = dsPolice.Part.NewPartRow();
row.code = "S000510";
row.name = "My New Part";
row.type = 0;
row.cost = 5.5000M;
dsPolice.Part.AddPartRow(row);

row = dsPolice.Part.NewPartRow();
row.code = "S000610";
row.name = "Your New Part";
row.type = 1;
row.cost = 6.6660M;
dsPolice.Part.AddPartRow(row);

2) Now the Police application needs to share this information with the Fire application, so we’ll use the txToCommon Transformation to transform the data in the PoliceDataSet into an XML string that represents the Common format:

string xmlCommon = this.TransformToCommon(dsPolice.GetXml());

3) That xmlCommon string can now be transmitted over the wire (via a Web Service or other mechanisms) to the Fire application.

4) The Fire application uses the txToLocal Transformation to transform the Common XML that it has received remotely, to an XML string that represents the FireDataSet and the new XML is used to fill the FireDataSet with data the Fire application can now use.

// Transform the Common XML string that you receive "over the wire" 
// to an XML string that represents the FireDataSet schema.
string xmlFire = this.TransformToLocal(receivedCommonXml);

// Use the Xml to fill a new FireDataSet, so your Fire application can now use the data
FireDataSet dsFire = new FireDataSet();
StringReader sr = new StringReader(xmlFire);
dsFire.ReadXml(sr, XmlReadMode.Fragment);

Take a look at the string returned from dsFire.GetXml() once you’re done and you’ll see that the code column has now become FireCode and the name column has now become FireName. It should look like this:

<FireDataSet>
<Part>
<FireCode>S000510</FireCode>
<FireName>My New Part</FireName>
</Part>
<Part>
<FireCode>S000610</FireCode>
<FireName>Your New Part</FireName>
</Part>
</FireDataSet>

Your Transformation is now complete and you’ve come over to the Dark Side (a lame attempt at Stars Wars humor).

Happy coding!  :0)