In the industry that I work in, Public Safety, there is a standard called NIEM, which stands for National Information Exchange Model (https://en.wikipedia.org/wiki/National_Information_Exchange_Model). It is an XML-based standard for sharing information between law enforcement agencies, first responders and other organizations. The various exchange scenarios are documented/described as XML schemas (.xsd’s), which are very complex schemas.
I have been using simple Typed DataSets since the early days of .NET (1.0 was buggy, 1.1 was much better). They are easy to fill from a database and easy to pass around between the layers/tiers of any application that I have written. However, to pass data from a law enforcement application to a fire department application, for example, there needs to be a common schema between them. Clearly, the simple Typed DataSet I use for a Police application will look totally different than the Typed DataSet I use for a Fire application. The way to solve this dilemma is to transmit the data with a common schema … NIEM to the rescue!
Transformations are used to Transform the Police DataSet to the common NIEM schema, then transmit the data as XML in the NIEM format to the Fire application, which then Transforms the NIEM XML data it has received into the schema for the Fire DataSet. That’s the process, in a nutshell.
In this blog post, I will show how to utilize Transformations to accomplish this. I am not going to show any of our actual Transformation schemas (.xslt) because that’s proprietary to our company. The NIEM schemas aren’t proprietary, but our own schemas are.
Also, I have to say that creating the .xslt’s without a tool is impossibly hard to do. I do not recommend it at all. I have used Altova’s MapForce and it is incredibly easy to use its graphic interface. I highly recommend it. Here is a link to information about MapForce, http://www.altova.com/mapforce.html, along with a link to what they call their Mission Kit, a whole toolbox of XML tools: http://www.altova.com/xml_tools.html (the Mission Kit includes MapForce and other goodies). They have free trial downloads if you want to try before you buy or just to experiment with. MapForce Basic Edition is $249. I do not work for Altova, I just use their MapForce product. There are probably other products on the market that will do the same thing, but you’d have to look for them yourself if you’re interested.
OK, now that that’s out of the way, let me show you some code. This is a simplified structure, just to show you the basic concepts of how to do this. In reality, I have many more Transformations and store them in a Dictionary. For the example I’m showing you here, I have two Typed DataSets, (PoliceDataSet and FireDataSet), and the common schema that I’m calling FakeCommonInformation (which would be something like the NIEM schema I mentioned above, if this were a real application).
Here are the two Transform variables and two methods to call to do the Transformations that I’ll be using in this sample code:
// be sure you have a using System.Xml.Xsl;
private XslCompiledTransform txToCommon;
private XslCompiledTransform txToLocal;
public string TransformToCommon(string Xml)
{
StringReader sr = new StringReader(Xml);
XmlReader xr = new XmlTextReader(sr);
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
xw.Formatting = Formatting.Indented;
this.txToCommon.Transform(xr, xw);
return sw.ToString();
}
public string TransformToLocal(string Xml)
{
StringReader sr = new StringReader(Xml);
XmlReader xr = new XmlTextReader(sr);
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
xw.Formatting = Formatting.Indented;
this.txToLocal.Transform(xr, xw);
return sw.ToString();
}
Here’s the code to initialize those Transform variables. Notice that they are loaded from individual .xslt files.
// Instantiate and Load Transformations
this.txToCommon = new XslCompiledTransform();
this.txToCommon.Load(@"..\..\PoliceDataSet-to-PartsCommon.xslt");
this.txToLocal = new XslCompiledTransform();
this.txToLocal.Load(@"..\..\PartsCommon-to-FireDataSet.xslt"); // path and filename
There are ways to actually compile all your .xslt files into one DLL, but I think that I’ll leave that for the next post I write.
All these schemas are not really Police, Fire or NIEM … it’s just that I had these test files lying around and re-used them for this post. Let’s just pretend that the columns and data contained in the DataSets actually have something to do with Public Safety. And here’s the scenario we’ll be mimicking with this sample code:
- A Police application gets data from it’s database and fills the PoliceDataSet.
- The application now needs to share this information with the Fire application.
- The txToCommon Transformation is used to transform the data in the PoliceDataSet into an XML string that represents the Common format.
- That Common XML string is transmitted over the wire (via a Web Service or other mechanisms) to the Fire application.
- The Fire application uses the txToLocal Transformation to transform the Common XML to an XML string that represents the FireDataSet and the new XML is used to fill the FireDataSet with data it can use.
Now, I’m going to show you the schemas I used to do all this, so that you can take the pieces and put them in your own test application to see how well it works.
First, here’s the PoliceDataSet.xsd :
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="PoliceDataSet">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="code" type="xs:string"/>
<xs:element name="name" type="xs:string" minOccurs="0"/>
<xs:element name="type" type="xs:integer" minOccurs="0"/>
<xs:element name="deleted" type="xs:integer" minOccurs="0"/>
<xs:element name="cost" type="xs:decimal" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
Next, the FireDataSet.xsd:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="FireDataSet">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="FireCode" type="xs:string"/>
<xs:element name="FireName" type="xs:string" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
Notice the difference between the two DataSets. The Police has 5 columns, whereas the Fire only has 2 columns. In the Police, two of the columns are “code” and “name”, whereas in the Fire, those same two entities are called “FireCode” and “FireName”. The Transformations take care of matching those up.
And, of course, we can’t forget the Common schema (the actual file is called PartsCommon.xsd):
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="FakeCommonInformation">
<xs:annotation>
<xs:documentation> Hello World</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="Parts" minOccurs="0">
<xs:annotation>
<xs:documentation>Parts Needed</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence maxOccurs="unbounded">
<xs:element name="Part">
<xs:complexType>
<xs:sequence>
<xs:element name="code" type="xs:string"/>
<xs:element name="name" type="xs:string" minOccurs="0"/>
<xs:element name="type" type="xs:integer" minOccurs="0">
<xs:annotation>
<xs:documentation>0 (Zero) for single ingredient and 1 (One) for recipe</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="deleted" type="xs:integer" minOccurs="0"/>
<xs:element name="cost" type="xs:decimal" minOccurs="0"/>
<xs:element name="messageID" type="xs:string" minOccurs="0"/>
<xs:element name="quantityUnitsId" type="xs:integer" minOccurs="0"/>
<xs:element name="stockUnit" type="xs:string" minOccurs="0"/>
<xs:element name="UnitofMeasure" type="xs:string" minOccurs="0"/>
<xs:element name="tolerance" minOccurs="0" maxOccurs="1">
<xs:complexType>
<xs:sequence>
<xs:element name="toleranceType" type="xs:integer"/>
<xs:sequence>
<xs:element name="banding" minOccurs="1" maxOccurs="unbounded">
<xs:complexType>
<xs:choice>
<xs:sequence>
<xs:element name="lowerTolerance" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
<xs:element name="upperTolerance" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
<xs:element name="band" type="xs:decimal" minOccurs="1" maxOccurs="1"/>
</xs:sequence>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Combinations" minOccurs="0">
<xs:annotation>
<xs:documentation>Mix combinations of Parts.</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence maxOccurs="unbounded">
<xs:element name="Combination">
<xs:complexType>
<xs:sequence>
<xs:element name="partCode" type="xs:string">
<xs:annotation>
<xs:documentation>Validated against a Part entry that represents a Combination</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="locationCode" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>Must previously have been entered using</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="yield" type="xs:integer" minOccurs="0"/>
<xs:element name="minProducts" type="xs:integer" minOccurs="0"/>
<xs:element name="maxProducts" type="xs:integer" minOccurs="0"/>
<xs:element name="addToStock" type="xs:integer" minOccurs="0"/>
<xs:element name="expiryDays" type="xs:double" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="version" type="xs:decimal" use="required" fixed="2.1"/>
</xs:complexType>
</xs:element>
</xs:schema>
As you can see, this schema looks absolutely nothing like typical DataSet schema. It’s much more complex and more hierarchical than simple tables and rows of columns in each table.
So, what do the Transformation .xslt’s look like?
Here’s the PoliceDataSet-to-PartsCommon.xslt:
<?xml version="1.0" encoding="UTF-8"?>
<!--
This file was generated by Altova MapForce 2012r2sp1
YOU SHOULD NOT MODIFY THIS FILE, BECAUSE IT WILL BE
OVERWRITTEN WHEN YOU RE-RUN CODE GENERATION.
Refer to the Altova MapForce Documentation for further details.
http://www.altova.com/mapforce
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match="/">
<FakeCommonInformation>
<xsl:attribute name="xsi:noNamespaceSchemaLocation" namespace="http://www.w3.org/2001/XMLSchema-instance">PartsCommon.xsd</xsl:attribute>
<Parts>
<xsl:for-each select="PoliceDataSet/Part">
<Part>
<code>
<xsl:value-of select="string(code)"/>
</code>
<xsl:for-each select="name">
<name>
<xsl:value-of select="string(.)"/>
</name>
</xsl:for-each>
<xsl:for-each select="type">
<type>
<xsl:value-of select="string(number(string(.)))"/>
</type>
</xsl:for-each>
<xsl:for-each select="deleted">
<deleted>
<xsl:value-of select="string(number(string(.)))"/>
</deleted>
</xsl:for-each>
<xsl:for-each select="cost">
<cost>
<xsl:value-of select="string(number(string(.)))"/>
</cost>
</xsl:for-each>
</Part>
</xsl:for-each>
</Parts>
</FakeCommonInformation>
</xsl:template>
</xsl:stylesheet>
And the PartsCommon-to-FireDataSet.xslt:
<?xml version="1.0" encoding="UTF-8"?>
<!--
This file was generated by Altova MapForce 2012r2sp1
YOU SHOULD NOT MODIFY THIS FILE, BECAUSE IT WILL BE
OVERWRITTEN WHEN YOU RE-RUN CODE GENERATION.
Refer to the Altova MapForce Documentation for further details.
http://www.altova.com/mapforce
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match="/">
<FireDataSet>
<xsl:attribute name="xsi:noNamespaceSchemaLocation" namespace="http://www.w3.org/2001/XMLSchema-instance">FireDataSet.xsd</xsl:attribute>
<xsl:for-each select="FakeCommonInformation/Parts/Part">
<Part>
<FireCode>
<xsl:value-of select="string(code)"/>
</FireCode>
<xsl:for-each select="name">
<FireName>
<xsl:value-of select="string(.)"/>
</FireName>
</xsl:for-each>
</Part>
</xsl:for-each>
</FireDataSet>
</xsl:template>
</xsl:stylesheet>
And now, the code to run through the above scenario:
1) A Police application gets data from it’s database and fills the PoliceDataSet (I’m just going to add 2 rows manually):
// Fill the DataSet (from database or wherever);
PoliceDataSet dsPolice = new PoliceDataSet();
var row = dsPolice.Part.NewPartRow();
row.code = "S000510";
row.name = "My New Part";
row.type = 0;
row.cost = 5.5000M;
dsPolice.Part.AddPartRow(row);
row = dsPolice.Part.NewPartRow();
row.code = "S000610";
row.name = "Your New Part";
row.type = 1;
row.cost = 6.6660M;
dsPolice.Part.AddPartRow(row);
2) Now the Police application needs to share this information with the Fire application, so we’ll use the txToCommon Transformation to transform the data in the PoliceDataSet into an XML string that represents the Common format:
string xmlCommon = this.TransformToCommon(dsPolice.GetXml());
3) That xmlCommon string can now be transmitted over the wire (via a Web Service or other mechanisms) to the Fire application.
4) The Fire application uses the txToLocal Transformation to transform the Common XML that it has received remotely, to an XML string that represents the FireDataSet and the new XML is used to fill the FireDataSet with data the Fire application can now use.
// Transform the Common XML string that you receive "over the wire"
// to an XML string that represents the FireDataSet schema.
string xmlFire = this.TransformToLocal(receivedCommonXml);
// Use the Xml to fill a new FireDataSet, so your Fire application can now use the data
FireDataSet dsFire = new FireDataSet();
StringReader sr = new StringReader(xmlFire);
dsFire.ReadXml(sr, XmlReadMode.Fragment);
Take a look at the string returned from dsFire.GetXml() once you’re done and you’ll see that the code column has now become FireCode and the name column has now become FireName. It should look like this:
<FireDataSet>
<Part>
<FireCode>S000510</FireCode>
<FireName>My New Part</FireName>
</Part>
<Part>
<FireCode>S000610</FireCode>
<FireName>Your New Part</FireName>
</Part>
</FireDataSet>
Happy coding! :0)
