Understanding BizTalk Map Process Flow

October 9th, 2012

Several blogs on BizTalk that I have read recently state that the BizTalk mapper is a top-down mapper. A top-down mapper starts processing at the first node and progresses downward, node-by-node, through the output schema. Stating that the BizTalk mapper is a top-down mapper is misleading because that is not technically accurate. Lets look at some examples.

 

 

First we build a simple schema that we will use in the map. As you can see in the picture the schema consists of a root node,one record node, and two child elements in the record. We then attach an Uppercase functoid to the record and both elements. The first Uppercase functoid contains the value "1", the second the value "2", and the third the value "3". We turn off output validation for the map so that we can output text in the record node Parent.

When we run the map the output appears as seen here:

 

 

Notice that the value "1" appears after the values "2" and "3" in the output. This is the method of outputting values in a record, so the fact that the "1" appears in the output after the "2" and "3" values doesn't indicate that the execution order was anything other than "Parent", "ChildOne", "ChildTwo".

 

 

We add one node to the schema (Globals) to get us a place to attach a script that defines a global integer and sets the value to zero. Here is the code in that scriptoid.

 

int counter = 0;
public void Globals ()

{}

We use an Equals functoid with the second value set to "false" to block the output of the Globals node. Since the scriptoid attached to the Equals functoid outputs no value, the Globals node will not be output.

The scriptoid attached to the node Parent contains this script:

public int incCounter ()
{
return ++counter;
}

The script increments the counter and outputs the new value to the node Parent. Since the counter was initialized to zero, we expect the value to be output here to be "1".

Two more scriptoids are attached to the nodes ChildOne and ChildTwo. Both contain the code:

public int retCounter ()
{
return counter;
}

Since the counter was incremented in the scriptoid attached to the node Parent, we expect the value output in both these nodes to be "1" as well. Here is the output of the map:

 

 

Oops. This is not what we expected. Both child nodes contain the value "0", which is the initialized value of the variable. The record node outputs the value "1" indicating that the scriptoid attached to the node Parent did work. This is a clear indication that the code attached to the record node did not execute until after the code in the child nodes.

But.... Not all aspects of the record are executed after the children. Here is another map, identical to the last one except that we've added an Equals functoid (values "1" and "2") that will always return "false", thus blocking the output of the record Parent.

 

 

 

The output of this map is:

 

 

Now the record Parent and all of the children of Parent have been blocked. In this map the Equals functoid must have been executed before the Parent node or the child nodes.

Bottom Line:

As a rule, links to child nodes are executed before connections to their parent nodes. An exception to that rule is that connections which control the output of a parent node are executed before connections to the child nodes.

BizTalk EDI Validation Errors: Invalid Character

October 4th, 2012

Here's another error about which we get a lot of calls. The reason many folks have trouble identifying the cause of the error is because the error report

from BizTalk, while accurate in fact, is misleading in context. The event viewer shows two entries for this error, drilling down to the error message in the suspended message in the BT Administration Group overview shows the same message.

Description: A message received by adapter "FILE" on receive location "TestIB" with URI "C:\TestIB\*.edi" is suspended.

Error details: An output message of the component "Unknown " in receive pipeline "Microsoft.BizTalk.Edi.DefaultPipelines.EdiReceive,

Microsoft.BizTalk.Edi.EdiPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" is suspended due to the following error.

Error encountered during parsing. The X12 transaction set with id '0029' contained in functional group with id '29', in interchange with id '000000029',

with sender id 'BTS-SENDER ', receiver id 'RECEIVE-PARTNER' is being suspended with following errors:

Error: 1 (Field level error)

SegmentID: LIN

Position in TS: 26

Data Element ID: LIN03

Position in Segment: 3

Data Value:

6: Invalid character in data element

What do these errors say? Well the error is in an element (1 (Field level error) in an LIN segment (SegmentID: LIN). The LIN with the error is the 26th segment in the transaction (Position in TS: 26), counting from and including the ST segment. The error occurs in the LIN03 element (Data Element ID: LIN03) which is the third element in the LIN segment (Position in Segment: 3).

So far so good, but the next two lines are confusing. They appear to say that the Data Value is 6, which is an invalid character. Actually no data value is given, as the line should read Data Value: "", Data Value: Unknown, or some statement that makes clear that no value is being given. The last line (6: Invalid character in data element) actually identifies the specific error message (message 6, which is "Invalid character in data element").

Let's look at the segment in question. It appears:

LIN*2*IN*CBPOLINEITEM:NO1*VN*210RNU*UP*784231909492~

The error messages indicate that the problem is in the third element (CBPOLINEITEM:NO1) and that the problem is that one of the characters is invalid. But all of the characters in that element are valid X12 characters for text. So what's the problem? To see the cause of the problem we have to look at the ISA segment from the interchange in which the transaction was received. Here it is:

ISA*00* *00* *ZZ*BTS-SENDER *ZZ*RECEIVE-PARTNER*121003*1517*U*00401*000000029*0*T*:~

The ISA16 element, the last element before the segment separator ("~") is the element that defines the component element separator to be used in all transactions in this interchange. In this ISA, the element separator is defined as a colon (":"). Since the LIN03 is not a composite element, it may not contain a colon. That is the problem in this message.


Understanding X12 error messages in BizTalk

September 20th, 2012

Understanding errors that occur when parsing X12 messages is   difficult even if you understand X12 syntax rules, and extremely   difficult when you have little or no experience with X12 standards.

This blog entry examines a few common errors that occur when parsing  X12 messages in BizTalk and explains why they occur and offers possible   solutions.   We use the properly formed transaction below, an X12 855 PO Acknowledgement.   This is a simple instance of an X12 message.

ST*855*000000001
BAK*00*AD*P439254*20100204
PO1*1*4*CA***IN*5710503
PID*F****BULK S1050
PO1*2*25*CA***IN*5710503
PID*F****BULK S1050
PO1*3*8*CA***IN*5710503
PID*F****BULK S1050
PO1*4*16*CA***IN*5710503
PID*F****BULK S1050
PO1*5*1*CA***IN*5710503
PID*F****MT ELITE S309
PO1*6*1*CA***IN*5710503
PID*F****NO POWDER S860
CTT*6
SE*16*000000001

ERROR ONE

The first error we will introduce is one of the most, if not the most, common parsing error occurring with X12 -- the Invalid Code error.  To cause the error, we will change the BAK segment by replacing the valid code "00" with the invalid code "xx".

BAK*xx*AD*P439254*20100204

When we pass the modified 855 through the EDI pipeline we get the error message:

Invoking component...
LINE 1: E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855badcode.edi: error   BEC2004: Non Segment level: [5] >X12_00401_855\\BAK\\BAK01\\ The   element has an invalid value according to its data type.

LINE 2: E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855badcode.edi:   error BEC2004: Segment level  : Segment ID - BAK : Position - 2 [8]   Segment Has Data Element Errors

LINE 3: E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855badcode.edi:   error BEC2004: Segment level  : Field Error : Segment ID - BAK : Field   Position - 1 Current value - xx [7] Invalid code value

LINE 4: E:\\SOLUTIONS\\BLOG\\X12Errors\\X12_00401_855.xsd: error   BEC2004: Validate Instance failed for schema X12_00401_855.xsd, file:   <file:///E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855badcode.edi>

Component invocation succeeded.

Line numbers (e.g., LINE 1:) were added for clarity.

Lines 1-3 of the error each contain a number in brackets -- [5], [8], and [7] to be specific.  These numbers identify the error message string. They   do not apply to the X12 transaction, nor are they useful in resolving   the error.

LINE 1 tells us that the error is at the segment level ("Non Segment level".   This means that the problem is with an X12 element.  LINE 1 also identifies the name of the element in which the error occurs ("BAK01") and indicates that element contains an ("invalid value according to its data type").  This is a misleading statement, since the value in the BAK01 is the correct data type.

LINE 2 identifies the specific BAK segment that contains the error by giving the position of the segment in the message ("BAK : Position - 2").  This indicates that the error occurred in the second segment in the message.

LINE 3 gives us the specific error.   The element affected is the first element in the segment ("Field Postion - 1") .  The error is that the value in that element is invalid ("[7] Invalid Code Value").   Finally, the error indicates that the value in the element is ("xx").

LINE 4 is extraneous text that serves no real value as it provides no information pertinent to the error (except the filename path, which is also given in every other line).

Fundamentally then, this error listing says this 855 message contained a BAK segment in the second line of the message which contained an invalid code value of ("xx") in the first element (BAK01.)

So what does that mean?  In X12, some elements are limited as to which values they may contain. The BAK01 is one such element. You can see the acceptable values by looking at the (Collection) list under Enumeration in the element properties in the schema. Your choices for repairing this problem are either to put a valid value in that field in the message or to add the invalid value to the code list.


ERROR TWO

Now let''s remove the BSN segment and see what happens.  The message now appears:

ST*855*000000001
PO1*1*4*CA***IN*5710503
PID*F****BULK S1050
PO1*2*25*CA***IN*5710503
PID*F****BULK S1050
PO1*3*8*CA***IN*5710503
PID*F****BULK S1050
PO1*4*16*CA***IN*5710503
PID*F****BULK S1050
PO1*5*1*CA***IN*5710503
PID*F****MT ELITE S309
PO1*6*1*CA***IN*5710503
PID*F****NO POWDER S860
CTT*6
SE*16*000000001

When we validate the this 855 we get the following error:

Invoking component...

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855 missingseg.edi: error BEC2004: Non Segment level : [4] Number of included segments do not match

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855 missingseg.edi:   error BEC2004: Non Segment level: [5] X12_00401_855\\ The element   ''X12_00401_855'' in namespace ''http://schemas.microsoft.com/BizTalk/EDI/X12/2006'' has invalid child element ''PO1Loop1'' in   namespace ''http://schemas.microsoft.com/BizTalk/EDI/X12/2006''. List of possible elements expected:   ''http://schemas.microsoft.com/BizTalk/EDI/X12/2006:BAK''.

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855 missingseg.edi: error BEC2004: Segment level: Segment ID - BAK : Position - 2 : [3] Mandatory Segment Missing

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855 missingseg.edi: error BEC2004: Segment level  : Segment ID - PO1 : Position - 2 : [2] Unexpected segment

E:\\SOLUTIONS\\BLOG\\X12Errors\\X12_00401_855.xsd:   error BEC2004: Validate Instance failed for schema X12_00401_855.xsd,  file:   file:///E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855missingseg.edi>

Component invocation succeeded.

Frankly, this error message is difficult to interpret because only one of the five lines actually refers to the specific issue that caused this error.

Line 1 tells us that an error occurred at the element level ("Non Segment level") and that the error was ("[4] Number of included segments do not match").  The error message does not tell you what caused this or where the problem is.

You must understand X12 syntax in order to interpret this statement. X12 syntax requires the SE01 element to contain the count of  the number of segments in the message, from ST to SE, inclusive.  In this case, the SE01 contains the value ("16") but due to the missing BSN segment there are only fifteen segments in the message

Line 2 adds to the confusion as it also reports a secondary error, ("invalid child element ''POLoop1''").  The PO1Loop1 element is not a valid construct in the X12 message but is a node in the XML schema. Here is where an X12 knowledgeable person may become confused as this error does not reference the contents of the   855

At this point, nothing has pointed to the exact error in the 855.  Line 3 is the first line that does so.  Line 3 states precise information about the error.  First, the type of error is identified as ("[3] Mandatory Segment Missing").   Second, the location of the missing segment is given as ("Position - 2"),  indicating that the missing segment should have been the second line in the 855.  Finally the name of the missing segment ("BAK") is provided.

Line 4 reports another secondary error, the X12 version of the XML error reported in Line 2. Line 4 tells us that the PO1 segment found in line two 0f the 855 should not be there ("Segment ID - PO1 : Position - 2 :  [2] Unexpected segment").

<p>Line five is superfluous.

Why the confusing list of errors?  Well, BizTalk tries to report all errors found in a message.  Some engines stop parsing at the first  error; others are able to report all errors correctly. BizTalk does not take into account previously discovered errors when continuing to parse the message and report errors.  Once the mandatory BAK segment is found to be missing, BizTalk should report this error and then continue to parse the 855 as if the BAK were there.

Note that an error such as this must be reported to the sending party so that they can resubmit the 855.

ERROR THREE

Finally, let''s modify the fifth PID segment in the 855 so that a mandatory element is missing.  The new PID is:

PID*****MT ELITE S309

And here is the error produced when we run the 855 with this segment:

Invoking component...

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855missingele.edi: error BEC2004: Non Segment level&nbsp; : [5] X12_00401_855\\PO1Loop1\\PIDLoop1\\PID_2\\ The element ''PID_2'' in namespace ''http://schemas.microsoft.com/BizTalk/EDI/X12/2006'' has invalid child element ''PID05''. List of possible elements expected: ''PID01''.

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855missingele.edi: error BEC2004: Segment level: Segment ID - PID : Position - 12 : [8] Segment Has Data Element Errors

E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855missingele.edi: error BEC2004: Segment level: Field Error : Segment ID - PID : Field Position - 1 : Current value - : [1] Mandatory data element missing

E:\\SOLUTIONS\\BLOG\\X12Errors\\X12_00401_855.xsd:   error BEC2004: Validate Instance failed for schema X12_00401_855.xsd,   file:   <file:///E:\\SOLUTIONS\\BLOG\\X12Errors\\Sample855missingele.edi>.

Component invocation succeeded.

Line 1 almost gets the error correct, but not quite.  The line indicates that the error is at the element level ("Non Segment Level"), that the error occurs in a PID segment ("[5] X12_00401_855\\PO1Loop1\\PIDLoop1\\PID_2"), and that the error is that an ("invalid child element") was found and that this element was a ("PID05") .  Oops, this last is incorrect. All the PID05 elements in this 855 are correct. Line 1 does state that the ("PID01 was expected"), but this is a poorly stated.  Notice that the XML path is reported rather than using the X12 syntax, which adds confusion.

Line 2 is a simple, correct statement.  It says that the PID segment ("Segment ID - PID"), the 12th segment ("Position - 12") in the 855, has an error in one or more of its data elements ("[8] Segment has data element errors").

Line 3 says that the first element ("Field Position - 1") of the subject PID segment is the problem in that the elment is mandatory but missing ("Mandatory data element missing").   However, line 3 throws in a very confusing statement by saying that the current value is...well, who knows? The problem is that the statement  ("current value -") does not give a current value.  Of  course that is because there is no value in the element. Because Line 3 doesn't show an empty value, many folks read the line incorrectly as ("current value - [1]") which seems to say that the bad value is "1".  That is a problem with the syntax of the error message.

Line 4 is superfluous.

SUMMARY

These three examples show how difficult reading the error messages may be.&nbsp; You can make the process easier by:

    Never assuming that the error message lines are in the correct order.
    Never assuming that all the lines are related to the same error.
    Always reading through all lines to identify the actual error that has occurred.
    Being cautious of the &quot;current value -&quot; problem where there is no value.
If you have a particular X12 parsing error that you would like to see here, please send it to us.

Nested loops to Single Loop

July 5th, 2012

Here is the problem posed by a developer. The input file is:

<Header>
  <PONbr>PONbr</PONbr>
  <PODate>20100721</PODate>
  <Name>Name</Name>
  <CustID>CustID
  <VendEDICode>VendEDICode</VendorEDICode>
  <VendAcct>VendAcct</VendAcct>
  <Buyer>Buyer</Buyer>
  <POAmt>10.4</POAmt>
  <POItemTotal>10.4</POItemTotal>
  <LineCntr>10<LineCntr>
  <LineItems>
    <PONbr></PONbr>
    <LineRef>1</LineRef>
    <QtyOrd>1</QtyOrd>
    <PurchUnit>EA</PurchUnit>
    <UnitCost>1</UnitCost>
    <VendItemID>11111</VendItemID>
    <NDC>11111</NDC>
    <UPC>11111</UPC>
    <TranDesc>TranDesc11</Transdesc>
  </LineItems>
  <LineItems>
    <PONbr></PONbr>
    <LineRef>2</LineRef>
    <QtyOrd>2</QtyOrd>
    <PurchUnit>EA</PurchUnit>
    <UnitCost>2</UnitCost>
    <VendItemID>22222</VendItemID>
    <NDC>22222</NDC>
    <UPC>22222</UPC>
    <TransDesc>TranDesc22</Transdesc>
  </LineItems>
</Header>

The area of interest is the LineItems node, which is a looping node. There are two instances of this node in the input file. The data inside the LineItems node is flat. The LineItems node data must be output to a target loop, the PO loop.  The relationship between the two loops is 1:1, so for the most part this is an easy mapping.  The problem is that the PO1 loop has a child loop, the PID loop.  Data from the one node in the source, the TransDesc node, must be output in the PID loop.  This is a simple instance of one-to-many looping.

Two output loops, one a child of the other, often cannot be driven by the data or with looping functoids.  The fact that one loop is a child of the other confuses the mapping engine.  There is a simple technique that works in this case.  The desired output, looking at only the PO1 loop, is:

<ns0:PO1Loop1>
  <ns0:PO1>
    <PO101></PO101>
    <PO102>1</PO102>
    <PO103>EA</PO103>
    <PO104>1</PO104>
    <PO106>VN</PO106>
    <PO107>11111</PO107>
    <PO108>ND</PO108>
    <PO109>11111</PO109>
    <PO110>UP</PO110>
    <PO111>11111</PO111>
  </ns0:PO1>
  <ns0:PID_2>
    <PID01>F</PID01>
    <PID05>TranDesc11</PID05>
  </ns0:PID_2>
</ns0:PIDLoop1>
<ns0:PO1Loop1>
  <ns0:PO1>
    <PO101></PO101>
    <PO102>2</PO102>
    <PO103>EA</PO103>
    <PO104>2</PO104>
    <PO106>VN</PO106>
    <PO107>22222</PO107>
    <PO108>ND</PO108>
    <PO109>22222</PO109>
    <PO110>UP</PO110>
    <PO111>22222</PO111>
  </ns0:PO1>
  <ns0:PID_2>
    <PID01>F</PID01>
    <PID05>TranDesc22</PID05>
  </ns0:PID_2>
</ns0:PIDLoop1>

The PO1 portion of the map is shown in the figure below.  The looping functoid ensures that one PO1Loop1 is created for each source LineItems node.  The three Uppercase functoids put constants into their target nodes.  Everything else is a simple mapping link from source node to target node.

The code in the Scripting functoid is:

<!--******************************************************
Creates the PID segment
*******************************************************-->
<xsl:template name="PID">
<xsl:param name="description" />
  <xsl:if test="$description != ''''">
    <xsl:element name="ns0:PIDLoop1">
      <xsl:element name="ns0:PID_2">
        <xsl:element name="PID01">F</xsl:element>
        <xsl:element name="PID05"><xsl:value-of select="$description"/></xsl:element>
      </xsl:element>
    </xsl:element>
  </xsl:if>
</xsl:template>

The data from the TranDesc node is the input to the script.  The script uses XSLT to build the entire PIDLoop1 node, including the PID_2 node and the PID101 and PID102 data nodes.

This same technique can be used in many other similar instances, most notably to force creation of N3 nodes in an N1 loop.

EDI Validation Errors: Segment Syntax

June 22nd, 2012

This post discusses the error message:

Event Type: Error
Event Source: EDI Subsystem
Event Category: BizTalk Server 2006
Event ID: 24
Date: 8/3/2010
Time: 12:53:52 PM
User: N/A
Computer: X
Description:
Error encountered: ERROR (81), interchangenr 10243 :
  The check condition has not been met. Contact the sender.source format: [5 00401 ,X12-4010]
  source document: [810 004010DEFAULT X,Invoice]
 source segment: [data#6,def#7,tag=N1 ,name=810_004010_DEFAULT_X_  source element: [def#4,elm#4,comp#0,name=810_004010_DEFAULT_X_], value: [ST] undefined!
  ((810_004010_DEFAULT_X_ <> [] OR
  810_004010_DEFAULT_X_ <> [] ) AND
  ((810_004010_DEFAULT_X_ == [] AND
  810_004010_DEFAULT_X_ == [] ) OR
  (810_004010_DEFAULT_X_ <> [] AND
  810_004010_DEFAULT_X_The error is specific to X12 EDI validation.

X12 Segments have conditions that describe mandatory relationships that must or must not exist between certain elements in a segment. The N1 segment is one example. The check conditions for an N1 segment require that if the N103 element is present then the N104 element MUST be present. The N1 segment also requires that EITHER the N102 element OR the N103/N104 element MUST be present.

Other type of check condtions are:
  IF a is present, then EITHER c OR e MUST be present
  IF a is present, then d MUST NOT be present.

A segment may have only one or may have multiple inter-related check conditions.

In BizTalk, whether or not check conditions are enforced by an X12 schema depends on an annotation in the schema which controls the activation of the check conditions: X12ConditionDesignator_Check.

By default the annotation is set to NO in standard X12 schemas and to YES in HIPAA X12 schemas. Thus in out of the box usage of the schemas the check conditions will not be enforced in standard X12 schemas but will be enforced in HIPAA X12 schemas.

CAUTION: Do not deactivate check conditions in HIPAA schemas without ensuring that doing so is allowed by your client and by Federal law as not enforcing check conditions means that this element of X12 standards validation is not occurring.