SNIP Validation for EDI Documents

SNIP Validation refers to a series of constraints that can be applied to EDI documents to ensure EDI data is HIPAA-compliant. SNIP Validation support is therefore an important factor to consider when choosing an EDI processing solution.

• EDI Specifications
  • EDI Standards
  • Versions
  • Document Types
• SNIP Validation
  • SNIP Type 1
  • SNIP Type 2
  • SNIP Type 3
  • SNIP Type 4
  • SNIP Type 5
  • SNIP Type 6
  • SNIP Type 7
• Implementing SNIP Validation in Arc
  • Arc’s EDI Connectors
  • Implementing SNIP Type 1
  • Implementing SNIP Types 2 and 3
  • Implementing SNIP Type 4
    • Example
  • Implementing SNIP Types 5-7
    • General Scripting Concepts Relevant to Validation
    • SNIP Type 5 Scripting
    • SNIP Type 6 Scripting
    • SNIP Type 7 Scripting
• Conclusion

In order to understand SNIP Validation, it is important to first understand EDI specifications themselves. With that in mind, this article includes three main sections:

• An overview of EDI specifications
• A description of SNIP Validation Levels
• A guide to implementing SNIP Validation within Arc

The first two sections of this article are intended to be useful for anyone implementing an EDI solution, not just those using Arc. The third section is intended to serve as a resource for those who are using or evaluating Arc and are interested in ensuring HIPAA-compliance.

EDI Specifications are guidelines for creating business documents with which different companies can establish a shared data language and understanding. EDI solutions can enforce the guidelines governing these documents more or less strictly. More lenient enforcement may prevent unnecessary errors from being thrown, while more strict enforcement may prevent data processing issues further down the line.

Thus, the EDI specifications alone cannot determine what an EDI processing solution should consider valid or invalid EDI data. The SNIP Validation levels, described in the next section, add explicit rules for how the EDI specification should be enforced, as well as additional enforcement rules (at the higher levels of validation).

To first understand EDI specifications, it is useful to divide different specifications into three layers of hierarchy:

• EDI Standards
• Versions (releases)
• Document types

First, at the top level of the hierarchy, there are several different standards under the umbrella of ‘EDI,’ including:

• X12 (i.e. ANSI X12)
• EDIFACT
• TRADACOMS

These standards all accomplish the same end, namely ensuring that two parties agree on how to interpret business data, but are not interoperable. Documents that adhere to the X12 standard, for example, would not be valid EDIFACT documents.

Each EDI standard has multiple versions of the standard. New versions are occasionally released with updates to the rules defined by the standard. The parties in an EDI exchange must agree on both the EDI standard to use (e.g. X12) and the version within that standard.

The updates released in new versions are typically incremental, so the core elements of each standard remain the same throughout each version. An X12 document will look similar to another X12 document, even if they use different versions of the standard. However, some aspects of the EDI data may be unintelligible to a party using the wrong version.

Common examples of X12 versions include:

• 00401
• 00403
• 00501

Common examples of EDIFACT versions include:

• D96A
• D96B
• D97A
• D97B

Each version of each standard defines a set of document types. Each document type is designed with a specific business exchange in mind; for example, a different set of rules governs a Purchase Order document than governs a Healthcare Enrollment Claim document.

Each document type is defined via an individual schema file. The schema file contains information on the expected number and order of individual EDI segments/elements. In addition to the document-specific schemas, each version has a general schema file containing information about segments/elements that is applicable to all document types (e.g. the set of possible values for certain elements, etc).

For two parties to set up an EDI trading relationship, therefore, they must agree on the EDI standard, the version within that standard, and the specific document types (schemas) relevant to their data exchange.

SNIP Validation describes seven levels of data validation that are related to, but separate from, the schemas mentioned in the previous section. Each level, or “type,” increases the strictness of data constraints for the document. The types are cumulative, meaning that enforcing SNIP Validation Type 4 also requires enforcing the rules within Types 3, 2, and 1.

Adding SNIP Validation to an EDI processing solution can help explicitly determine how closely an EDI document must adhere to the schemas defined in the EDI standard. Additionally, higher levels of SNIP Validation can ensure that the data contained in the EDI document is valid for use in a sensitive or regulated environment, such as a HIPAA-compliant EDI solution. This section contains a summary of the constraints established by SNIP Validation Type.

SNIP Type 1 validates the basic syntactical integrity of the EDI data. This includes the requirements that only valid EDI segments appear in the document, and that they appear according to the order defined in the EDI schemas.

Type 1 alone does not introduce constraints additional to those already imposed by the EDI document schemas, and enforcing these constraints is necessary for successful parsing of the EDI data. Any EDI processing solution, therefore, should support SNIP Type 1 by default.

SNIP Type 2 validates the number of times EDI segments, elements, and qualifiers appear in the document. This includes ensuring that required segments/elements are present, and also that the repeat counts for repeated segments falls within the allowed range.

Type 2, like Type 1, enforces rules defined in the EDI document schemas, but these rules are not necessarily integral to parsing the EDI data. A lenient EDI processing solution might therefore not enforce Type 2 validation by default.

SNIP Type 3 validates that the sum of each claim line item is equal to the total claim amount. Type 3 is where SNIP Validation progresses from simply validating the structure of the EDI segments against the EDI document schemas to validating the data content within these segments. Ensuring that the claim total is correct helps prevent problematic financial discrepancies.

Type 3 is not likely to be supported by EDI processing solutions that do not specifically implement SNIP Validation for the purposes of processing EDI documents subject to HIPAA.

SNIP Type 4 validates inter-segment value relationships: if element A has the value ‘X,’ then element B must have the value ‘Y’ or ‘Z’. Type 4 validation ensures the data within the EDI document is not only valid according to the EDI schema, but also valid with respect to other values within the same document.

Type 4, like Type 3, requires healthcare-specific data validation that is not likely to be supported in general EDI processing solutions. Additionally, the specific codes and if-then relationships involved in Type 4 validation may differ from document to document and implementation to implementation.

SNIP Type 5 validates specific code sets for values within the range accepted by HIPAA. Some EDI fields contain a code value, and only a certain set of codes are valid under HIPAA standards. SNIP Type 5 ensures that code-fields contain a code recognized by these standards.

Examples of HIPAA standards that define an accepted set of codes include:

• NDC (National Drug Code) codes
• ICD (International Statistical Classification of Diseases and Related Health Problems) codes
• CPT (Current Procedural Terminology) codes

Type 5 validation requires cross-referencing EDI data with external resources, and thus implementation requires both a knowledge of these external resources and additional configuration to provide these resources to an EDI processing solution.

SNIP Type 6 validates that differences in healthcare services are taken into account when creating claims data records. For example, EDI segments (records) may have different requirements if an EDI document relates to a chiropractic service, rather than a psychiatric service. Type 6 validation ensures the structure of EDI data matches the service for which the EDI document is intended.

Type 6 validation involves more specific data value validation and may require additional work to implement these validation rules in an EDI processing solution.

SNIP Type 7 validates special requirements for certain EDI trading partners:

• Medicare
• Medicaid
• Indian Health

SNIP Type 7 rules are provided in the Implementation Guide for these specific trading partners and are not relevant when exchanging EDI documents with other partners, such as private insurers.

Arc’s EDI processing capabilities include built-in support for lower SNIP Validation levels. Higher SNIP Validation levels may require configuring a Validate Connector as an additional step of data processing, and the highest levels of SNIP Validation may require scripting to implement the validation logic.

This section will describe how each level of SNIP Validation can be achieved within Arc. Before covering each of these levels, though, it may be useful for newer or perspective users of Arc to describe how the application handles EDI processing.

Arc includes many EDI Connectors that translate various EDI formats into XML and/or generate EDI documents out of XML. This process of converting EDI data into XML follows Arc’s general approach of using XML as an intermediary data format during integration flows. Using XML as a common data format both streamlines data transformation through the XML Map Connector and also ensures Arc can integrate data from disparate data sources and formats. Thus, data is often converted into XML early in the flow, transformed in the middle, and then converted from XML to some target format toward the end of the Flow.

When EDI Connectors convert EDI data to XML, Arc also performs validation of the EDI structure against the appropriate EDI document schema. By default, this validation only includes what is necessary to ensure the EDI can be translated correctly into XML: segment ordering, opening and closing of segments, etc. EDI Connectors also have a setting in the Advanced settings tab called SNIP Validation, which tightens the validation requirements for EDI data when enabled.

As will be explained in the following subsections, enabling the SNIP Validation setting in the EDI connector is all that is required to ensure the first three levels of SNIP Validation.

To achieve higher levels of SNIP Validation, an additional Validate Connector or custom ArcScript may be required. These additions to the Flow will come after the EDI connector, so the data to be validated will be XML (the EDI connector will have already translated the EDI into XML). The details for this additional validation are provided below.

Arc implements SNIP Type 1 automatically with its built-in EDI processing. No settings need to be enabled nor do any further steps need to be taken.

To ensure SNIP Type 2 and/or 3 Validation, the SNIP Validation setting should be enabled in the Advanced tab of the EDI Connector responsible for translating the EDI into XML. No further steps are required.

SNIP Type 4 validation requires adding a Validate Connector to the Flow handling inbound EDI data. The Validate Connector can be configured to implement Type 4 logic: if elementA contains value ‘X,” then elementB must be one of a set of values, e.g. ‘Y’ or ‘Z.’ Note that the Validate Connector expects XML input, so it should be added after the EDI connector that translates inbound EDI files into XML.

Correctly implementing this logic in the Validate Connector requires some understanding of the EDI data involved. Users must be familiar with the specific EDI elements that have this ‘if A, then B’ relationship and be able to trace the xpath to these elements in the translated XML.

Once the xpaths and relationships between EDI elements are known, the Validate Connector can be configured with rules that implement these requirements in the Arc Flow. These rules check that values at both xpaths fall within an expected set and use the operators ‘not equals’, ‘regex match,’ and the boolean ‘OR’ to accomplish the ‘if-then’ logic. An example of implementing this logic is provided below.

Take as an example a simple Type 4 requirement: If elementA has the value ‘IL,’ then elementB must be ‘40,’ ‘41,’ or ‘42.’

There are two steps to consider when implementing this in the Validate Connector:

1. Find the xpaths at which the relevant values are held
2. Use ‘equals,’ ‘regex match,’ and ‘OR’ to implement the if-then relationship

For the sake of this example, imagine a simple xpath to our two relevant elements:

• /Items/path/elementA
• /Items/path/elementB

These will be used in the ‘xpath’ field for the Validate Connector rules.

Next, consider the logical relationship between these elements: If elementA has a specific value (‘IL’), then elementB must have a specific value (a number in the range 40-42). Another way of expressing this relationship is the following: either elementA is not the specific value (‘IL’), or elementB must have the specific value (40-42). This formulation of the relationship can be expressed with two rules in the Validate Connector, connected via an ‘OR’:

Implementing this relationship in the Validate Connector, along with an understanding of the xpaths for both elementA and elementB, would look like this:

This approach to validation rules can be implemented for each pair of elements that need to have a ‘Type 4’ validation relationship.

Higher levels of SNIP validation require writing custom ArcScript to implement the requirements. This section will provide an overview of the concepts necessary to begin scripting for each validation level, and specific help on an implementation script can always be requested at arcsupport@cdata.com.

ArcScript is best applied after the EDI file has been translated to XML. It includes operations like xmlDOMSearch to loop over XML data at a specified xpath. So, the recommended Arc Flow design is to place a Script Connector directly after the EDI Connector in charge of translating EDI documents into XML, and configure that Script Connector with the appropriate validation script.

Within an xmlDOMSearch, the xpath formatter can retrieve a value at a specified xpath (relative to the xpath provided in the xmlDOMSearch call). An example of using this formatter is provided at the bottom of this subsection.

The arc:set keyword is used to store a value in an attribute (i.e. a variable). The arc:if keyword can be used to implement conditional logic. The arc:select keyword can be used to check whether a value is within a specified set of cases.

When a validation script encounters EDI data that fails validation, it needs to throw an error. Errors in ArcScript are thrown with the arc:throw keyword.

Here is a snippet of sample ArcScript representative of the kind of logic necessary for scripting validation rules:

<!-- set input parameters of xmlDOMSearch -->
<arc:set attr="input.uri" value="[FilePath]" />
<arc:set attr="input.xpath" value="/Interchange/FunctionalGroup/TransactionSet/TX-00501-837">

<!-- loop over the input file at the given xpath -->
<arc:call op="xmlDOMSearch" in="input">
  <!-- retrieve specific values from the XML; elementA and elementB -->
  <arc:set attr="elementA" value="[xpath('HLLoop1/NM1Loop2/NM1/NM101')]" />
  <arc:set attr="elementB" value="[xpath('HLLoop1/NM1Loop2/NM1/NM103')]" />

  <!-- check elementA against a list of possible values with a 'select' statement -->
  <arc:select value="[elementA]">

    <!-- for each possible elementA value, see if elementB has a valid value -->
    <arc:case value="IL">
      <arc:if exp="[elementB | notequals(40)] && [elementB | notequals(41)] && [elementB | notequals(42)]">
        <!-- invalid value detected, throw an error -->
        <arc:throw code=1 desc="ElementB had an invalid value of [elementB]." />
      </arc:if>
    </arc:case>
    <arc:case value="PR">
      <arc:if exp="[elementB | notequals(43)] && [elementB | notequals(44)]">
        <!-- invalid value detected, throw an error -->
        <arc:throw code=1 desc="ElementB had an invalid value of [elementB]." />
      </arc:if>
    </arc:case>
  </arc:select>
</arc:call>

SNIP Type 5 requires reading external resources to check for a range of possible values. Users are responsible for the process of obtaining these external resources.

Once obtained, values can be read from these resources using a range of ArcScript features, depending on the specific type of resource:

• Databases can be queried with the dbQuery operation
• Text files can be read with the fileRead operation
• CSV files can be read with the csvListRecords operation

Once the values from the external authority have been read, the process of implementing these restrictions follows similar syntax to the snippet provided in the previous subsection.

SNIP Type 6 requires validating specific value relationships based on the service listed in the EDI file. The scripting approach for this validation follows the same principles as other data validation scripts, with an additional step of first retrieving the healthcare service value to use when validating each individual data element.

These scripts should follow the same sort of ‘arc:select’ and ‘arc:if’ approach described above, though the actual logic tree may include more levels and elements.

SNIP Type 7 requires direct input from the external EDI trading partner. The specifics of implementing these rules is therefore dependent upon the partner, however the same conditional logic tools (‘arc:select’, ‘arc:if’) should be used to implement these checks.

SNIP Validation is an important consideration for any EDI processing solution that will handle healthcare documents, and thus be subject to HIPAA restrictions. Arc includes simple 1-click support for the baseline levels of SNIP Validation and provides a Validate Connector and ArcScript to implement more specific validation rules.

For help configuring your own Validate Connector or writing a validation script, please contact our support team at arcsupport@cdata.com.