XML-based Rules: Automating Business Context Categorization to Produce Semantic Interoperability: The rules-based ebXML approach

Eduardo Gutentag
Arofan Gregory

Abstract

Standardized business semantics allow business applications to share not only a specific, standard syntax, but also to communicate directly without a lot of costly analysis and integration. The ebXML activity has been writing such business rules in XML. We examine the concept of rules, their relationship to documentation, what it means to express rules in a "syntax-free" mode as opposed to writing them in XML, and the process of building rules. We then highlight some of the issues related to the DTDs for rules and how they translate into XML instances; we also elucidate the concept of semantic interoperability and how to achieve it.

Keywords: Semantics; Interoperability; DTD

Eduardo Gutentag

Eduardo Gutentag has been involved in the SGML/XML world for over 10 years, and has participated in many SGML/XML related activities at W3C and OASIS (even when it was still called SGML Open), as well as at ebXML. He is currently a Senior Staff Engineer at the XML Technology Center of Sun Microsystems. He is mainly responsible for those sections in this paper that describe the rules.

Arofan Gregory

Arofan Gregory is the technical manager of the XML Common Business Library at Commerce One, and has been active both in the ebXML Core Components work and earlier in the CommerceNet eCo initiative. He has presented at various SGML and XML conferences, and has been active in this technology arena for the past decade, working in commercial publishing and as a consultant on document management and e-commerce applications. He is mainly responsible for those sections in this paper that describe the business context.

XML-based Rules

Automating Business Context Categorization to Produce Semantic Interoperability: The rules-based ebXML approach

Eduardo Gutentag [Sun Microsystems, XML Technology Center]
Arofan Gregory [Commerce One, Commerce One Labs, xCBL Division]

Extreme Markup Languages 2001® (Montréal, Québec)

Copyright © 2001 Eduardo Gutentag and Arofan Gregory. Reproduced with permission.

Background: the problem

Variation within Business Semantics

For many years, efforts have been made to standardize business semantics, so that business applications could share not only a specific, standard syntax, but also be able to communicate directly with one another without the benefit of a lot of costly analysis and integration. To summarize in a simplistic way the approach to this kind of standardization, the idea was to have a single syntax and a single, standardized set of semantics for all people doing business.

This approach was behind some very effective efforts, notably ANSI X12 and UN/EDIFACT, which have been widely adopted and used within the traditional EDI community. That community, however, has recognized that the future of e-business begins to look somewhat different: the traditional point-to-point technology based on trading partner agreements will no longer scale in a world where the integration cycle has been shortened into seconds or minutes, rather than weeks or months.

Further, trying to solve problems of scope by splitting semantics into vertical "industry" vocabularies has proven to be too simplistic: by its nature, a supply chain exists in more than one industry vertical. The use of a top-down vocabulary, identified by the biggest buyers at the top of the supply chain, requires the smallest suppliers to use multiple "industry" dialects as a result of their business. Multiply this by the cost of integration in the traditional EDI world, and you can see why the entire supply chain was rarely automated. Simple organization into industry verticals did not solve the problem.

ebXML [1] represents an effort to further define how different enterprises can express their semantics in ways that will achieve a higher level of "out of the box" interoperability. The key to this approach has been to understand that differences in business semantics are not driven merely by ignorance, but by actual business needs in the real world. Different businesses have different needs, and these needs come from many different, related directions.

ebXML Core Components

The ebXML Core Components Working Group attempted to address the issues around the variation within business vocabularies. There are a number of pieces in this equation, and before delving too far into the particular aspects of context rules, a general familiarity with the concepts behind what the group has created is needed.
Core Components

These are a set of low-level components, determined to have common semantics across a wide variety of business vocabularies and domains. they range from simple, data-containing constructs such as "price" or "quantity" to more complex, compound structures such as "addresses". These constructs are expressed as models, where each piece of data is clearly defined, given a UID, a name, and a type. The core constructs are not bound to any particular syntax (for example, a particular number might be expressed as an element or attribute in a particular vocabulary, but the models do not make this distinction, nor any other assumptions about the capabilities of the syntax based on the core components models).

Context Classifications

For each of the context areas, a standard classification was established, such that any group referring to a particular business context would describe that context in exactly the same terms as anyone else making the same reference. The classifications were generally drawn from existing standard classification schemes already used in the electronic commerce world. These schemes represent values hierarchies in most cases: no syntax for their realization was specified.

Assembly and Context Rules

Given the core components models, and the existence of a set of standard classification schemes, it remained to provide for a way of taking the core components, identifying the business contexts in which they were relevant, and then grouping and modifying the components to fit the needs of a particular business situation. This topic will receive a far more thorough treatment below - suffice it to say that two sets of rules were created, both expressed as XML DTDS: one for assembling a set of core components into a useful business document, and the next for modifying the assembly into the exact structure needed to carry data between two trading partners.

Semantic Interoperability Document

Although it was assumed that development-time or run-time processing of the core components, contexts, and rules would be informed by some syntax binding to produce an XML schema or DTD with which business would be conducted, it was also seen as possible to output a document that reflected the modified semantic structures before they were bound to a particular syntax. This artifact allows automatic processing of pure semantic structures, such that an automatic comparison could be made of two such documents, even if they did not share a single syntax binding.

The need for this is non-obvious, but it allows much more powerful application of the technology. If I can see that you require a "price" field with a name of "DiscountPrice" - which you will express as an attribute in a DTD - and I require a field called "DiscountedPrice" with the same value - which I express as an element - we still need to be able to determine that these constructs are the same thing. This is easier to do before the constructs are bound to a particular syntactic expression.

These pieces fit together in a fairly straightforward way. When you and I want to become trading partners and do business, we both have to state our own data requirements for each type of business we engage in. Our requirements are expressed as rules, couched in terms of the context of each type of business we do, measured by standard classifications, which modify a standard set of semantic constructs, the core components. When we do business, we can uniquely identify the one context in which our transaction is taking place. We can then consult the intersection of our rules to determine the data needs that are common, and use the same comparison to identify where there is a gap between our ability to provide data and the other's requirement for it. Many of the aspects of business-to-business integration can be automated with this rich set of information.

Business Context

The word "context" has come to represent the particular use of a set of business semantics in a particular application. The idea behind recognizing context is two-fold: we can accurately describe the exact use of semantics situation-by-situation; and we can render this in such a way that automated processing becomes possible to make integration and agreement of the minimum needed set of semantics for a particular transaction more efficient.

The concept of business context has been around for a long time, and it sits implicitly behind the definition of any business vocabulary. Suppose I write a "DTD" for use in the electronics industry when sending an order to a supplier, for example. The way this is captured in a DTD, however, tends to be very messy. I have to lump together the potentially disparate requirements of all trading partners within a community, and then express them in a single document, with, at best, some documentation. Tools for making distinctions about particular sets of requirements are very few.

What ebXML set out to do was to make the context of a business transaction explicit in a way that was both fully documented and machine-readable. This information has many applications, such as:

  1. Capturing knowledge of the semantics built into data structures during analysis - usually lost after development ends - for automated processing and use in future integrations and at run time.
  2. Identifying the basic agreements between disparate vocabularies, and identifying the exact points of conflict or ambiguity.
  3. Narrowing the scope of the problem to the minimum necessary data set for a particular transaction.

These scenarios will be examined in greater depth later in the presentation.

Context Analysis

ebXML set out to solve a very complex problem - to categorize every possible relevant business situation with a set of standard descriptors. The sources of inspiration were many:

  • Statistics categorization, product and process categorizations, and standard business vocabulary categorizations (ISO codes, UN/SPSC classification, etc.)
  • Implicit assumptions written into various existing process and data standards such as RosettaNet, OTA, and the work coming out of various parts of UN/CEFACT and X12
  • Access to the business teams analyzing the core components

The initial list was a long one, and the work has not been finished, but what has been produced as a list of major context drivers [9] is useful:

  • Business Process
  • Geopolitical
  • Product/Service
  • Legislative/Conventional
  • Industry/Sector
  • Role

It is recognized that many other context drivers are possible: "temporal" was one such candidate, rejected because of difficulties in defining exactly what was meant. Look at the problem: when I attach the value "US" to a particular data structure, this clearly tells me it is for use within the United States. What about attaching a bit of data that says "2 years" or "February 12, 2003"? What do these say about the data to which they are attached? Too many supporting semantics are required to make a useful "temporal" context category. While many business problems such as effectivity are clearly related, they were deemed too difficult for solution with the initial release of ebXML.

Business Process Context

This is perhaps the most fundamental of all business contexts. If I am placing an order, I will send you one set of data. If I am approving a request for payment, I will send you another. If I am acknowledging receipt of a shipment of goods, I send you a third. These differences form the basis for the major distinctions between doctypes in traditional business vocabularies. Coming up with a standard classification of business processes was easy within the ebXML initiative, too, because the Business Process group was establishing a catalog of common business processes[2]. All that was needed was to align with that classification scheme, and to allow for trading partners to extend the scheme where necessary in a formally regular fashion.

Geopolitical Context

Much variation in business information structure is driven by geographical and political differences. This includes many purely cultural influences as well. (A good example is the fashion in which ex-British Dominion countries tend to organize payments information one way, while it is generally done a different way elsewhere in the world. No body of law requires these two structures, they are merely cultural differences.) If one can state the geopolitical domain within which a given data structure is used, then differences and similarities can be easily recognized. This classification is hierarchical, so that generalizations may be made about entities such as the European Union, for example. Also, more than a single value is permitted: a single address format might be used for doing business in both Germany and Austria, for example, although not be applicable throughout all of Europe.

It is also possible to specify data requirements deriving from the crossing of a specific border. Especially in cross-border trade scenarios, different data structures may be used depending on which borders a shipment will cross while moving from supplier to buyer. Sufficient descriptive power was needed to express those places where data variation was the result of cross-border requirements. Again, the classification was made extensible to recognize geopolitical entities that may not exist in current classification schemes.

Product/Service Context

One of the major drivers in business document variation is the product or service being described. While there are many common schemes for generically describing goods and services, none is universally applicable. Particularly within supply chain scenarios, where Bill of Material processing is common, product descriptions are highly variable. (Bill of materials applications rely on an incoming order being broken down into a set of component orders, hence the need for good descriptions of goods and their component materials.) Common product classifications were adopted for this purpose: UN/SPSC was recommended, but others may be needed for a more complete description of a particular commodity. Again, this is an extensible hierarchy.

Legislative/Conventional Context

Another way of describing the data requirements for specific business situations stems from the relevant bodies of law and/or adherence to standards. A mechanism was needed for explaining that a piece of data was structured in accordance with some set of requirements that could be clearly identified. This was deemed to not be a hierarchical classification, as no standard hierarchy exists today for describing all legal, standard, and conventional rules regarding structuring and provision of data.

Industry/Sector Context

Many data requirements derive from membership within a particular sector (government, healthcare, etc.) or vertical industry (aerospace, automotive, petrochemical, etc.). In many cases, standards organizations exist that have published a set of message definitions for their particular industry vertical. This context factor is probably second in importance only to business process as a critical part of defining what data is needed and how it is structured. Standard hierarchies of industry sectors were used, but extensions were allowed.

Role Context

Although originally seen as a part of business context, it was determined that associating an explicit role (which is not always associated with one party in a business process) could help explain why data was structured in some particular way. For example, if I give you contact information for shipping and handling, I will probably give you a postal address, a telephone number, e-mail, and possibly fax information. If I give you the contact information for a contact to be used for a hazardous spill during shipment, I won't expect you to be mailing anything to them, hence no postal address. Two types of slightly divergent "roles" - both contacts, but with different purposes, require different data structures. And they are not determined strictly by the business process. For roles, standard flat codelists were recommended for use, as employed by X12 and UN/EDIFACT today. Again, there is a provision for extension.

How These Factors Combine

You will note than any given data structure may be "driven" by a number of different context factors. A good example of this is the one about a hazardous contact: although it is the role of the contact that defines what information needs to be supplied, it would be a regulatory context factor requiring that a hazardous contact be supplied in the first place. Essentially, we can think of each context factor as an axis, with any given conceptual point being identified by a value along one or more of these axes.

Why XML? Why Now?

Given that the problems around semantic interoperability and harmonization have existed in the EDI application space for years, and that the EDI community certainly developed a concept of "business context" to help solve this problem, why is it that the arrival of XML [3] on the scene has coincided with this approach to the problem? There are a number of reasons for this, relating to the evolution of XML technology standards and technology approaches.

The creation of XML schemas have given us a rich set of information about types, and the tools to manipulate this essentially object-oriented approach. While "object-EDI" was not a new idea, the tools for handling it have been enormously improved, in the form of various XML schema languages. The aspects of greatest importance are type relationships, polymorphism, extension, and local renaming. The application of context uses all of these features to make automation of the rules-based approach feasible.

XSLT [4] is another type of XML technology that has had an impact on the way problems that have always involved transformation can be solved. Part of the impact is due in part to XSLT's widespread adoption by the XML community, which has led to a different way of looking at documents, from static pieces of information to potential targets of dynamic manipulation. This has been aided in no small measure by the fact that schemas themselves, being XML documents, can be transformed by XSLT.

Furthermore, a transformation can itself be expressed as an XML document; it is also a precursor to the kinds of rules syntax that can be easily expressed in XML to solve this type of problem.

Perhaps the greatest challenge facing ebXML is that it is meant to avoid being modeled specifically in a single syntax. Different syntaxes - EDI as opposed to any form of XML - and even different validation schemas for XML, typically a DTD or one of many schema languages - have different requirements in terms of how they capture concepts such as typing, that are core to solving the semantic problems. The challenge is to agree on an object-model of the semantic structure, and then enable different syntax bindings to map to one another, automatically. ebXML has this as a primary requirement at the semantic level.

Scenarios for Use

Now that you have a sense of how the ebXML Core Components group was looking at the problem, it may help to outline in more detail the envisioned applications of this technology. There are two basic scenarios, the development-time scenario and the run-time scenario. Based on the understanding that the current generation of e-commerce technology is only capable of supporting the use of a fixed data structure description (DTD or schema), it was determined that both would require support.

The way the application of context was envisioned, the ebXML repository [5] would contain many different artifacts: the DTDs/schemas created by the development process; the core component library and relevant context classification schemes; and the rules that created the DTDs/schemas from the core components and context classifications. At run time, you might simply use the DTDs/schemas, consulting the other resources at development time while integrating with a trading partner. It was assumed, however, that these resources might also be useful at run-time at some point in the future. In either case, having this information available to you in processable form is a benefit.

Below are the three main scenarios envisioned for the application of context to information reuse in business-to-business e-commerce messages:

Preservation of Analysis Metadata

One absolute use for this approach is the preservation of the information that goes into the definition of a static message format. When I am designing an Advance Ship Notice for the US automotive industry, I consciously make design choices based on the needs of the industry, the business process, and the geographical region. In the past, this information has been inconsistently documented. Now, it is captured formally in a systematic fashion, and made available to processing by the tools that conduct business or the tools that enable integrators. This is the most obvious application of this technology, but it is not without value.

It is worth noting that many industry-specific standards groups may wish to "retroactively" describe their reasoning for already-published XML DTDs or other document definitions. In general, it may well be that communities of interest - rather than individual trading partners - will standardize on a certain set of data structures.

Determination of Data Agreement and Conflict

If trading partners have expressed their data requirements for different business situations using the mechanism described, it becomes possible for integrators to compare the data requirements of two trading partners automatically. Even in situations where the same component is given different local names, comparisons can be made by processing the rules associated with a specific business situation. Once the transaction(s) the trading partners wish to make are known, it should be possible to know exactly which set of data fields are required, and how modified, based on some specific set of core components. A process could be written to directly compare the interoperability documents (or resultant schemas, assuming a common syntax binding) to identify absolute points of commonality and conflict, based on the minimum data requirements of the trading partners, and to further identify those areas where there are probable agreements or conflicts. While this does not give us the ability to completely automate integration, it does remove much of the costly human time spent identifying these same cases.

Automatic Generation of Mappings

Another process that can be automated is the generation of mappings between different vocabularies and or trading-partner-specific document types. This is related to the point above: if we can automatically identify points of absolute and probable agreement, then we can write programs that equate the post-syntax-binding structures to each other.

Let us look at how such a process would work. First, we can determine from the schemas/DTDs or semantic interoperability documents the UIDs associating a particular element or attribute with a core component. We can also trace back to the rules that performed assembly and modification on these constructs. Because we will know the context in which a particular transaction will take place, we should be able to map from field-to-field across different trading partner document types, and even across disparate syntax bindings.

This will not be a 100% complete map, but it will considerably reduce the extent of effort required to create one, thus making semantic transformations much easier, and easing trading partner integration.

Rules

What are rules

Once the concept of context as a driver for business transactions has been established, and various relevant contexts have been identified, a methodology is needed for writing rules such that when applied to general schemas in different contexts, new schemas specific to those contexts can be produced.

The immediate question that comes to mind is "why do you need rules, isn't this a documentation issue?".

The facile answer is that yes, indeed rules are the same as documentation. Consider the case of drugs - in the US they are always accompanied by tiny bits of documentation, such as "If you are allergic to peanut butter, consult a physician before taking". Well, that certainly can be expressed by a rule of the form "if a=b, then x"

And not much has been gained, except obscurity.

On the other hand, as conditions and counter indications become increasingly complex, documentation becomes increasingly harder to write and understand. Consider, for instance, trying to express the following on the back of a bottle of aspirin:

    if a=b, then
        if b!=c, but (c=d and d!=e) or d!=f then
            if g=h and (h=i or j=k) then
            z or y but not x

Rules can express succinctly a variety of conditions and results that are interdependent and heavily qualified. Plain documentation, on the other hand, while it can express the same conditions and results in a human readable manner, cannot do so either in a succinct way or in a machine readable format. And that is the crux of the matter: while one of the requirements for the assembly and context rules was that they had to be human readable, they also had to be machine readable. And their basic expression had to be syntax free, inasmuch as this is achievable.

Syntax free expression of rules

What does "syntax-free expression" or "syntax-independent expression" mean? It could be argued that there is indeed no such thing; that any time you express something you are using some kind of syntax, and/or that you are assuming one. As arguments go, this one is pretty much unbeatable.

So let's think instead in purely pragmatic terms:

  1. it means that you do not use a syntax for the expression of the basic terms of the rules
  2. it means that you try to avoid assumptions regarding what the input will be on which the rules will be applied.
  3. it means that you try to avoid assumptions regarding what the output will be after the application of the rules.

In other words, the expression of the rules should be such that they could process EDI and output RELAX NG[6], or they could process W3C XML Schema[7] and output EDI. In the same spirit, the ultimate expression of the rules could be as XML files, or as sed scripts or even as prose documentation.

The way we chose to express the basic rule components in the Document Assembly and Context Rules[8] document was, then, in the form of a table. The following is just an excerpt from the table that can be seen at http://ebxml.org/specs/ebCCDOC.pdf, section 7.1

Table 1: Sample of rules
Name Type Occurrence Default Description
    Assembly    
Assemble complex +   List of assembled Core Components
Context complex required   List of contexts used in this assembly
@name string optional   Name of collection of assembled document schemas.
@version string optional   Version of the Assembly Rules document.
    Assemble    
CreateElement complex +   List of Core Components
CreateGroup complex *   Create a group of elements
@name string required   Name of the document schema being assembled

What the above table snippet provides is:

  1. a vocabulary of rules
  2. constraints on the relationships between the rules
  3. something that business people can read

This is not earth-shattering by any means, but it does have clear implications:

  1. Anybody can read the rules - they are quite close to textual documentation, yet they are formalized
  2. There is no connection between the rules and any given syntax for expressing them - they are unbound

In spite of our stated good intentions, the table is not strictly neutral - witness the '@' notation to indicate attributes. A future version of the table may eliminate this.

Building rules

The above rules could easily be expressed in BNF notation, and the appropriate code could then be generated, as in the following pseudo-code:

function Assembly(name, version) {
    string A_name;
    output(name);
    output(version);
    Assemble(A_name);
    Assemble(A_name);
    }
function Assemble(A_name) {
    string name;
    output(A_name);
    CreateElement(name);
    CreateGroup(name);
    }
etc.

The obvious disadvantage of the above is that there is no ready-made way to verify that you can indeed have more than one Assemble() statement within the Assembly() function. You just have to trust that the code-generating software is bug-free. This is a leap of faith that few people and businesses are willing to do.

One obvious solution is to bind the rules to XML. First by translating the table into a DTD (although any other schema language would be equally appropriate in this case):

<!ELEMENT Assembly (Assemble+)>
<!ATTLIST Assembly
    version CDATA #IMPLIED
    id      ID    #IMPLIED
    idref   IDREF #IMPLIED
>
and then by creating the appropriate rule in XML:
<Assembly version="1.0">
    <Assemble name="PurchaseOrder" id="PO">
    ...
    </Assemble>
    <Assemble name="PurchaseOrderResponse" id="POR">
    ...
    </Assemble>
</Assembly>

In other words, what we have now is a series of rules expressed in XML, to be executed generally in document order, designed to do a very specific task. One great advantage is that anybody with a minimum of XML knowledge can read, modify or write the rules. This advantage by itself explains why we refrained from following our first inclination, which was to write the rules as XSLT stylesheets.

Here is where we could lie barefacedly and say that once the rules tables were laid down, writing the DTDs for Assembly and Context Rules was an almost mechanical process. However, since no one would believe us anyway, the truth is that although the above statement is true for about 80% of the rules, there was a recalcitrant 20% that made us go back and forth between the rules and the DTDs; it was only when actually doing the syntax binding that we realized some of the problems presented by the rules as written at that time.

One clear example of this centered around the eternal conundrum of what should be an element and what should be an attribute.

For instance, the original translation of the rules table into DTD for CreateElement was:

<!ELEMENT CreateElement
            (Name?, (CreateGroup|Rename|UseElement|Condition|Annotation)*)>
<!ATTLIST CreateElement
            type    CDATA        #IMPLIED
            minOccurs    CDATA   #IMPLIED
            maxOccurs    CDATA   #IMPLIED
            id            ID     #IMPLIED
            idref        IDREF   #IMPLIED
            location    CDATA    #IMPLIED
>

This resulted in instances like:

<CreateElement type="someType" minOccurs="0" maxOccurs="1"
               id="someelement" location="someUUID">
    <Name>SomeName</Name>
</CreateElement>

While the above seems proper, and it is certainly a valid instantiation, it soon became apparent (soon being used in a relative manner) that @type, @minOccurs and @maxOccurs are used, just like <Name>, to modify the output of the rules, whereas @id is used to "qualify" the <CreateElement> element itself. As a result of this, the three attributes were changed into elements, and the rules tables were changed accordingly.

<!ELEMENT CreateElement
        (Name?, Type?, MinOccurs?, MaxOccurs?,
        (CreateGroup|Rename|UseElement|Condition|Annotation)*)>
<!ATTLIST CreateElement
            id            ID       #IMPLIED
            idref        IDREF     #IMPLIDE
            location    CDATA      #IMPLIED
>

Now the appropriate instance will look like this:

<CreateElement id="someelement" location="someUUID">
    <Name>SomeName</Name>
    <Type>SomeType</Type>
    <MinOccurs>0</MinOccurs>
    <MaxOccurs>1</MaxOccurs>
</CreateElement>

It could be argued that if the rules table had been properly syntax neutral, changes in the representation of the components, from element to attribute and vice versa, should not have affected the table at all. That is probably true.

Types of rules

Two types of rules were defined: Assembly rules and Context rules. This is not readily apparent from a quick read of the rules table, where all the rules are presented in one single table; it is only by reading them carefully that one discerns the existence of two separate roots. The alternative was to create three tables, one for Assembly, one for Context and one for common rules. This split may occur in a future revision.

Assembly Rules

Assembly rules are used to put together one or more core components into usable schemas. Each Assemble rule can be, optionally, output to a different file. Also, ID/IDREF relationships can be established across Assemble rules for convenience sake. Each Assemble rule can create groups or elements; in the latter case, a location (from which the element is brought in) can be specified either as a UUID or as a URI. Elements can be grouped and renamed, and in some cases these actions can be conditioned depending on context.

Conceptually, Assembly Rules are somewhat trivial and uninteresting, so we will not spend much time on them.

Context Rules

Once a schema has been assembled out of Core Components, Context rules can be applied to it in order to refine it. As stated above, in “Context Analysis”, six contexts have been firmly established, though there is room to extend them in the future if necessary:

  • Industry
  • Business Process
  • Product
  • Geopolitical
  • Legislative
  • Role

Depending on the context, then, elements can be added and deleted to the schema, or their occurrence can be constrained; attributes can also be added (while specifying their type, values, etc.) or deleted, and enumerated attribute values can be modified.

Why two kinds of rules

One of the hardest and more controversial decisions was actually the separation of the rules into Assembly and Context rules. Originally the intention was that one would apply Context rules as the schema was being assembled. This, in the face of it, makes a lot of sense - if you want a purchase order that's appropriate for a given industry within some regions in Canada that reflect NAFTA constraints, why should you have to first assemble a generic purchase order and then apply a second set of rules when you can do the whole thing in one easy step?

The answer is that it is not, in fact, one easy step. One of the principles of the rules is that they apply to the source document, not to in-memory representation of rules previously applied to that source. Thus, writing one single set of rules that both assemble and apply complex context rules can become a rather challenging enterprise, since you have to keep at all times in your mind a representation of what the original document was and to what parts of that document, no matter how it's already been manipulated, the rules apply. Furthermore, applying everything at once prevents modularization, both of the rules and of the output. Instead of being able to create a schema for purchase orders, to which you can then apply context rules for a given industry, that can then be further constrained for a region, and constrained again by legal considerations, you end up with one big spaghetti that cannot be unraveled without breaking something in the process. And you cannot save the "intermediate" output represented by the assembled schema for further reuse (that is, you cannot register it and deposit in a repository for others to use for the simple reason that there is no output until all the rules have been applied.)

One final reason is that uniting Assembly and Context rules would have made the resulting instances unintelligible to their intended audience, viz. business people.

On the other hand, there are occasions when context can play an important role in the assembly process. Consider the case of assembling an Invoice. In Switzerland, for instance, there may be tax information that is not needed anywhere else in the world. Undoubtedly this could be added to a generic invoice, and refined out every time a context-driven refinement is performed. But the end result of this would be that generic documents would be always a maximal union of all possible needs, to be weeded out in almost all contexts. A simpler solution is to acknowledge this and allow conditional assembly, while excluding conditional refinement.

DTDs for rules

The schemas derivable from the rules tables are quite simple:

The Assembly DTD has 7 main elements (and a few ancillary ones, plus attributes): Assembly, Assemble, CreateGroup, CreateElement, UseElement, Rename and Condition.

The Context Rules DTD has 10 main elements (and a few ancillary ones, plus attributes): ContextRules, Rule, Condition, Action, Add, Rename, CreateGroup, Element, Attribute and Use.

Both DTDs can be seen in their entirety at http://ebxml.org/specs/ebCCDOC.pdf, sections 7.2 and 7.3

Unfortunately, there are some constraints that cannot be expressed through DTDs. For instance, a CreateElement element that has an idref attribute should be empty; or applications should ignore its content.

Another example is the Attribute element, which has four optional elements in its content model, of which at least one must be present. If the value of the applyTo attribute of Action is an attribute, there is no need to specify the Name again. If only Value is specified, the intention must be to add or subtract a given value to or from an attribute's enumerated list. All of which is harder to explain in documentation than with a few examples.

The use of the Attribute element is particularly gnarly. For instance, the following adds an element and an attribute:

         <Add before="Address">
            <Element >
              <Name>QualifyingInfo</Name>
              <Type>QualifyingInfoType</Type>
              <Attribute>
                <Name>Privacy</Name>
                <Type>yes | no</Type>
                <Use>default</Use>
                <Value>yes</Value>
              </Attribute
            </Element>>
         </Add>

The following deletes an attribute from an element:

       <Action applyTo="//Party">
          <Subtract>
            <Attribute >
              <Name>TaxIdentifier</Name>
            </Attribute>
          </Subtract>
        </Action>

And the following adds a possible value to an existing attribute's enumerated list:

     <Action applyTo="//QualifyingInfo/@Convictions">
        <Add>
          <Attribute>
            <Value>Unknown</Value>
          </Attribute>
        </Add>
     </Action>

XML instances

In order to understand how all of the above is supposed to work, let's concentrate on one very imaginary example.

Suppose you want to create a Purchase Order and a Purchase Order Response. At a minimum, you need a schema that has two Parties and a bunch of items being purchased (well, at least one.) You also need to take into consideration the location of the parties, since that would affect how addresses are written, and possibly the Industry in which this transaction takes place, since that could affect how the Items are described.

So the first thing you do is assemble a generic PO and a generic POR. You create a group that contains two parties, one for the buyer and one for the seller, and you create another group for the Items. This creation process is done by using the Assembly rules for extracting the appropriate core components from a repository and arranging them in the desired order.

<Assembly>
<Assemble name="PurchaseOrder" id="PO">
        <CreateGroup>
        <CreateElement location="UUID" id="Buyer">
        <Name>Buyer</Name>
        <Type>PartyType</Type>
        <CreateGroup>
         <UseElement name="Name"/>
         <UseElement name="Address">
            <CreateGroup id="fred">
              <CreateGroup type="choice">
                <UseElement name="BuildingName"/>
                <UseElement name="BuildingNumber"/>
              </CreateGroup>
              <UseElement name="StreetName"/>
              <UseElement name="City"/>
              <UseElement name="State"/>
              <UseElement name="ZIP"/>
              <UseElement name="Country"/>
            </CreateGroup>
          </UseElement>
        </CreateGroup>
        <Condition test="$Geopolitical='United States'">
          ...
        </Condition>
      </CreateElement>
      <CreateElement location="UUID" id="Seller">
          ...
      </CreateElement>
    </CreateGroup>
    ...
</Assemble>
<Assemble name="PurchaseOrderReceipt" id="POR">
    <CreateGroup>
      <CreateElement idref="Seller">
      </CreateElement>
      <CreateElement idref="Buyer">
      </CreateElement>
    </CreateGroup>
</Assemble>
</Assembly>

One interesting aspect of the Assembly rules is the use of ID/IDREF relationships between rules located in different Assemble containers. For instance, given an Assemble container for a purchase order, in which <CreateElement>s for Buyer and for Seller are meticulously created, you can then add another Assemble container for a purchase order receipt, and instead of repeating the whole process, you can refer to the ID for the Buyer and to the ID for the seller:

Once you have the schema, you can then apply Context rules to modify, for instance, the addresses of the parties, depending on what the location context is. And you can modify the Items, depending on what the industry context is.

<?xml version="1.0"?>
<!DOCTYPE ContextRules SYSTEM "contextrules.dtd">
<ContextRules id="CalAer">
  <Rule apply="hierarchical">
    <Taxonomy context="Geopolitical"
      ref="http://ebxml.org/classification/ISO3166"/>
    <Taxonomy context="Industry"
      ref="http://ebxml.org/classification/industry/aviation"/>
    <Condition test="$Geopolitical='United States'">
      <Action applyTo="//Buyer/Address">
        <Occurs>
          <Element >
            <Name>State</Name>
          </Element>
        </Occurs>
        <Add after="@id='fred'">
          <CreateGroup type="choice">
            <Element >
              <Name>Floor</Name>
              <Type>string</Type>
            </Element>
            <Element >
              <Name>Suite</Name>
              <Type>string</Type>
            </Element>
          </CreateGroup>
        </Add>
      </Action>
    </Condition>
  </Rule>
  <Rule order="10">
    <Taxonomy context="Geopolitical"
    ref="http://ebxml.org/classification/ISO3166"/>
    <Condition test="$Business Process='RFQ'">
      <Condition test="Industry='Insurance'">
        <Action applyTo="//Party">
          <Add before="Address">
            <Element >
              <Name>QualifyingInfo</Name>
              <Type>QualifyingInfo</Type>
              <Attribute>
                <Name>Privacy</Name>
                <Type>yes | no</Type>
                <Use>default</Use>
                <Value>yes</Value>
              </Attribute>
              <Attribute>
                <Name>Accuracy</Name>
                <Type>CDATA</Type>
                <Use>required</Use>
              </Attribute>
              <Annotation>
                <Documentation>What this element is for.
                </Documentation>
              </Annotation>
            </Element>
          </Add>
        </Action>
      </Condition>
      <Condition test="$Industry='Travel'">
        <Action applyTo="//Party">
          <Subtract>
            <Attribute >
              <Name>TaxIdentifier</Name>
            </Attribute>
          </Subtract>
        </Action>
      </Condition>
    </Condition>
  </Rule>
  <Rule>
    <Taxonomy context="Industry"
      ref="http://ebxml.org/classification/Industry/Automotive"/>
    <Condition test="$Industry='Automotive'">
      <Action applyTo="//QualifyingInfo">
        <Add>
          <Element >
            <Name>DrivingRecord</Name>
            <Type>DrivingRecord</Type>
          </Element>
          <Element >
            <Name>CarDescription</Name>
            <Type>CarDescription</Type>
          </Element>
          <Element >
            <Name>DrivingHabits</Name>
            <Type>DrivingHabits</Type>
          </Element>
        </Add>
        <Rename from="@Convictions" to="@DrivingConvictions"/>
      </Action>
      <Action applyTo="//QualifyingInfo/@Convictions">
        <Add>
          <Attribute>
            <Value>Unknown</Value>
          </Attribute>
        </Add>
      </Action>
    </Condition>
  </Rule>
</ContextRules>

The DTD for Context Rules offers also some interesting issues.

The attribute order was originally named "priority". But this was too easily confused with the use of "priority" in XSLT. In XSLT, when two actions apply to the same node, disambiguation is done either through element order or through the use of "priority", but in no case are both actions applied: only one is ever used. In our case, however, the intention is that all the rules should be applied, either in document order or according to the order attribute.

The <Condition> element also needs some clarification. Initially this was an empty element at the top of <Rule>. So each rule had a condition, followed by a definition of the rule, which would apply only if the condition was true. But this proved cumbersome, since complex conditions required writing many rules and keeping track of their order.

There was also some talk at the beginning of our discussions that the condition itself should be expressed as a series of XML elements and attributes, such as:

<Condition>
    <Context>
        <Name>Geopolitical</Name>
        <Value]United States</Value>
    </Context>
</Condition>
but this was soon killed in favor of XPath.

The <Taxonomy> element is intimately related to <Condition>. It (together with the apply attribute of <Rule> provides semantic metainformation regarding the condition. In other words, two rules with conditions that are character-by-character identical may be semantically distinct, depending on whether the taxonomy is applied exactly of hierarchically.

For example:

<Rule apply="exact">
    <Taxonomy context="Geopolitical" ref="someRef">
    </Taxonomy>
    <Condition test="$Geopolitical='California'">
    ...
</Rule>
is semantically different from
<Rule apply="hierarchical">
    <Taxonomy context="Geopolitical" ref="someRef">
    </Taxonomy>
    <Condition test="$Geopolitical='California'">
    ...
</Rule>
because in the first case only the Californian context should be taken into account, while in the second case the whole hierarchy, of which California is a leaf, should be taken into account.

Semantic Interoperability

The issue of whether two schemas have been produced following the same rules and with the same semantics is crucial for interoperability between businesses. The basic question is: given a W3C XML Schema and a DTD, both purporting to have been assembled and context-refined following the same rules, how can one be sure this is true? Worse: given two documents written in the same syntax, based on the same rules, how can we be sure they have the same semantics? The answer is given by the notion of a Semantic Interoperability document.

Given any schema created through Assembly and Context rules, it should be possible to create a document that conforms to the Semantic Interoperability schema, describing how the schema was created: what assembly and context rules were followed; what contexts were applied; what are the components; what are their children and in what order; and what are the relationships amongst nested, unnamed groups of child components. Thus, two schemas that produce equivalent SIDs can be considered semantically interoperable.

For example, given the following fragments:

<Item>
  <Name></Name>
  <Description></Description>
  <Price></Price>
</Item>
and
<Item>
  <Number></Number>
  <Price></Price>
  <Description></Description>
</Item>
the only way to know whether they are semantically equivalent is to examine their SIDs - was Rename used to change Name into Number? Were the components's UIDs the same in the registry? Was the order of the components within Item supposed to be any order, or was the order supposed to be fixed?

The current SID DTD can be seen at http://ebxml.org/specs/ebCCDOC.pdf, Section 9.1, and an example of a SID document can be seen ibid, section 11.3

Implementation

The Assembly and Context rules are not intended to be, per se, rules that can be applied directly through a specialized piece of software. Rather, the intention is that they should be used as machine readable rules that current mechanisms can exploit. For instance, one possible implementation would be an XSLT stylesheet which, upon reading the rules, would create another XSLT stylesheet that would apply the semantics of the rules.

Why is the above statement necessary? Because it is very common for people to misinterpret things like the Context element, and think that it applies immediately when the rules are read. However, that is not the intention. Although there is no explicit prohibition of a one-pass process, the expectation is that the process will be two-pass (at least conceptually), and that Context refinement, for instance, will be applied only during the second pass.

Conclusion

In this paper we have explored the business background against which the need for an explicit statement of rules grew; we have also seen what these rules are, how they are expressed, and what are some of the interesting things that can be said about them.

We believe that the assertion that one cannot speak of business schemas without introducing the concept of context (and the capability for refining schemas according to it) is perhaps one of the most important legacies of the ebXML effort. Without context rules, that part of ebXML that deals with business schemas is indistinguishable from the EDI efforts, with all the problems that this implies: monolithic, all encompassing schemas, or thousands of canonical schema variations. With context rules, the landscape changes: schema negotiation, dynamic schema generation, semantic interoperability amongst schemas, all become possible, and we believe that in the coming years they will enable ebXML-based business transactions to fulfill the promise of cheap, off-the-shelf, globally available e-commerce. Much needs to be done yet; what we have at present is just a first approximation.


Bibliography

[1] http://ebxml.org

[10] Naming Convention for Core Components v1.04 (http://ebxml.org/specs/ebCCNAM.pdf)

[2] Catalog of Common Business Processes v 1.0 (http://ebxml.org/specs/bpPROC.pdf)

[3] Extensible Markup Language (XML) 1.0 (Second Edition) (http://www.w3.org/TR/2000/REC-xml-20001006)

[4] XSL Transformations (XSLT) Version 1.0 (http://www.w3.org/TR/xslt)

[5] Registry Services Specification v1.0 (http://ebxml.org/specs/ebRS.pdf)

[6] RELAX NG (http://oasis-open.org/committees/relax-ng/)

[7] W3C XML Schema Language (http://www.w3.org/XML/Schema#dev)

[8] Document Assembly and Context Rules v1.04 (http://ebxml.org/specs/ebCCDOC.pdf)

[9] Catalogue of Context Drivers v1.04 (http://ebxml.org/specs/ccDRIV.pdf)



XML-based Rules

Eduardo Gutentag [Sun Microsystems, XML Technology Center]
Arofan Gregory [Commerce One, Commerce One Labs, xCBL Division]