# ServingXML Examples

### Daniel Parker

### Shaun Woodrow

Contributor of "trades" example

### Dimitrios Varsos

Contributor of "tasks" and "timesheets" examples

### Pedro Mendoza

Contributor of "hot" example

### Pierre-Alexandre Losson

Contributor of "invoices" example

### Aswin Majjiga

Contributor of "books-csv" example

### Dudley

Contributor of Java properties "nested tags" examples

### alexbena

Contributor of "CONV" example

### Zach

Contributor of "ars" example

### Sapna

Contributor of "BNF like flat file" example

### Bill Donovan

Contributor of edifact example

### Ahmadin

Contributor of abtest example

### Scott

Contributor of swath example

### Peter Scholz

Contributor of "segments" example

### Wonne Keysers

Contributor of "persons" example

### Ken Brewer

Contributor of "students" example

### Bala Murali

Contributor of "book orders" and "checked books" examples

### etdwh

Contributor of "opdef" example

### David Leal Valma

Contributor of "field has max width" example

### Anton Melumad

Contributor of "optional elements" example

### Lance Zant

Contributor of "XML-flat header-detail" example

### dealogic

Contributor of XML to flat "sequence" example

### Jim Williams

Contributor of XML to flat "multiple summaries" example

### Alvin

Contributor of XML to flat "book-listing" example

### Jim Nurii

Contributor of flat to XML and XML to flat "special char" examples

### Jose Ignacio Santa Cruz

Contributor of flat to XML "invoice" example

### Valentine

Contributor of XML to flat "ungrouping" example

### Ravikumar Tadysetty

Contributor of XML to flat "all" example

### José Santos

Contributor of XML to flat "formating distinct record lines" example

### Ramesh Ramasamy

Contributor of XML to flat "book hierarchy" example

### Gordon Zhang

Contributed additions to "exotics" example

### Roopashree Hirasave

Contributed "X12 XML to Flat File" example

### Ralph Kutzner

Contributed Flat-to-XML example "Computing an Aggregate Value over Multiple Records"

### Randal Cobb

Contributed Flat-to-XML example "Mapping Two Records at the Top of a File to a Header Element in the XML Output"

### Nicolas Denis

Contributed Flat-to-XML example "A UTF-8 positional flat file with double byte characters"

### Johan Hammar

Contributed Flat-to-XML example "Converting a positional flat file with variable field widths to XML"

### Asanga Porage

Contributed the "eobclaims" examples, the flat-to-XML "Flat File to Summary and Detail XML Documents", and the XML-to-flat "Summary and Detail XML Documents to a Flat File"

### Luca Buraggi

Contributed the XML-to-Database "Load Master Detail" example.

### Graham Hannington

Contributed the Flat-File-to-XML "SMF" example.

### Sajeev Nair

Contributed the Flat-File-to-XML "Record Concatenation" example.

### Gabriele Siviero

Contributed the Flat-File-to-XML "Invoic96A" example.

The source for console app examples may be found in the distribution under the `samples` directory.

Before running the sample console applications, be sure to build the servingxml project. Note that the examples will be deployed under the `target/servingxml/samples` directory. See Getting Started for details.

## Flat File to XML

### Delimited Fields, Single Record Type

#### Converting a CSV File to XML Using Default Record Mapping (countries-default)

This example shows a simple resources script that will map a CSV file to XML using default record mapping.

The input file is a CSV file.

Figure 1. Input flat file countries.csv

    code,name
    #ABW,ARUBA
    ATF,"FRENCH SOUTHERN TERRITORIES, D.R. OF"
    VUT,VANUATU
    WLF,WALLIS & FUTUNA ISLANDS

This file has a number of properties.

- The first row is a header with comma-separated column labels.

- The second row is commented out.

- The third row has a field value, enclosed in quotes, that contains an embedded comma.

- The fifth row has a field value that contains a '&' symbol, which is a special character in XML, and must therefore be escaped in XML output as the entity '&amp;'.

Shown below is the simplest possible resources script that will read this data and output it as XML.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="countries">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="countries-doc"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="countries-doc" name="countries">
        <sx:flatFileReader>
          <sx:commentStarter value="#"/>
          <sx:fieldDelimiter value=","/>
          <sx:urlSource url="data/countries.csv"/>
        </sx:flatFileReader>
      </sx:recordContent>

    </sx:resources>

This resources script produces the following output.

    <?xml version="1.0" encoding="utf-8"?>
    <countries>
      <record>
        <code>ATF</code>
        <name>FRENCH SOUTHERN TERRITORIES, D.R. OF</name>
      </record>
      <record>
        <code>VUT</code>
        <name>VANUATU</name>
      </record>
      <record>
        <code>WLF</code>
        <name>WALLIS &amp; FUTUNA ISLANDS</name>
      </record>
    </countries>

Note the following properties of the XML output.

- The root element name comes from the sx:recordContent element's `name` attribute, if it has one, and if not it defaults to `document` .

- The record element name defaults to `record` .

- Each field is mapped to an element, and the name of the element defaults to the column label.

#### Converting a CSV File to XML Using Custom Record Mapping (countries-custom)

An sx:recordMapping element positioned inside an sx:recordContent element allows for custom record mapping to XML. This is optional, since there is a default mapping that emits the canonical XML representation of records. But typically the XML you want is different from the canonical representation, you may want a field mapped to an attribute rather than an element, for example, or perhaps some additional literal elements around the field mappings.

Continuing with the previous example, suppose you want the following output.

    <?xml version="1.0" encoding="utf-8"?>
    <countries>
      <country countryCode="ATF">
        <countryName>FRENCH SOUTHERN TERRITORIES, D.R. OF</countryName>
      </country>
      <country countryCode="VUT">
        <countryName>VANUATU</countryName>
      </country>
      <country countryCode="WLF">
        <countryName>WALLIS &amp; FUTUNA ISLANDS</countryName>
      </country>
    </countries>

The following resources script does the job.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="countries">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="countries"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="countries">
        <sx:flatFileReader>
          <sx:urlSource url="data/countries.csv"/>
          <sx:flatFile ref="countriesFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="countriesToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="countriesFile">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="country">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="code"/>
            <sx:delimitedField name="name"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="countriesToXmlMapping">
        <countries>
          <sx:onRecord>
            <country>
              <sx:fieldElementMap field="name" element="countryName"/>
              <sx:fieldAttributeMap field="code" attribute="countryCode"/>
            </country>
          </sx:onRecord>
        </countries>
      </sx:recordMapping>

    </sx:resources>

You can also take the default mappings for some fields and map others explicitly. For example,

    <sx:recordMapping id="countries2xml">
      <countries>
        <sx:onRecord>
          <country>
            <sx:defaultFieldElementMap fields="*" except="code"/>
            <sx:fieldAttributeMap field="code" attribute="countryCode"/>
          </country>
        </sx:onRecord>
      </countries>
    </sx:recordMapping>

Here, the sx:defaultFieldElementMap element maps all the fields in the record to elements with the same names, except the field name "code". This field is handled separately, with the sx:fieldAttributeMap element.

#### Converting a CSV file to XML with Record Validation (countries-validated)

The example below shows how to convert a flat file to XML. Each row in the flat file is validated with Sun's multi schema validator (MSV), and if an error is encountered, the row is discarded, but processing continues. In addition, the output XML is also validated with Sun's MSV, and if that should fail, processing is stopped.

This time the input file has an invalid country code in the first record.

    code,name
    ATFX,"FRENCH SOUTHERN TERRITORIES, D.R. OF"
    VUT,VANUATU
    WLF,WALLIS & FUTUNA ISLANDS

The desired output is

    <?xml version="1.0" encoding="utf-8"?>
    <countries>
      <country countryCode="ATF">
        <countryName>FRENCH SOUTHERN TERRITORIES, D.R. OF</countryName>
      </country>
      <country countryCode="VUT">
        <countryName>VANUATU</countryName>
      </country>
      <country countryCode="WLF">
        <countryName>WALLIS &amp; FUTUNA ISLANDS</countryName>
      </country>
    </countries>

The following resources script does the transformation.

Figure 2. Resources script resources-countries.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="countries">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="countries"/>
            <msv:schemaValidator>
              <sx:urlSource url="data/countries.xsd"/>
            </msv:schemaValidator>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType ref="country"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="country" name="country">
        <sx:fieldDelimiter value=","/>
        <sx:delimitedField name="code"/>
        <sx:delimitedField name="name"/>
      </sx:flatRecordType>

      <sx:flatFile id="discardFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="countryDiscard">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="message"/>
            <sx:flatRecordType ref="country"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="countriesToXmlMapping">
        <countries>
          <sx:onRecord>
            <country>
              <sx:fieldElementMap field="name" element="countryName"/>
              <sx:fieldAttributeMap field="code" attribute="countryCode"/>
            </country>
          </sx:onRecord>
        </countries>
      </sx:recordMapping>

      <sx:recordContent id="countries">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:urlSource url="data/countriesWithDiscards.csv"/>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileReader>
          <msv:recordValidator>
            <sx:urlSource url="data/country-record.xsd"/>
          </msv:recordValidator>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <sx:modifyRecord>
              <sx:newField name="message" value="{$sx:message}"/>
            </sx:modifyRecord>
            <sx:flatFileWriter>
              <sx:fileSink file="output/countryDiscards.csv"/>
              <sx:flatFile ref="discardFile"/>
            </sx:flatFileWriter>
          </sx:discardHandler>
        </sx:recordStream>
        <sx:recordMapping ref="countriesToXmlMapping"/>
      </sx:recordContent>

    </sx:resources>

The schema to validate an individual country record is shown below.

Figure 3. XML Schema file country-record.xsd

    <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

     <!-- This element's name matches the value of the name attribute in the sx:flatRecordType element. -->
     <xs:element name="country" type="CountryType"/>

     <xs:complexType name="CountryType">
      <xs:sequence>
       <xs:element name="code" type="CountryCode"/>
       <xs:element name="name" type="xs:string"/>
      </xs:sequence>
     </xs:complexType>

      <xs:simpleType name='CountryCode'>
        <xs:restriction base='xs:string'>
          <xs:length value='3' fixed='true'/>
        </xs:restriction>
      </xs:simpleType>
    </xs:schema>

You run this example on the command line by entering

    servingxml -r resources-countries.xml -o output/countries.xml countries

The following error message will be written to the log:

    Error in record "country" on line 1.  the length of the value is 4, but the required length is 3.

The remaining two rows will be written to the output file.

#### Converting flat files with fields that end either with a delimiter or a maximum width (field has max width)

The example below shows how to convert a flat file with start/end record delimiters to XML.

The input file is shown below.

    1231234
    1 123
    12 12

This file has the following structure.

- The first record consists of a field of length 3 ("123") followed by a field of length 4 ("1234")

- The second line consists of a field of length 1 ("1") followed by a field of length 3 ("123")

Each field has a maximum width, but the field may be terminated by a delimiter (here whitespace) before it attains that width.

The desired output is shown below.

    <?xml version="1.0" encoding="utf-8"?>
    <document>
      <record>
        <field1>123</field1>
        <field2>1234</field2>
      </record>
      <record>
        <field1>1</field1>
        <field2>123</field2>
      </record>
      <record>
        <field1>12</field1>
        <field2>12</field2>
      </record>
    </document>

The following resources script does the transformation.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="fieldHasMaxWidth">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="fieldHasMaxWidthDoc"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="fieldHasMaxWidthDoc" name="document">
        <sx:flatFileReader>
          <sx:urlSource url="data/fieldHasMaxWidth.txt"/>
          <sx:flatFile ref="fieldHasMaxWidthFile"/>
        </sx:flatFileReader>
      </sx:recordContent>

      <sx:flatFile id="fieldHasMaxWidthFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="record">
            <sx:fieldDelimiter>
              <sx:whitespaceSeparator/>
            </sx:fieldDelimiter>
            <sx:delimitedField name="field1" maxWidth="3"/>
            <sx:delimitedField name="field2" maxWidth="4"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-fieldHasMaxWidth.xml
      -o output/fieldHasMaxWidth.xml fieldHasMaxWidth

#### Converting a flat file to XML with one level of grouping (tasks)

The example below shows how to convert a flat file to XML with grouping levels.

The input file is a CSV file.

Figure 4. Input flat file tasks.csv

    project_id, task_name, task_start, task_finish
    4001,task1,01/01/2003,01/31/2003
    4001,task2,02/01/2003,02/28/2003
    4001,task3,03/01/2003,03/31/2003
    4002,task1,04/01/2003,04/30/2003
    4002,task2,05/01/2003,05/31/2003

The desired output is shown below.

Figure 5. Output XML file tasks.xml

     <?xml version="1.0" encoding="utf-8" ?>
    <Projects>
      <Project projectID="4001">
        <Tasks>
          <Task name="task1" start="01/01/2003" finish="01/31/2003" />
          <Task name="task2" start="02/01/2003" finish="02/28/2003" />
          <Task name="task3" start="03/01/2003" finish="03/31/2003" />
        </Tasks>
      </Project>
      <Project projectID="4002">
        <Tasks>
          <Task name="task1" start="04/01/2003" finish="04/30/2003" />
          <Task name="task2" start="05/01/2003" finish="05/31/2003" />
        </Tasks>
      </Project>
    </Projects>

The following resources script does the transformation.

Figure 6. Resources script resources-tasks.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="tasks">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="tasks"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="tasks">
        <sx:flatFileReader>
          <sx:urlSource url="data/tasks.csv"/>
          <sx:flatFile ref="tasksFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="tasksToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="tasksFlatFile">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="task">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="project_id"/>
            <sx:delimitedField name="task_name"/>
            <sx:delimitedField name="task_start"/>
            <sx:delimitedField name="task_finish"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="tasksToXmlMapping">
        <Projects>
          <sx:groupBy fields="project_id">
            <Project>
              <sx:fieldAttributeMap field="project_id" attribute="projectID"/>
              <Tasks>
                <sx:onRecord>
                  <Task>
                    <sx:fieldAttributeMap field="task_name" attribute="name"/>
                    <sx:fieldAttributeMap field="task_start" attribute="start"/>
                    <sx:fieldAttributeMap field="task_finish" attribute="finish"/>
                  </Task>
                </sx:onRecord>
              </Tasks>
            </Project>
          </sx:groupBy>
        </Projects>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-tasks.xml -o output/tasks.xml tasks

#### Converting a flat file to XML with many levels of grouping (timesheets)

The example below shows how to convert a flat file to XML with many grouping levels.

The input file is a CSV file.

Figure 7. Input flat file timesheets.csv

    TimePeriod,Start,Finish,Resource,Task,ActualDate,Amount
    1,2/9/2004,2/15/2004,Joe_100,1001,2/9/2004,8
    1,2/9/2004,2/15/2004,Joe_100,1001,2/10/2004,4
    1,2/9/2004,2/15/2004,Joe_100,1002,2/11/2004,8
    1,2/9/2004,2/15/2004,Joe_100,1003,2/10/2004,4
    1,2/9/2004,2/15/2004,Joe_100,1003,2/12/2004,8
    1,2/9/2004,2/15/2004,Mark_101,1001,2/10/2004,8
    1,2/9/2004,2/15/2004,Mark_101,1001,2/14/2004,8
    1,2/9/2004,2/15/2004,Mark_101,1006,2/9/2004,8
    1,2/9/2004,2/15/2004,Mark_101,1008,2/12/2004,4
    1,2/9/2004,2/15/2004,Mark_101,1008,2/13/2004,4
    2,2/16/2004,2/22/2004,Joe_100,1001,2/16/2004,8
    2,2/16/2004,2/22/2004,Joe_100,1001,2/17/2004,4
    2,2/16/2004,2/22/2004,Joe_100,1002,2/17/2004,4

The desired output is shown below.

Figure 8. Output XML file timesheets.xml

    <?xml version="1.0" encoding="utf-8"?>
    <TimePeriods>
      <TimePeriod start="2/9/2004" finish="2/15/2004">
        <Timesheets>
          <Timesheet resource="Joe_100">
            <TimesheetEntries>
              <TimesheetEntry task="1001">
                <DailyActuals>
                  <Actual actualDate="2/9/2004" amount="8"/>
                  <Actual actualDate="2/10/2004" amount="4"/>
                </DailyActuals>
              </TimesheetEntry>
              <TimesheetEntry task="1002">
                <DailyActuals>
                  <Actual actualDate="2/11/2004" amount="8"/>
                </DailyActuals>
              </TimesheetEntry>
              <TimesheetEntry task="1003">
                <DailyActuals>
                  <Actual actualDate="2/10/2004" amount="4"/>
                  <Actual actualDate="2/12/2004" amount="8"/>
                </DailyActuals>
              </TimesheetEntry>
            </TimesheetEntries>
          </Timesheet>
          <Timesheet resource="Mark_101">
            <TimesheetEntries>
              <TimesheetEntry task="1001">
                <DailyActuals>
                  <Actual actualDate="2/10/2004" amount="8"/>
                  <Actual actualDate="2/14/2004" amount="8"/>
                </DailyActuals>
              </TimesheetEntry>
              <TimesheetEntry task="1006">
                <DailyActuals>
                  <Actual actualDate="2/9/2004" amount="8"/>
                </DailyActuals>
              </TimesheetEntry>
              <TimesheetEntry task="1008">
                <DailyActuals>
                  <Actual actualDate="2/12/2004" amount="4"/>
                  <Actual actualDate="2/13/2004" amount="4"/>
                </DailyActuals>
              </TimesheetEntry>
            </TimesheetEntries>
          </Timesheet>
        </Timesheets>
      </TimePeriod>
      <TimePeriod start="2/16/2004" finish="2/22/2004">
        <Timesheets>
          <Timesheet resource="Joe_100">
            <TimesheetEntries>
              <TimesheetEntry task="1001">
                <DailyActuals>
                  <Actual actualDate="2/16/2004" amount="8"/>
                  <Actual actualDate="2/17/2004" amount="4"/>
                </DailyActuals>
              </TimesheetEntry>
              <TimesheetEntry task="1002">
                <DailyActuals>
                  <Actual actualDate="2/17/2004" amount="4"/>
                </DailyActuals>
              </TimesheetEntry>
            </TimesheetEntries>
          </Timesheet>
        </Timesheets>
      </TimePeriod>
    </TimePeriods>

The following resources script does the transformation.

Figure 9. Resources script resources-timesheets.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="timesheets">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="timesheets"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="timesheets">
        <sx:flatFileReader>
          <sx:urlSource url="data/timesheets.csv"/>
          <sx:flatFile ref="timesheetsFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="timesheetsToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="timesheetsFlatFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="task">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="TimePeriod"/>
            <sx:delimitedField name="Start"/>
            <sx:delimitedField name="Finish"/>
            <sx:delimitedField name="Resource"/>
            <sx:delimitedField name="Task"/>
            <sx:delimitedField name="ActualDate"/>
            <sx:delimitedField name="Amount"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="timesheetsToXmlMapping">
        <TimePeriods>
          <sx:groupBy fields="Start">
            <TimePeriod>
              <sx:fieldAttributeMap field="Start" attribute="start"/>
              <sx:fieldAttributeMap field="Finish" attribute="finish"/>
              <Timesheets>
                <sx:groupBy fields="Start Resource">
                  <Timesheet>
                    <sx:fieldAttributeMap field="Resource" attribute="resource"/>
                    <TimesheetEntries>
                      <sx:groupBy fields="Start Resource Task">
                        <TimesheetEntry>
                          <sx:fieldAttributeMap field="Task" attribute="task"/>
                          <DailyActuals>
                            <sx:onRecord>
                              <Actual>
                                <sx:fieldAttributeMap field="ActualDate" attribute="actualDate"/>
                                <sx:fieldAttributeMap field="Amount" attribute="amount"/>
                              </Actual>
                            </sx:onRecord>
                          </DailyActuals>
                        </TimesheetEntry>
                      </sx:groupBy>
                    </TimesheetEntries>
                  </Timesheet>
                </sx:groupBy>
              </Timesheets>
            </TimePeriod>
          </sx:groupBy>
        </TimePeriods>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-timesheets.xml -o output/timesheets.xml timesheets

#### Converting a flat file with multi-valued fields to XML (family_data)

The example below shows how to convert a flat file with multi-valued fields to XML.

The input file is a pipe delimited flat file with semicolon sub-delimiters.

Figure 10. Input flat file employees.txt

    father_name|mother_name|children
    Matthew|Sarah|
    Scott||Damian;Janet;Paul

The desired output is shown below.

Figure 11. Output XML file multivalued.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <eg:families xmlns:eg="http://examples.com/">
      <eg:family>
        <eg:father-name>Matthew</eg:father-name>
        <eg:mother-name>Sarah</eg:mother-name>
        <eg:children/>
      </eg:family>
      <eg:family>
        <eg:father-name>Scott</eg:father-name>
        <eg:mother-name/>
        <eg:children>
          <eg:child>Damian</eg:child>
          <eg:child>Janet</eg:child>
          <eg:child>Paul</eg:child>
        </eg:children>
      </eg:family>
    </eg:families>

The following resources script does the transformation.

Figure 12. Resources script resources-multivalued.xml

    <?xml version="1.0"?>
    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                        xmlns:eg="http://examples.com/">

      <sx:service id="families">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="families"/>
            <!-- sx:removeEmptyElements elements="eg:children"/ -->
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="families">
        <sx:flatFileReader>
          <sx:flatFile ref="family-records"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="families-to-xml-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="family-records">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="family">
            <sx:fieldDelimiter value="|"/>
            <sx:delimitedField name="father_name"/>
            <sx:delimitedField name="mother_name"/>
            <sx:delimitedField name="children">
              <sx:subfieldDelimiter value=";"/>
            </sx:delimitedField>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="families-to-xml-mapping">
        <eg:families xmlns:eg="http://examples.com/">
          <sx:onRecord>
            <eg:family>
              <sx:fieldElementMap field="father_name" element="eg:father-name"/>
              <sx:fieldElementMap field="mother_name" element="eg:mother-name"/>
              <eg:children>
                <sx:fieldElementSequenceMap field="children" element="eg:child"/>
              </eg:children>
            </eg:family>
          </sx:onRecord>
        </eg:families>
      </sx:recordMapping>

    </sx:resources>

##### Remarks

- Note that Matthew and Sarah have no children. If you wanted to remove the empty `myns:children` element, uncomment the sx:removeEmptyElements element in the sx:transform block of the resources script,

You can run this example on the command line by entering

    servingxml -r resources-family_data.xml
        -i data/family_data.txt -o output/family_data.xml families

#### Search and Replace with Regular Expressions in Field Mappings (ars)

The example below shows how to convert a flat file to XML using search and replace with regular expressions in field mappings.

The input file is a carot (^) delimited flat file with a header.

Figure 13. Input flat file ars.dat

    ProjectID^Project_Name^Description^PLAN_IT_RESC_DAYS^PLAN_IT_COST^ANNUAL_EBIT^ACTUAL_NPV^ACTUAL_IRR
    1234-00-0005^XOG new project insert^This is to test and verify the XOG'ing of new Project data from the ARS database.^430^59,627^100,351^3^9.2

Note the following features of this data.

- The first row is a header row.

- The fourth and fifth fields contain embedded commas in numbers, which we want to strip out.

The desired output is shown below.

Figure 14. Output XML file ars.xml

    <?xml version="1.0" encoding="utf-8"?>
    <NikuDataBus xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
      <Header action="write" externalSource="NIKU" objectType="project" version="6.0.11"/>
      <Projects>
        <Project name="XOG new project insert" projectID="1234-00-0005" description="This is to test and verify the XOG'ing of new Project data from the ARS database.">
          <CustomInformation>
            <PLAN_IT_RESOURCE_DAYS>430</PLAN_IT_RESOURCE_DAYS>
            <PLAN_IT_COST>59627</PLAN_IT_COST>
            <ANNUAL_EBIT>100351</ANNUAL_EBIT>
            <ACTUAL_NPV>3</ACTUAL_NPV>
            <ACTUAL_IRR>9.2</ACTUAL_IRR>
          </CustomInformation>
          <General addedBy="XOG" addedDate="03/26/2005 8:10:09 PM"/>
        </Project>
      </Projects>
    </NikuDataBus>

The following resources script does the transformation.

Figure 15. Resources script resources-ars.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="ars">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="ars"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="ars">
        <sx:flatFileReader>
          <sx:flatFile ref="arsFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="arsToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="arsFlatFile">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="detail">
            <sx:fieldDelimiter value="^"/>
            <sx:delimitedField name="ProjectID"/>
            <sx:delimitedField name="Project_Name"/>
            <sx:delimitedField name="Description"/>
            <sx:delimitedField name="PLAN_IT_RESC_DAYS"/>
            <sx:delimitedField name="PLAN_IT_COST"/>
            <sx:delimitedField name="ANNUAL_EBIT"/>
            <sx:delimitedField name="ACTUAL_NPV"/>
            <sx:delimitedField name="ACTUAL_IRR"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="arsToXmlMapping">
        <NikuDataBus xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

          <Header version="6.0.11" action="write" objectType="project" externalSource="NIKU"/>
          <Projects>
            <sx:groupBy fields="ProjectID">
              <Project>
                <sx:fieldAttributeMap field="Project_Name" attribute="name"/>
                <sx:fieldAttributeMap field="ProjectID" attribute="projectID"/>
                <sx:fieldAttributeMap field="Description" attribute="description"/>
                <sx:onRecord>
                  <CustomInformation>
                    <sx:fieldElementMap field="PLAN_IT_RESC_DAYS" element="PLAN_IT_RESOURCE_DAYS"/>
                    <sx:elementMap element="PLAN_IT_COST">
                      <sx:findAndReplace searchFor ="," replaceWith ="">
                        <sx:toString value="{PLAN_IT_COST}"/>
                      </sx:findAndReplace>
                    </sx:elementMap>
                    <sx:elementMap element="ANNUAL_EBIT">
                      <sx:findAndReplace searchFor ="," replaceWith ="">
                        <sx:toString value="{ANNUAL_EBIT}"/>
                      </sx:findAndReplace>
                    </sx:elementMap>
                    <sx:fieldElementMap field="ACTUAL_NPV" element="ACTUAL_NPV"/>
                    <sx:fieldElementMap field="ACTUAL_IRR" element="ACTUAL_IRR"/>
                  </CustomInformation>
                  <General addedBy="XOG">
                    <sx:fieldAttributeMap attribute="addedDate">
                      <sx:formatDateTime format="MM/dd/yyyy h:mm:ss a">
                        <sx:currentDateTime/>
                      </sx:formatDateTime>
                    </sx:fieldAttributeMap>
                  </General>
                </sx:onRecord>
              </Project>
            </sx:groupBy>
          </Projects>
        </NikuDataBus>
      </sx:recordMapping>

    </sx:resources>

Note the following points about this script.

- The sx:currentDateTime element evaluates to the current date with a lexical representation as defined for xs:dateTime of XML Schema Part 2: Datatypes .

- The sx:formatDateTime formats the XML Schema lexical representation of a date using a format string which must must follow the syntax specified for the JDK SimpleDateFormat class..

You can run this example on the command line by entering

    servingxml -r resources-ars.xml -i data/ars.txt -o output/ars.xml ars

#### Converting a Tab Delimited Flat File to XML (tab-delimited)

The example below shows how to convert a flat file with tab delimiters to XML.

The input file is a tab (0x09) delimited flat file with a header.

Figure 16. Input flat file tab_delimited_employees.txt

    employee-no	employee-name	dept	salary
    00000001	Smith, Matthew	sales	150,000.00
    00000002	Brown, Sarah	sales	89,000.00
    00000003	Oberc, Scott	finance	110,000.00
    00000004	Scott, Colette	sales	75,000.00

The desired output is shown below.

Figure 17. Output XML file employees.xml

    <?xml version="1.0" encoding="utf-8"?>
    <employees>
      <employee>
        <employee-no>00000001</employee-no>
        <employee-name>Smith, Matthew</employee-name>
        <department>sales</department>
        <salary>150,000.00</salary>
      </employee>
      <employee>
        <employee-no>00000002</employee-no>
        <employee-name>Brown, Sarah</employee-name>
        <department>sales</department>
        <salary>89,000.00</salary>
      </employee>
      <employee>
        <employee-no>00000003</employee-no>
        <employee-name>Oberc, Scott</employee-name>
        <department>finance</department>
        <salary>110,000.00</salary>
      </employee>
      <employee>
        <employee-no>00000004</employee-no>
        <employee-name>Scott, Colette</employee-name>
        <department>sales</department>
        <salary>75,000.00</salary>
      </employee>
    </employees>

The following resources script does the transformation.

Figure 18. Resources script resources-tab_delimited_employees.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="employee-data"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="employee-data" name="employees">
        <sx:flatFileReader>
          <sx:urlSource url="data/tab_delimited_employees.txt"/>
          <sx:flatFile ref="employee-file"/>
        </sx:flatFileReader>
      </sx:recordContent>

      <sx:flatFile id="employee-file">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:fieldDelimiter value="\t"/>
          <!-- Another way of specifying a horizontal tab character -->
          <!-- <sx:fieldDelimiter value="&#x09;"/> -->
          <sx:flatRecordType name="employee">
            <sx:delimitedField name="employee-no"/>
            <sx:delimitedField name="employee-name"/>
            <sx:delimitedField name="department"/>
            <sx:delimitedField name="salary"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

Note the following points about this script.

- The tab delimiter may be specified either with the `\t` symbol, or, alternatively, with the numerical entity reference &#x09; numerical entity

You can run this example on the command line by entering

    servingxml -o output/employees.xml
        -r resources-tab_delimited_employees.xml employees

### Delimited Fields, Multiple Record Types

#### Converting a flat file with multiple record formats to XML with grouping (exotics)

The example below shows how to convert a pipe delimited flat file with multiple record formats to XM with grouping.

The input file is a pipe delimited file.

Figure 19. Input flat file exotics.txt

    SWAP|1234567|FLOAT|PAY|CAD|BA|2010/04/28|10000000.00
    Cap|1234567||CAD|SELL|2010/04/28|0.03
    SWAP|1234567|FIX|REC|CAD
    Cap|1234568||CAD|SELL|2010/04/28|0.05

The desired output is shown below.

Figure 20. Output XML file exotics.xml

    <exotics>
      <trade tradeId="1234567">
        <option>SWAP</option>
        <swapLeg style="FLOAT">
          <side>PAY</side>
          <maturityDate original="2010/04/28">2001/04/29</maturityDate>
          <currency>CAD</currency>
          <index>BA</index>
          <notional>10000000.00</notional>
        </swapLeg>
        <capSell>
          <buySell>SELL</buySell>
          <maturityDate original="2010/04/28">2001/04/28</maturityDate>
          <strike>0.03</strike>
        </capSell>
        <swapLeg style="FIX">
          <side>REC</side>
        </swapLeg>
      </trade>
      <trade tradeId="1234568">
        <option>Cap</option>
        <capBuy>
          <buySell>BUY</buySell>
          <maturityDate original="2010/04/28">2001/04/28</maturityDate>
          <strike>0.05</strike>
        </capBuy>
      </trade>
    </exotics>

The following resources script does the transformation.

Figure 21. Resources script resources-exotics.xml

    <?xml version="1.0"?>

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="exotics">
    	<sx:serialize>
    	  <sx:transform>
    		<sx:content ref="exotics"/>
    	  </sx:transform>
    	</sx:serialize>
      </sx:service>

      <sx:recordContent id="exotics">
    	<sx:flatFileReader>
    	  <sx:fileSource file="data/exotics.txt"/>
    	  <sx:flatFile ref="exoticsFlatFile"/>
    	</sx:flatFileReader>
    	<sx:recordMapping ref="exoticsToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="exoticsFlatFile">
    	<sx:flatFileBody>
    	  <sx:flatRecordTypeChoice>
    		<sx:fieldDelimiter value="|"/>
    		<sx:delimitedField name="recordType"/>
    		<sx:delimitedField name="tradeId"/>
    		<sx:delimitedField name="style"/>
    		<sx:when test="recordType='Cap'">
    		  <sx:flatRecordType name='Cap'>
    			<sx:fieldDelimiter value="|"/>
    			<sx:delimitedField name="recordType"/>
    			<sx:delimitedField name="tradeId"/>
    			<sx:delimitedField name="placeholder1"/>
    			<sx:delimitedField name="currency"/>
    			<sx:delimitedField name="buySell"/>
    			<sx:delimitedField name="maturityDate"/>
    			<sx:delimitedField name="strike"/>
    		  </sx:flatRecordType>
    		</sx:when>
    		<sx:when test="recordType='SWAP' and style='FLOAT'">
    		  <sx:flatRecordType name='SwapFloatingLeg'>
    			<sx:fieldDelimiter value="|"/>
    			<sx:delimitedField name="recordType"/>
    			<sx:delimitedField name="tradeId"/>
    			<sx:delimitedField name="style"/>
    			<sx:delimitedField name="side"/>
    			<sx:delimitedField name="currency"/>
    			<sx:delimitedField name="index"/>
    			<sx:delimitedField name="maturityDate"/>
    			<sx:delimitedField name="notional"/>
    		  </sx:flatRecordType>
    		</sx:when>
    		<sx:when test="recordType='SWAP' and style='FIX'">
    		  <sx:flatRecordType name='SwapFixedLeg'>
    			<sx:fieldDelimiter value="|"/>
    			<sx:delimitedField name="recordType"/>
    			<sx:delimitedField name="tradeId"/>
    			<sx:delimitedField name="style"/>
    			<sx:delimitedField name="side"/>
    			<sx:delimitedField name="placeholder1"/>
    			<sx:delimitedField name="currency"/>
    		  </sx:flatRecordType>
    		</sx:when>
    	  </sx:flatRecordTypeChoice>
    	</sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="exoticsToXmlMapping">
    	<exotics>
    	  <sx:groupBy fields="tradeId">
    		<trade><option>
    		  <sx:choose>
    			<sx:when test="recordType='Cap' or (recordType='SWAP' and style='FLOAT')">
    			  <sx:toString value="{recordType}"/>
    			</sx:when>
    		  </sx:choose></option>
    		  <sx:fieldAttributeMap field="tradeId" attribute="tradeId"/>
    		  <sx:onRecord recordType="Cap">
    			  <sx:choose>
    				<sx:when test="buySell='BUY'">
    		 	<capBuy>
    			  <sx:fieldElementMap field="buySell" element="buySell"/>
    			  <sx:elementMap element="maturityDate">
    				<sx:fieldAttributeMap field="maturityDate" attribute="original"/>
    			  	<sx:choose>
    			  		<sx:when test="maturityDate='2010/04/28'">
    			  			<sx:toString value="2001/04/28"/>
    			  		</sx:when>
    			  	</sx:choose>
    			  </sx:elementMap>
    			  <sx:fieldElementMap field="strike" element="strike"/>
    			</capBuy>
    				</sx:when>
    				<sx:otherwise>
    		 	<capSell>
    			  <sx:fieldElementMap field="buySell" element="buySell"/>
    			  <sx:elementMap element="maturityDate">
    				<sx:fieldAttributeMap field="maturityDate" attribute="original"/>
    			  	<sx:choose>
    			  		<sx:when test="maturityDate='2010/04/28'">
    			  			<sx:toString value="2001/04/28"/>
    			  		</sx:when>
    			  	</sx:choose>
    			  </sx:elementMap>
    			  <sx:fieldElementMap field="strike" element="strike"/>
    			</capSell>
    				</sx:otherwise>
    			  </sx:choose>
    		  </sx:onRecord>
    		  <sx:onRecord recordType="SwapFloatingLeg">
    			<swapLeg>
    			  <sx:fieldAttributeMap field="style" attribute="style"/>
    			  <sx:fieldElementMap field="side" element="side"/>
    			  <maturityDate>
    				<sx:fieldAttributeMap field="maturityDate" attribute="original"/>
    			  	<sx:choose>
    			  		<sx:when test="maturityDate='2010/04/28'">
    			  			<sx:toString value="2001/04/29"/>
    			  		</sx:when>
    			  	</sx:choose>
    			  </maturityDate>
    			  <sx:fieldElementMap field="currency" element="currency"/>
    			  <sx:fieldElementMap field="index" element="index"/>
    			  <sx:fieldElementMap field="notional" element="notional"/>
    			</swapLeg>
    		  </sx:onRecord>
    		  <sx:onRecord recordType="SwapFixedLeg">
    			<swapLeg>
    			  <sx:fieldAttributeMap field="style" attribute="style"/>
    			  <sx:fieldElementMap field="side" element="side"/>
    			</swapLeg>
    		  </sx:onRecord>
    		</trade>
    	  </sx:groupBy>
    	</exotics>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-exotics.xml -o output/exotics.xml exotics

#### Converting flat files with record mappings that have optional elements (optional elements)

The example below shows how to convert a flat file to XML by applying an XSLT transform to the canonical XML representation of each record.

The input file is shown below.

    262025218|John|Smith|657827168||1|Main Street|London|45879|56|||||
    262091012|Jane|Bell||000000000000|105|Penny Lane|Oxford|12345|56|1|Main Street|London|45879|56

Each line has an id, followed by a first and last name, followed by a customer address, and optionally a billing address. If the house number and street of the billing address are empty, the rule is that the entire billing address be omitted from the XML output.

The desired output is shown below.

    <?xml version="1.0" encoding="utf-8"?>
    <mns:Message xmlns:mns="www.abc.com/abc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="www.abc.com/abc abc_v_1.1.xsd ">
       <wf_id>262025218</wf_id>
       <timeStamp>2006-02-11T22:18:21</timeStamp>
       <MessageData>
          <addCustomerRequest>
             <foreName>John</foreName>
             <surName>Smith</surName>
             <custAddress>
                <houseNumber>1</houseNumber>
                <street>Main Street</street>
                <town>London</town>
                <postCode>45879</postCode>
                <country>56</country>
             </custAddress>
          </addCustomerRequest>
       </MessageData>
       <wf_id>262091012</wf_id>
       <timeStamp>2006-02-11T22:18:21</timeStamp>
       <MessageData>
          <addCustomerRequest>
             <foreName>Jane</foreName>
             <surName>Bell</surName>
             <custAddress>
                <houseNumber>105</houseNumber>
                <street>Penny Lane</street>
                <town>Oxford</town>
                <postCode>12345</postCode>
                <country>56</country>
             </custAddress>
             <billAddress>
                <houseNumber>1</houseNumber>
                <street>Main Street</street>
                <town>London</town>
                <postCode>45879</postCode>
                <country>56</country>
             </billAddress>
          </addCustomerRequest>
       </MessageData>
    </mns:Message>

The following resources script does the transformation.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="addCustomer">
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:content ref="addCustomer"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:flatFile id="customerFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="customer">
            <sx:fieldDelimiter value="|"/>
            <sx:delimitedField name="rowid"/>
            <sx:delimitedField name="forename"/>
            <sx:delimitedField name="surname"/>
            <sx:delimitedField name="vatnumber"/>
            <sx:delimitedField name="ninumber"/>
            <sx:delimitedField name="custaddr_house_nr"/>
            <sx:delimitedField name="custaddr_street"/>
            <sx:delimitedField name="custaddr_town"/>
            <sx:delimitedField name="custaddr_postcode"/>
            <sx:delimitedField name="custaddr_country"/>
            <sx:delimitedField name="billaddr_house_nr"/>
            <sx:delimitedField name="billaddr_street"/>
            <sx:delimitedField name="billaddr_town"/>
            <sx:delimitedField name="billaddr_postcode"/>
            <sx:delimitedField name="billaddr_country"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordContent id="addCustomer">
        <sx:flatFileReader>
          <sx:fileSource file="data/optionalElements.txt"/>
          <sx:flatFile ref="customerFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="addCustomerToXmlMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="addCustomerToXmlMapping">
        <mns:Message xmlns:mns="www.abc.com/abc"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="www.abc.com/abc abc_v_1.1.xsd">
          <sx:onRecord>
            <sx:fieldElementMap field="rowid" element="wf_id"/>
            <timeStamp>
              <sx:formatDateTime format = "yyyy-MM-dd'T'HH:mm:ss">
                <sx:currentDateTime/>
              </sx:formatDateTime>
            </timeStamp>
            <sx:transformRecord>
              <sx:xslt>
                <xsl:transform version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
                  <xsl:template match="customer">
                    <MessageData>
                      <addCustomerRequest>
                        <xsl:element name="foreName">
                          <xsl:value-of select="forename"/>
                        </xsl:element>
                        <xsl:element name="surName">
                          <xsl:value-of select="surname"/>
                        </xsl:element>
                        <custAddress>
                          <xsl:element name="houseNumber">
                            <xsl:value-of select="custaddr_house_nr"/>
                          </xsl:element>
                          <xsl:element name="street">
                            <xsl:value-of select="custaddr_street"/>
                          </xsl:element>
                          <xsl:element name="town">
                            <xsl:value-of select="custaddr_town"/>
                          </xsl:element>
                          <xsl:element name="postCode">
                            <xsl:value-of select="custaddr_postcode"/>
                          </xsl:element>
                          <xsl:element name="country">
                            <xsl:value-of select="custaddr_country"/>
                          </xsl:element>
                        </custAddress>
                        <xsl:if test="string(billaddr_house_nr) and string(billaddr_street)">
                          <billAddress>
                            <xsl:element name="houseNumber">
                              <xsl:value-of select="billaddr_house_nr"/>
                            </xsl:element>
                            <xsl:element name="street">
                              <xsl:value-of select="billaddr_street"/>
                            </xsl:element>
                            <xsl:element name="town">
                              <xsl:value-of select="billaddr_town"/>
                            </xsl:element>
                            <xsl:element name="postCode">
                              <xsl:value-of select="billaddr_postcode"/>
                            </xsl:element>
                            <xsl:element name="country">
                              <xsl:value-of select="billaddr_country"/>
                            </xsl:element>
                          </billAddress>
                        </xsl:if>
                      </addCustomerRequest>
                    </MessageData>
                  </xsl:template>
                </xsl:transform>
              </sx:xslt>
            </sx:transformRecord>
          </sx:onRecord>
        </mns:Message>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-optionalElements.xml
      -o output/optionalElements.xml optionalElements

#### Converting a Java properties file to XML (messages)

The example below shows how to convert a Java properties file to XML.

The input file is a Java properties file.

Figure 22. Input property file messages.properties

    parser.ope.2=')' or '-[' or '+[' or '&[' is expected.
    parser.factor.5=A back reference or an anchor or a lookahead or a lookbehind \
    is expected in a conditional pattern.
    parser.next.2='?' is not expected.  '(?:' or '(?=' or '(?!' or '(?<' or '(?#' or '(?>'?
    #parser.next.3='(?<=' or '(?<!' is expected.

This file has a number of features.

- The entries are key-value pairs separated by the '=' symbol.

- The value in the first line contains an '&' symbol, which is a special character in XML, and will be escaped in the output file.

- The second line ends with a continuation character.

- The value in the third line contains an '=' symbol.

- The last line starts with a comment symbol.

The desired output is shown below.

Figure 23. Output XML file messages.xml

    <?xml version="1.0" encoding="utf-8"?>
    <messages>
      <parser.ope.2>')' or '-[' or '+[' or '&amp;[' is expected.</parser.ope.2>
      <parser.factor.5>A back reference or an anchor or a lookahead or a lookbehind is expected in a conditional pattern.</parser.factor.5>
      <parser.next.2>'?' is not expected.  '(?:' or '(?=' or '(?!' or '(?&lt;' or '(?#' or '(?&gt;'?</parser.next.2>
    </messages>

The following resources script does the transformation.

Figure 24. Resources script resources-messages.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="messages">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="messages"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="messages">
        <sx:flatFileReader>
          <sx:urlSource url="data/messages.properties"/>
          <sx:flatFile>
            <sx:recordDelimiter continuation="\" value="\r\n"/>
            <sx:recordDelimiter continuation="\" value="\n"/>
            <sx:commentStarter value="#"/>
            <sx:flatFileBody>
              <sx:flatRecordType name="property">
                <sx:delimitedField name="name">
                  <sx:fieldDelimiter value="="/>
                </sx:delimitedField>
                <sx:delimitedField name="value"/>
              </sx:flatRecordType>
            </sx:flatFileBody>
          </sx:flatFile>
        </sx:flatFileReader>

        <sx:recordMapping>
          <messages>
            <sx:onRecord>
              <sx:fieldElementMap field="value" element="{name}"/>
            </sx:onRecord>
          </messages>
        </sx:recordMapping>
      </sx:recordContent>

    </sx:resources>

Note the following points about this script.

- The '=' delimiter is associated only with the `name` field, not the `value` field.

- There is no delimiter specified for the `value` field, nor for the record as a whole. This means that the `value` field will contain all the text between the first '=' symbol and the record delimiter, including any more '=' symbols.

You can run this example on the command line by entering

    servingxml -r resources-messages.xml -o output/messages.xml
        messages

#### Transforming a Java properties file with extra name field delimiters to produce nested tags (nested tags 1)

This example shows one way to write a resources script that will convert a Java properties file to XML with nested tags.

The input file is a Java properties file.

Figure 25. Input property file multipart-keys.properties

    image.text.info1=hello world 1
    image.text.info2=hello \
    world 2
    image.text.info3=hello world 3

This file has a number of features.

- The entries are key-value pairs separated by an '=' symbol.

- Each name consists of three parts separated by a '.' symbol

- The second line ends with a continuation character.

The desired output is shown below.

Figure 26. Output XML file nested-tags1.xml

    <?xml version="1.0" encoding="utf-8"?>
    <messages>
      <image>
        <text>
          <info1>hello world 1</info1>
          <info2>hello world 2</info2>
          <info3>hello world 3</info3>
        </text>
      </image>
    </messages>

Note that each level of nesting corresponds to separate part of the key.

The following resources script does the transformation.

Figure 27. Resources script resources-props2nested1.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="messages">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="messages"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="messages">

        <sx:flatFileReader>
          <sx:flatFile>
            <sx:commentStarter value="#"/>
            <sx:commentStarter value="//"/>
            <sx:recordDelimiter continuation="\" value="\r\n"/>
            <sx:recordDelimiter continuation="\" value="\n"/>
            <sx:flatFileBody>
              <sx:flatRecordType name="property">
                <sx:delimitedField name="key1">
                  <sx:fieldDelimiter value="."/>
                </sx:delimitedField>
                <sx:delimitedField name="key2">
                  <sx:fieldDelimiter value="."/>
                </sx:delimitedField>
                <sx:delimitedField name="key3">
                  <sx:fieldDelimiter value="="/>
                </sx:delimitedField>
                <sx:delimitedField name="value"/>
              </sx:flatRecordType>
            </sx:flatFileBody>
          </sx:flatFile>
        </sx:flatFileReader>

        <sx:recordMapping>
          <messages>
            <sx:groupBy fields="key1">
              <sx:elementMap element="{key1}">
                <sx:groupBy fields="key2 key2">
                  <sx:elementMap element="{key2}">
                    <sx:onRecord>
                      <sx:fieldElementMap field="value" element="{key3}"/>
                    </sx:onRecord>
                  </sx:elementMap>
                </sx:groupBy>
              </sx:elementMap>
            </sx:groupBy>
          </messages>
        </sx:recordMapping>

      </sx:recordContent>

    </sx:resources>

This script relies on reading each record as four fields.

- `key1` is terminated with a '.' delimiter
- `key2` is terminated with a '.' delimiter
- `key3` is terminated with an '=' delimiter
- `value` is terminated with a record delimiter

You can run this example on the command line by entering

    servingxml -i data/multipart-keys.properties
        -o output/nested-tags1.xml -r resources-props2nested1.xml messages

#### Transforming a Java properties file with name subfield delimiters to produce nested tags (nested tags 2)

This example shows another way to write a resources script that will convert a Java properties file to XML with nested tags.

The properties input file and the desired XML output file are the same as the previous example.

This time the resources script is written as follows.

Figure 28. Resources script resources-props2nested2.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="messages">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="messages"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="messages">

        <sx:flatFileReader>
          <sx:flatFile>
            <sx:commentStarter value="#"/>
            <sx:commentStarter value="//"/>
            <sx:recordDelimiter continuation="\" value="\r\n"/>
            <sx:recordDelimiter continuation="\" value="\n"/>
            <sx:flatFileBody>
              <sx:flatRecordType name="property">
                <sx:delimitedField name="name">
                  <sx:fieldDelimiter value="="/>
                  <sx:subfieldDelimiter value="."/>
                </sx:delimitedField>
                <sx:delimitedField name="value"/>
              </sx:flatRecordType>
            </sx:flatFileBody>
          </sx:flatFile>
        </sx:flatFileReader>

        <sx:recordMapping>
          <messages>
            <sx:groupBy fields="name[1]">
              <sx:elementMap element="{name[1]}">
                <sx:groupBy fields="name[1] name[2]">
                  <sx:elementMap element="{name[2]}">
                    <sx:onRecord>
                      <sx:fieldElementMap field="value" element="{name[3]}"/>
                    </sx:onRecord>
                  </sx:elementMap>
                </sx:groupBy>
              </sx:elementMap>
            </sx:groupBy>
          </messages>
        </sx:recordMapping>

      </sx:recordContent>

    </sx:resources>

This script relies on reading each record as two fields, a multi-valued name field and a value field.

- `name` has three values that may be indexed as 1...3
- `value` has one value

You can run this example on the command line by entering

    servingxml -i data/multipart-keys.properties
        -o output/nested-tags2.xml -r resources-props2nested2.xml messages

#### Reordering a group of records before mapping to XML (multiple summaries)

The example below shows how to reorder a group of records before mapping them to XML.

The input file is shown below.

    HEADER
    S 001
    S 002
    B 001
    S 003
    B 002
    TRAILER

Each 'B' record represents a summary of the 'S' records since the last summary..

Suppose you want to turn it into the following:.

    <?xml version="1.0" encoding="utf-8"?>
    <DOCUMENT>
      <HEADER/>
      <B value="001">
        <S value="001"/>
        <S value="002"/>
      </B>
      <B value="003">
        <S value="003"/>
      </B>
      <TRAILER/>
    </DOCUMENT>

If the summary records came before the details, rather than after, the mapping would be straightforward: see the examples with sx:innerGroup grouping elements. This suggests reordering the records as they come in.

The following resources script does the transformation.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="multipleSummaries">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="multipleSummariesContent"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="multipleSummariesContent">
        <sx:flatFileReader>
          <sx:urlSource url="data/multipleSummaries.txt"/>
          <sx:flatFile ref="multipleSummariesFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="multipleSummariesMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="multipleSummariesMapping">
        <DOCUMENT>
          <sx:groupChoice>

            <sx:innerGroup startTest="sx:current/Header"
                           endTest="sx:previous/Header">
              <sx:onRecord>
                <HEADER/>
              </sx:onRecord>
            </sx:innerGroup>

            <sx:innerGroup startTest="sx:current/Trailer"
                           endTest="sx:previous/Trailer">
              <sx:onRecord>
                <TRAILER/>
              </sx:onRecord>
            </sx:innerGroup>

            <sx:innerGroup startTest="sx:current/S"
                           endTest="sx:previous/B">
              <!-- reorder the records in this group - move the B records to the front -->
              <sx:reorderRecords recordTypes="B S"/>
                <B>
                  <sx:fieldAttributeMap field="value" attribute="value"/>
                  <sx:onRecord recordType='S'>
                    <S>
                      <sx:fieldAttributeMap field="value" attribute="value"/>
                    </S>
                  </sx:onRecord>
                </B>
            </sx:innerGroup>

          </sx:groupChoice>
        </DOCUMENT>
      </sx:recordMapping>

      <sx:flatFile id="multipleSummariesFile">
        <sx:flatFileBody>
          <sx:fieldDelimiter>
            <sx:whitespaceSeparator/>
          </sx:fieldDelimiter>
          <sx:flatRecordTypeChoice>
            <sx:delimitedField name="tag"/>
            <sx:when test="tag='HEADER'">
              <sx:flatRecordType name="Header">
                <sx:delimitedField name="tag"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='TRAILER'">
              <sx:flatRecordType name="Trailer">
                <sx:delimitedField name="tag"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='S'">
              <sx:flatRecordType name="S">
                <sx:delimitedField name="tag"/>
                <sx:delimitedField name="value"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='B'">
              <sx:flatRecordType name="B">
                <sx:delimitedField name="tag"/>
                <sx:delimitedField name="value"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-multipleSummaries.xml
      -o output/multipleSummaries.xml multipleSummaries

### Delimited Fields, Multiple Record Types, Repeating Groups

#### Mapping multiple repeating groups to XML (students-delim)

Suppose you have the following delimited file of students, their course grades, and their addresses.

Figure 29. Input flat file students-delim.txt

    JANE^ENGL^C-~MATH^A+|1972^BLUE^CHICAGO^IL~ATLANTA^GA

The file has the following layout.

name | the field delimiter ^ defines the end of the field
subject-grade | repeating group, repeating segments are separated by the repeat delimiter ~, the segment delimiter | defines the end of the group
year-born | the field delimiter ^ defines the end of the field
favorite-color | the field delimiter ^ defines the end of the field
address | repeating group, repeating segments are separated by the repeat delimiter ~, the segment delimiter | defines the end of the group

You want the XML output to look as follows.

Figure 30. Output XML file students-fix.xml

    <?xml version="1.0" encoding="utf-8"?>
    <StudentGrades>
      <StudentGrade>
        <Name>JANE</Name>
        <SubjectGrade>
          <Subject>ENGL</Subject>
          <Grade>C-</Grade>
        </SubjectGrade>
        <SubjectGrade>
          <Subject>MATH</Subject>
          <Grade>A+</Grade>
        </SubjectGrade>
        <YearBorn>1972</YearBorn>
        <FavoriteColor>BLUE</FavoriteColor>
        <Address>
          <City>CHICAGO</City>
          <State>IL</State>
        </Address>
        <Address>
          <City>ATLANTA</City>
          <State>GA</State>
        </Address>
      </StudentGrade>
    </StudentGrades>

The required resources script is

Figure 31. Resources script resources-students-delim.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="students">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="students-content"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="students-content">
        <sx:flatFileReader>
          <sx:flatFile ref="students-file"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="students-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="students-file">
        <sx:flatFileBody>
          <sx:flatRecordType name="student">
            <sx:fieldDelimiter value="^"/>
            <sx:repeatDelimiter value="~"/>
            <sx:segmentDelimiter value="|"/>
            <sx:delimitedField name="name"/>
            <sx:repeatingGroup name="grades">
              <sx:flatRecordType name="subject-grade">
                <sx:fieldDelimiter value="^"/>
                <sx:delimitedField name="subject"/>
                <sx:delimitedField name="grade"/>
              </sx:flatRecordType>
            </sx:repeatingGroup>
            <sx:delimitedField name="year-born"/>
            <sx:delimitedField name="favorite-color"/>
            <sx:repeatingGroup name="addresses">
              <sx:flatRecordType name="address">
                <sx:fieldDelimiter value="^"/>
                <sx:delimitedField name="city"/>
                <sx:delimitedField name="state"/>
              </sx:flatRecordType>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="students-mapping">
        <StudentGrades>
          <sx:onRecord>
            <StudentGrade>
              <sx:fieldElementMap field="name" element="Name"/>
              <sx:subrecordMapping repeatingGroup="grades">
                <sx:onRecord>
                  <SubjectGrade>
                    <sx:fieldElementMap field="subject" element="Subject"/>
                    <sx:fieldElementMap field="grade" element="Grade"/>
                  </SubjectGrade>
                </sx:onRecord>
              </sx:subrecordMapping>
              <sx:fieldElementMap field="year-born" element="YearBorn"/>
              <sx:fieldElementMap field="favorite-color" element="FavoriteColor"/>
              <sx:subrecordMapping repeatingGroup="addresses">
                <sx:onRecord>
                  <Address>
                    <sx:fieldElementMap field="city" element="City"/>
                    <sx:fieldElementMap field="state" element="State"/>
                  </Address>
                </sx:onRecord>
              </sx:subrecordMapping>
            </StudentGrade>
          </sx:onRecord>
        </StudentGrades>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-students-delim.xml -i data/students-delim.txt -o output/students-delim.xml
        students

#### Mapping a sequence of nested repeating groups to XML (invoice)

Suppose you have the following data file.

Figure 32. Input flat file invoice.txt

    12|2007-02-02|DocType|}
    SndId|SndName|SndAddress|SndZip|}
    RecId|RecName|RecAddress|RecZip|}
    ~
    1|Item 1|2|Kg|150|300|}
    2|Item 2|2|Kg|350|700|}
    3|Item 3|1|$|50|50|}
    4|Item 4|10|Unt|30|100|}
    ~
    1|Ref 1|Doc 1|Text|}
    2|Ref 2|Doc 2|Text|}
    ~
    1|Disc 1|Item 1|}
    2|Disc 2|Item 2|}
    3|Disc 3|Item 3|}
    ~
    Code a|Value 1|}
    Code z|Value 3|}
    Code x|Value 2|}
    Code w|Value 2|}
    Code y|Value 1|}
    ~
    !

The file has `n` sections separated by the "~" character. The first section has several non repeating records separated by a "}". But after that, there are 4 repeating sections separated by the same "~" as above. Each repeating section starts where the previous one ends.

The character "!" as the first character in the last line indicates the end of the data.

You want the XML output to look as follows.

Figure 33. Output XML file invoice.xml

    <?xml version="1.0" encoding="utf-8"?>
    <InvoiceDocument>
       <Header>
          <DocId>
             <Id>12</Id>
             <DocDate>2007-02-02</DocDate>
             <DocumentType>DocType</DocumentType>
          </DocId>
          <Sender>
             <SenderPassport>SndId</SenderPassport>
             <SenderFullName>SndName</SenderFullName>
             <SenderAddress>SndAddress</SenderAddress>
             <SenderZip>SndZip</SenderZip>
          </Sender>
          <Recipient>
             <RecipientPassport>RecId</RecipientPassport>
             <RecipientFullName>RecName</RecipientFullName>
             <RecipientAddress>RecAddress</RecipientAddress>
             <RecipientZip>RecZip</RecipientZip>
          </Recipient>
       </Header>
       <Detail>
          <Item>1</Item>
          <Description>Item 1</Description>
          <Quantity>2</Quantity>
          <Unit>Kg</Unit>
          <ItemPrice>150</ItemPrice>
          <TotalPrice>300</TotalPrice>
       </Detail>
       <Detail>
          <Item>2</Item>
          <Description>Item 2</Description>
          <Quantity>2</Quantity>
          <Unit>Kg</Unit>
          <ItemPrice>350</ItemPrice>
          <TotalPrice>700</TotalPrice>
       </Detail>
       <Detail>
          <Item>3</Item>
          <Description>Item 3</Description>
          <Quantity>1</Quantity>
          <Unit>$</Unit>
          <ItemPrice>50</ItemPrice>
          <TotalPrice>50</TotalPrice>
       </Detail>
       <Detail>
          <Item>4</Item>
          <Description>Item 4</Description>
          <Quantity>10</Quantity>
          <Unit>Unt</Unit>
          <ItemPrice>30</ItemPrice>
          <TotalPrice>100</TotalPrice>
       </Detail>
       <Reference>
          <RefId>1</RefId>
          <RefType>Ref 1</RefType>
          <RefDoc>Doc 1</RefDoc>
          <RefText>Text</RefText>
       </Reference>
       <Reference>
          <RefId>2</RefId>
          <RefType>Ref 2</RefType>
          <RefDoc>Doc 2</RefDoc>
          <RefText>Text</RefText>
       </Reference>
       <Discount>
          <DiscId>1</DiscId>
          <DiscDescription>Disc 1</DiscDescription>
          <DiscItem>Item 1</DiscItem>
       </Discount>
       <Discount>
          <DiscId>2</DiscId>
          <DiscDescription>Disc 2</DiscDescription>
          <DiscItem>Item 2</DiscItem>
       </Discount>
       <Discount>
          <DiscId>3</DiscId>
          <DiscDescription>Disc 3</DiscDescription>
          <DiscItem>Item 3</DiscItem>
       </Discount>
       <Code>
          <Code>Code a</Code>
          <Value>Value 1</Value>
       </Code>
       <Code>
          <Code>Code z</Code>
          <Value>Value 3</Value>
       </Code>
       <Code>
          <Code>Code x</Code>
          <Value>Value 2</Value>
       </Code>
       <Code>
          <Code>Code w</Code>
          <Value>Value 2</Value>
       </Code>
       <Code>
          <Code>Code y</Code>
          <Value>Value 1</Value>
       </Code>
    </InvoiceDocument>

The required resources script is

Figure 34. Resources script resources-invoice.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="invoice">
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:content ref="invoice-content"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="invoice-content">
        <sx:flatFileReader>
          <sx:flatFile ref="invoice-file"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="invoice-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="invoice-file">
        <sx:recordDelimiter value="\r\n!"/>
        <sx:recordDelimiter value="\n!"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="invoice" omitFinalRepeatDelimiter="false">
            <sx:repeatDelimiter value="~"/>
            <sx:repeatingGroup name="Header" count="1">
              <sx:flatRecordType name="Header">
                <sx:fieldDelimiter value="|"/>
                <sx:repeatDelimiter value="}"/>
                <!--sx:segmentDelimiter value="~"/-->
                <sx:repeatingGroup name="Document" count="1">
                  <sx:flatRecordType name="Doc">
                    <sx:delimitedField name="Id"/>
                    <sx:delimitedField name="DocDate"/>
                    <sx:delimitedField name="DocumentType"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
                <sx:repeatingGroup name="Sender" count="1">
                  <sx:flatRecordType name="Sender">
                    <sx:delimitedField name="SenderPassport"/>
                    <sx:delimitedField name="SenderFullName"/>
                    <sx:delimitedField name="SenderAddress"/>
                    <sx:delimitedField name="SenderZip"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
                <sx:repeatingGroup name="Recipient" count="1">
                  <sx:flatRecordType name="Recipient">
                    <sx:delimitedField name="RecipientPassport"/>
                    <sx:delimitedField name="RecipientFullName"/>
                    <sx:delimitedField name="RecipientAddress"/>
                    <sx:delimitedField name="RecipientZip"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:repeatingGroup>

            <sx:repeatingGroup name="Detail" count="1">
              <sx:flatRecordType name="Detail">
                <sx:fieldDelimiter value="|"/>
                <sx:repeatDelimiter value="}"/>
                <sx:repeatingGroup name="Item">
                  <sx:flatRecordType name="Item">
                    <sx:delimitedField name="Item"/>
                    <sx:delimitedField name="Description"/>
                    <sx:delimitedField name="Quantity"/>
                    <sx:delimitedField name="Unit"/>
                    <sx:delimitedField name="ItemPrice"/>
                    <sx:delimitedField name="TotalPrice"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:repeatingGroup>

            <sx:repeatingGroup name="Reference" count="1">
              <sx:flatRecordType name="Reference">
                <sx:fieldDelimiter value="|"/>
                <sx:repeatDelimiter value="}"/>
                <sx:segmentDelimiter value="~"/>
                <sx:repeatingGroup name="Ref">
                  <sx:flatRecordType name="Ref">
                    <sx:delimitedField name="RefId"/>
                    <sx:delimitedField name="RefType"/>
                    <sx:delimitedField name="RefDoc"/>
                    <sx:delimitedField name="RefText"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:repeatingGroup>

            <sx:repeatingGroup name="Discount" count="1">
              <sx:flatRecordType name="Discount">
                <sx:fieldDelimiter value="|"/>
                <sx:repeatDelimiter value="}"/>
                <sx:segmentDelimiter value="~"/>
                <sx:repeatingGroup name="Disc">
                  <sx:flatRecordType name="Disc">
                    <sx:delimitedField name="DiscId"/>
                    <sx:delimitedField name="DiscDescription"/>
                    <sx:delimitedField name="DiscItem"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:repeatingGroup>

            <sx:repeatingGroup name="Codes" count="1">
              <sx:flatRecordType name="Codes">
                <sx:fieldDelimiter value="|"/>
                <sx:repeatDelimiter value="}"/>
                <sx:segmentDelimiter value="~"/>
                <sx:repeatingGroup name="Code">
                  <sx:flatRecordType name="Code">
                    <sx:delimitedField name="Code"/>
                    <sx:delimitedField name="Value"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="invoice-mapping">
        <InvoiceDocument>
          <sx:onRecord>
            <sx:subrecordMapping repeatingGroup="Header">
              <sx:onRecord>
                <Header>
                  <sx:subrecordMapping repeatingGroup="Document">
                    <sx:onRecord>
                      <DocId>
                        <sx:fieldElementMap field="Id" element="Id"/>
                        <sx:fieldElementMap field="DocDate" element="DocDate"/>
                        <sx:fieldElementMap field="DocumentType" element="DocumentType"/>
                      </DocId>
                    </sx:onRecord>
                  </sx:subrecordMapping>
                  <sx:subrecordMapping repeatingGroup="Sender">
                    <sx:onRecord>
                      <Sender>
                        <sx:fieldElementMap field="SenderPassport" element="SenderPassport"/>
                        <sx:fieldElementMap field="SenderFullName" element="SenderFullName"/>
                        <sx:fieldElementMap field="SenderAddress" element="SenderAddress"/>
                        <sx:fieldElementMap field="SenderZip" element="SenderZip"/>
                      </Sender>
                    </sx:onRecord>
                  </sx:subrecordMapping>
                  <sx:subrecordMapping repeatingGroup="Recipient">
                    <sx:onRecord>
                      <Recipient>
                        <sx:fieldElementMap field="RecipientPassport" element="RecipientPassport"/>
                        <sx:fieldElementMap field="RecipientFullName" element="RecipientFullName"/>
                        <sx:fieldElementMap field="RecipientAddress" element="RecipientAddress"/>
                        <sx:fieldElementMap field="RecipientZip" element="RecipientZip"/>
                      </Recipient>
                    </sx:onRecord>
                  </sx:subrecordMapping>
                </Header>
              </sx:onRecord>
            </sx:subrecordMapping>

            <sx:subrecordMapping repeatingGroup="Detail">
              <sx:onRecord>
                <sx:subrecordMapping repeatingGroup="Item">
                  <sx:onRecord>
                    <Detail>
                      <sx:fieldElementMap field="Item" element="Item"/>
                      <sx:fieldElementMap field="Description" element="Description"/>
                      <sx:fieldElementMap field="Quantity" element="Quantity"/>
                      <sx:fieldElementMap field="Unit" element="Unit"/>
                      <sx:fieldElementMap field="ItemPrice" element="ItemPrice"/>
                      <sx:fieldElementMap field="TotalPrice" element="TotalPrice"/>
                    </Detail>
                  </sx:onRecord>
                </sx:subrecordMapping>
              </sx:onRecord>
            </sx:subrecordMapping>

            <sx:subrecordMapping repeatingGroup="Reference">
              <sx:onRecord>
                <sx:subrecordMapping repeatingGroup="Ref">
                  <sx:onRecord>
                    <Reference>
                      <sx:fieldElementMap field="RefId" element="RefId"/>
                      <sx:fieldElementMap field="RefType" element="RefType"/>
                      <sx:fieldElementMap field="RefDoc" element="RefDoc"/>
                      <sx:fieldElementMap field="RefText" element="RefText"/>
                    </Reference>
                  </sx:onRecord>
                </sx:subrecordMapping>
              </sx:onRecord>
            </sx:subrecordMapping>

            <sx:subrecordMapping repeatingGroup="Discount">
              <sx:onRecord>
                <sx:subrecordMapping repeatingGroup="Disc">
                  <sx:onRecord>
                    <Discount>
                      <sx:fieldElementMap field="DiscId" element="DiscId"/>
                      <sx:fieldElementMap field="DiscDescription" element="DiscDescription"/>
                      <sx:fieldElementMap field="DiscItem" element="DiscItem"/>
                    </Discount>
                  </sx:onRecord>
                </sx:subrecordMapping>
              </sx:onRecord>
            </sx:subrecordMapping>

            <sx:subrecordMapping repeatingGroup="Codes">
              <sx:onRecord>
                <sx:subrecordMapping repeatingGroup="Code">
                  <sx:onRecord>
                    <Code>
                      <sx:fieldElementMap field="Code" element="Code"/>
                      <sx:fieldElementMap field="Value" element="Value"/>
                    </Code>
                  </sx:onRecord>
                </sx:subrecordMapping>
              </sx:onRecord>
            </sx:subrecordMapping>
          </sx:onRecord>
        </InvoiceDocument>
      </sx:recordMapping>

    </sx:resources>

The idea is to map all the data up to the terminating "!" as one record, and to structure the record as a sequence of nested repeating groups.

You can run this example on the command line by entering

    servingxml  -r resources-invoice.xml -i data/invoice.txt -o output/invoice.xml
        invoice

### EDI Examples

#### EDI to XML (comptest)

The example below shows how to convert a composite field in an edi file to XML.

The input file is an edi file.

Figure 35. Input edi file comptest.txt

    BKG+1++4+Y:1:B:0:M:0:L:1'

This file has a number of features.

- The segment delimiter "'" terminates the segment.

- The field delimiter "+" terminates a field.

- Composite fields are delimited with ":".

The desired output is shown below.

Figure 36. Output XML file messages.xml

    <?xml version="1.0" encoding="utf-8"?>
    <edifact>
      <segment segmentType="BKG">
        <flightSegmentNumber>1</flightSegmentNumber>
        <accessID/>
        <bookingCodeCount>4</bookingCodeCount>
        <C704>
          <bookingCode>Y</bookingCode>
          <seatAvailabilityCount>1</seatAvailabilityCount>
        </C704>
        <C704>
          <bookingCode>B</bookingCode>
          <seatAvailabilityCount>0</seatAvailabilityCount>
        </C704>
        <C704>
          <bookingCode>M</bookingCode>
          <seatAvailabilityCount>0</seatAvailabilityCount>
        </C704>
        <C704>
          <bookingCode>L</bookingCode>
          <seatAvailabilityCount>1</seatAvailabilityCount>
        </C704>
      </segment>
    </edifact>

The following resources script does the transformation.

Figure 37. Resources script resources-comptest.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="edifact">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="edifact"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="edifact">
        <sx:flatFileReader>
          <sx:flatFile ref="edifactFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="edifactToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="edifactFlatFile">
        <sx:recordDelimiter value="'"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:fieldDelimiter value="+"/>
            <sx:delimitedField name="segmentType"/>
            <!-- edifact3 - Body Segments.-->
            <!-- BKG BOOKING CODE DATA .-->
            <sx:when test="segmentType='BKG'">
              <sx:flatRecordType name='BKG'>
                <sx:fieldDelimiter value="+"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="flightSegmentNumber"/>
                <sx:delimitedField name="accessID"/>
                <sx:delimitedField name="bookingCodeCount"/>
                <sx:repeatingGroup name="booking" count="{bookingCodeCount}">
                  <sx:flatRecordType name="C704">
                    <sx:fieldDelimiter value=":"/>
                    <sx:delimitedField name="bookingCode">
                    </sx:delimitedField>
                    <sx:delimitedField name="seatAvailabilityCount">
                    </sx:delimitedField>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="edifactToXmlMapping">
        <edifact>
          <!-- edifact3 - Body Segments.-->
          <!-- BKG BOOKING CODE DATA.-->
          <sx:onRecord recordType="BKG">
            <sx:elementMap element="segment">
              <sx:fieldAttributeMap field="segmentType" attribute="segmentType"/>
              <sx:fieldElementMap field="flightSegmentNumber" element="flightSegmentNumber"/>
              <sx:fieldElementMap field="accessID" element="accessID"/>
              <sx:fieldElementMap field="bookingCodeCount" element="bookingCodeCount"/>
              <sx:subrecordMapping repeatingGroup="booking">
                <sx:onRecord recordType="C704">
                  <C704>
                    <sx:fieldElementMap field="bookingCode" element="bookingCode"/>
                    <sx:fieldElementMap field="seatAvailabilityCount" element="seatAvailabilityCount"/>
                  </C704>
                </sx:onRecord>
              </sx:subrecordMapping>
            </sx:elementMap>
          </sx:onRecord>
        </edifact>
      </sx:recordMapping>

    </sx:resources>

Note the following points about this script.

- All the values of the composite field are put in the multiple valued field "C704".

- The sx:repeatingGroup element allows us to map these values, repeated two at a time, named as `bookingCode` and `seatAvailabilityCount` .

You can run this example on the command line by entering

    servingxml -i data/comptest.txt -r resources-comptest.xml
      -o output/comptest.xml edifact

#### EDI to XML with Escaped Delimiters (Invoic96A)

The example below shows how to convert a composite field in an edi file to XML.

The input file is an edi file.

Figure 38. Input edi file invoic96A.txt

    UNB+UNOA:3+0000008033253:14+01534730278:ZZ+080130:0000+04000020++++++1'
    UNH+1+INVOIC:D:96A:UN:EAN008'
    BGM+381+1000322+9'
    DTM+137:20080130:102'
    NAD+SU+++COMPANY XXXX ?+ YYYY+VIA ROMA, 1+CITTA?' DI?: GENOVA+GE+16163+IT'
    RFF+VA:05577790207'
    CUX+2:EUR:4+3'
    PAT+10E+ZZZ:::BONIFICO BANCARIO 90 GG D.F.+5:1:D:090'
    DTM+13:20080430:102'
    LIN+1+L02+8026495011408:EN'
    IMD+B++TU:::BAGUETTE PREC. GR.280 40PZ'
    QTY+47:5.000:PCE'
    QTY+59:40.000:PCE'
    MOA+203:54.130:EUR:4'
    PRI+AAA:10.830::::PCE'
    UNS+S'
    MOA+125:54.130'
    UNT+46+1'
    UNZ+1+04000020'

This file has a number of features.

- Records are terminated with `'`

- Fields are terminated with `+`

- Subfields are terminated with `:`

- The character ? is used as an escape character. For example, `COMPANY?: XXXX?+YYYY+` will be interpreted as: `COMPANY: XXXX+YYYY`

- Sometimes the input file could be formatted with a fixed 80 column width. Linefeed characters should be stripped.

The desired output is shown below.

Figure 39. Output XML file invoic96A.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Invoic96A>
       <UNB>
          <segmentType>UNB</segmentType>
       </UNB>
       <UNH>
          <segmentType>UNH</segmentType>
          <DE0062>1</DE0062>
          <S009>
             <S009>
                <DE0065>INVOIC</DE0065>
                <DE0052>D</DE0052>
                <DE0054>96A</DE0054>
                <DE0051>UN</DE0051>
                <DE0057>EAN008</DE0057>
             </S009>
          </S009>
          <DE0068/>
       </UNH>
       <BGM>
          <segmentType>BGM</segmentType>
          <C002>
             <C002>
                <DE1001>381</DE1001>
                <DE1131/>
                <DE3055/>
                <DE1000/>
             </C002>
          </C002>
          <DE1004>1000322</DE1004>
          <DE1225>9</DE1225>
          <DE4343/>
       </BGM>
       <DTM>
          <segmentType>DTM</segmentType>
          <C507>
             <C507>
                <DE2005>137</DE2005>
                <DE2380>20080130</DE2380>
                <DE2379>102</DE2379>
             </C507>
          </C507>
       </DTM>
       <NAD>
          <segmentType>NAD</segmentType>
          <DE3035>SU</DE3035>
          <C080>
             <C080>
                <DE3036-1>COMPANY XXXX + YYYY</DE3036-1>
                <DE3036-2/>
                <DE3036-3/>
                <DE3036-4/>
                <DE3036-5/>
                <DE3045/>
             </C080>
          </C080>
          <C059>
             <C059>
                <DE3042-1>VIA ROMA, 1</DE3042-1>
                <DE3042-2/>
                <DE3042-3/>
                <DE3042-4/>
             </C059>
          </C059>
          <DE3164>CITTA' DI: GENOVA</DE3164>
          <DE3229>GE</DE3229>
          <DE3251>16163</DE3251>
          <DE3207>IT</DE3207>
       </NAD>
       <RFF>
          <segmentType>RFF</segmentType>
          <C506>
             <C506>
                <DE1153>VA</DE1153>
                <DE1154>05577790207</DE1154>
                <DE1156/>
                <DE4000/>
             </C506>
          </C506>
       </RFF>
       <CUX>
          <segmentType>CUX</segmentType>
          <C504>
             <C504>
                <DE6347>2</DE6347>
                <DE6345>EUR</DE6345>
                <DE6343>4</DE6343>
                <DE6348/>
             </C504>
             <C504>
                <DE6347>3</DE6347>
                <DE6345/>
                <DE6343/>
                <DE6348/>
             </C504>
          </C504>
          <DE5402/>
          <DE6341/>
       </CUX>
       <PAT>
          <segmentType>PAT</segmentType>
          <DE4279>10E</DE4279>
          <C110>
             <C110>
                <DE4277>ZZZ</DE4277>
                <DE1131/>
                <DE3055/>
                <DE4276-1>BONIFICO BANCARIO 90 GG D.F.</DE4276-1>
                <DE4276-2/>
             </C110>
          </C110>
          <C112>
             <C112>
                <DE2475>5</DE2475>
                <DE2009>1</DE2009>
                <DE2151>D</DE2151>
                <DE2152>090</DE2152>
             </C112>
          </C112>
       </PAT>
       <DTM>
          <segmentType>DTM</segmentType>
          <C507>
             <C507>
                <DE2005>13</DE2005>
                <DE2380>20080430</DE2380>
                <DE2379>102</DE2379>
             </C507>
          </C507>
       </DTM>
       <LIN>
          <segmentType>LIN</segmentType>
          <DE1082>1</DE1082>
          <DE1229>L02</DE1229>
          <C212>
             <C212>
                <DE7140>8026495011408</DE7140>
                <DE7143>EN</DE7143>
                <DE1131/>
                <DE3055/>
             </C212>
          </C212>
          <DE1222/>
          <DE7083/>
       </LIN>
       <IMD>
          <segmentType>IMD</segmentType>
          <DE7077>B</DE7077>
          <DE7081/>
          <C273>
             <C273>
                <DE7009>TU</DE7009>
                <DE1131/>
                <DE3055/>
                <DE7008-1>BAGUETTE PREC. GR.280 40PZ</DE7008-1>
                <DE7008-2/>
                <DE3435/>
             </C273>
          </C273>
          <DE7383/>
       </IMD>
       <QTY>
          <segmentType>QTY</segmentType>
          <C186>
             <C186>
                <DE6063>47</DE6063>
                <DE6060>5.000</DE6060>
                <DE6411>PCE</DE6411>
             </C186>
          </C186>
       </QTY>
       <QTY>
          <segmentType>QTY</segmentType>
          <C186>
             <C186>
                <DE6063>59</DE6063>
                <DE6060>40.000</DE6060>
                <DE6411>PCE</DE6411>
             </C186>
          </C186>
       </QTY>
       <MOA>
          <segmentType>MOA</segmentType>
          <C516>
             <C516>
                <DE5025>203</DE5025>
                <DE5004>54.130</DE5004>
                <DE6345>EUR</DE6345>
                <DE6343>4</DE6343>
                <DE4405/>
             </C516>
          </C516>
       </MOA>
       <PRI>
          <segmentType>PRI</segmentType>
          <C509>
             <C509>
                <DE5125>AAA</DE5125>
                <DE5118>10.830</DE5118>
                <DE5375/>
                <DE5387/>
                <DE5284/>
                <DE6411>PCE</DE6411>
             </C509>
          </C509>
          <DE5213/>
       </PRI>
       <UNS>
          <segmentType>UNS</segmentType>
          <DE0081>S</DE0081>
       </UNS>
       <MOA>
          <segmentType>MOA</segmentType>
          <C516>
             <C516>
                <DE5025>125</DE5025>
                <DE5004>54.130</DE5004>
                <DE6345/>
                <DE6343/>
                <DE4405/>
             </C516>
          </C516>
       </MOA>
       <UNT>
          <segmentType>UNT</segmentType>
          <DE0074>46</DE0074>
          <DE0062>1</DE0062>
       </UNT>
       <UNZ>
          <segmentType>UNZ</segmentType>
          <F0036>1</F0036>
          <F0020>04000020</F0020>
       </UNZ>
    </Invoic96A>

The following resources script does the transformation.

Figure 40. Resources script resources-invoic96A.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="invoic96A-to-xml">
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:content ref="Invoic_96A_1"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="Invoic_96A_1" name="Invoic96A">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="edifactFlatFile"/>
          </sx:flatFileReader>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
          </sx:discardHandler>
        </sx:recordStream>
      </sx:recordContent>

      <sx:flatFile id="edifactFlatFile">
        <sx:recordDelimiter value="'" escapeCharacter="?"/>
        <!-- If a line is broken with a new line character, strip the new line character -->
        <sx:recordDelimiter value="\r\n" continuationSequence="\r\n"/>
        <sx:recordDelimiter value="\n" continuationSequence="\n"/>
        <!-- if a ' character is found after a ? (?')-->
        <!-- it is an escape sequence and should remain the ' character-->
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:fieldDelimiter value="+" escapeCharacter="?"/>
            <sx:delimitedField name="segmentType"/>

            <sx:when test="segmentType='UNA' or segmentType='UNA:'">
              <sx:flatRecordType name='UNA'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
              </sx:flatRecordType>
            </sx:when>

            <sx:when test="segmentType='UNB'">
              <sx:flatRecordType name='UNB'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
              </sx:flatRecordType>
            </sx:when>

            <sx:when test="segmentType='UNH'">
              <sx:flatRecordType name='UNH'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE0062"/>
                <sx:nonrepeatingGroup name="S009">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="S009">
                    <sx:delimitedField name="DE0065"/>
                    <sx:delimitedField name="DE0052"/>
                    <sx:delimitedField name="DE0054"/>
                    <sx:delimitedField name="DE0051"/>
                    <sx:delimitedField name="DE0057"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE0068"/>
                <sx:nonrepeatingGroup name="S010">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="S010">
                    <sx:delimitedField name="DE0070"/>
                    <sx:delimitedField name="DE0073"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='BGM'">
              <sx:flatRecordType name='BGM'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C002">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C002">
                    <sx:delimitedField name="DE1001"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE1000"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE1004"/>
                <sx:delimitedField name="DE1225"/>
                <sx:delimitedField name="DE4343"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='DTM'">
              <sx:flatRecordType name='DTM'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C507">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C507">
                    <sx:delimitedField name="DE2005"/>
                    <sx:delimitedField name="DE2380"/>
                    <sx:delimitedField name="DE2379"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='PAI'">
              <sx:flatRecordType name='PAI'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C534">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C534">
                    <sx:delimitedField name="DE4439"/>
                    <sx:delimitedField name="DE4431"/>
                    <sx:delimitedField name="DE4461"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE4435"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='FTX'">
              <sx:flatRecordType name='FTX'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE4451"/>
                <sx:delimitedField name="DE4453"/>
                <sx:nonrepeatingGroup name="C107">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C107">
                    <sx:delimitedField name="DE4441"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C108">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C108">
                    <sx:delimitedField name="DE4440-1"/>
                    <sx:delimitedField name="DE4440-2"/>
                    <sx:delimitedField name="DE4440-3"/>
                    <sx:delimitedField name="DE4440-4"/>
                    <sx:delimitedField name="DE4440-5"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE3453"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='NAD'">
              <sx:flatRecordType name='NAD'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE3035"/>
                <sx:nonrepeatingGroup name="C082">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C082">
                    <sx:delimitedField name="DE3039"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C058">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C058">
                    <sx:delimitedField name="DE3124-1"/>
                    <sx:delimitedField name="DE3124-2"/>
                    <sx:delimitedField name="DE3124-3"/>
                    <sx:delimitedField name="DE3124-4"/>
                    <sx:delimitedField name="DE3124-5"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C080">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C080">
                    <sx:delimitedField name="DE3036-1"/>
                    <sx:delimitedField name="DE3036-2"/>
                    <sx:delimitedField name="DE3036-3"/>
                    <sx:delimitedField name="DE3036-4"/>
                    <sx:delimitedField name="DE3036-5"/>
                    <sx:delimitedField name="DE3045"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C059">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C059">
                    <sx:delimitedField name="DE3042-1"/>
                    <sx:delimitedField name="DE3042-2"/>
                    <sx:delimitedField name="DE3042-3"/>
                    <sx:delimitedField name="DE3042-4"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE3164"/>
                <sx:delimitedField name="DE3229"/>
                <sx:delimitedField name="DE3251"/>
                <sx:delimitedField name="DE3207"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='FII'">
              <sx:flatRecordType name='FII'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE3035"/>
                <sx:nonrepeatingGroup name="C078">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C078">
                    <sx:delimitedField name="DE3194"/>
                    <sx:delimitedField name="DE3192-1"/>
                    <sx:delimitedField name="DE3192-2"/>
                    <sx:delimitedField name="DE6345"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C088">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C088">
                    <sx:delimitedField name="DE3433"/>
                    <sx:delimitedField name="DE1131-1"/>
                    <sx:delimitedField name="DE3055-1"/>
                    <sx:delimitedField name="DE3434"/>
                    <sx:delimitedField name="DE1131-2"/>
                    <sx:delimitedField name="DE3055-2"/>
                    <sx:delimitedField name="DE3432"/>
                    <sx:delimitedField name="DE3436"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE3207"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='RFF'">
              <sx:flatRecordType name='RFF'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C506">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C506">
                    <sx:delimitedField name="DE1153"/>
                    <sx:delimitedField name="DE1154"/>
                    <sx:delimitedField name="DE1156"/>
                    <sx:delimitedField name="DE4000"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='CTA'">
              <sx:flatRecordType name='CTA'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE3139"/>
                <sx:nonrepeatingGroup name="C056">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C056">
                    <sx:delimitedField name="DE3413"/>
                    <sx:delimitedField name="DE3412"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='COM'">
              <sx:flatRecordType name='COM'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C076">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C076">
                    <sx:delimitedField name="DE3148"/>
                    <sx:delimitedField name="DE3155"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='CUX'">
              <sx:flatRecordType name='CUX'>
                <sx:delimitedField name="segmentType"/>
                <sx:repeatingGroup name="C504" count="2">
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:repeatDelimiter value="+" escapeCharacter="?"/>
                  <!--<sx:segmentDelimiter value="+" escapeCharacter="?"/>-->
                  <sx:flatRecordType name="C504">
                    <sx:delimitedField name="DE6347"/>
                    <sx:delimitedField name="DE6345"/>
                    <sx:delimitedField name="DE6343"/>
                    <sx:delimitedField name="DE6348"/>
                  </sx:flatRecordType>
                </sx:repeatingGroup>
                <sx:delimitedField name="DE5402"/>
                <sx:delimitedField name="DE6341"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='PAT'">
              <sx:flatRecordType name='PAT'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE4279"/>
                <sx:nonrepeatingGroup name="C110">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C110">
                    <sx:delimitedField name="DE4277"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE4276-1"/>
                    <sx:delimitedField name="DE4276-2"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C112">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C112">
                    <sx:delimitedField name="DE2475"/>
                    <sx:delimitedField name="DE2009"/>
                    <sx:delimitedField name="DE2151"/>
                    <sx:delimitedField name="DE2152"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='PCD'">
              <sx:flatRecordType name='PCD'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C501">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C501">
                    <sx:delimitedField name="DE5245"/>
                    <sx:delimitedField name="DE5482"/>
                    <sx:delimitedField name="DE5249"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='MOA'">
              <sx:flatRecordType name='MOA'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C516">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C516">
                    <sx:delimitedField name="DE5025"/>
                    <sx:delimitedField name="DE5004"/>
                    <sx:delimitedField name="DE6345"/>
                    <sx:delimitedField name="DE6343"/>
                    <sx:delimitedField name="DE4405"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='ALC'">
              <sx:flatRecordType name='ALC'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE5463"/>
                <sx:nonrepeatingGroup name="C552">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C552">
                    <sx:delimitedField name="DE1230"/>
                    <sx:delimitedField name="DE5189"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE4471"/>
                <sx:delimitedField name="DE1227"/>
                <sx:nonrepeatingGroup name="C214">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C214">
                    <sx:delimitedField name="DE7161"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE7160-1"/>
                    <sx:delimitedField name="DE7160-2"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <!-- Detail segments -->
            <sx:when test="segmentType='LIN'">
              <sx:flatRecordType name='LIN'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE1082"/>
                <sx:delimitedField name="DE1229"/>
                <sx:nonrepeatingGroup name="C212">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C212">
                    <sx:delimitedField name="DE7140"/>
                    <sx:delimitedField name="DE7143"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C829">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C829">
                    <sx:delimitedField name="DE5495"/>
                    <sx:delimitedField name="DE1082"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE1222"/>
                <sx:delimitedField name="DE7083"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='PIA'">
              <sx:flatRecordType name='PIA'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE4347"/>
                <sx:nonrepeatingGroup name="C212">
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <!--<sx:segmentDelimiter value="+" escapeCharacter="?"/>-->
                  <sx:flatRecordType name="C212">
                    <sx:delimitedField name="DE7140"/>
                    <sx:delimitedField name="DE7143"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='IMD'">
              <sx:flatRecordType name='IMD'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE7077"/>
                <sx:delimitedField name="DE7081"/>
                <sx:nonrepeatingGroup name="C273">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C273">
                    <sx:delimitedField name="DE7009"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE7008-1"/>
                    <sx:delimitedField name="DE7008-2"/>
                    <sx:delimitedField name="DE3435"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE7383"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='QTY'">
              <sx:flatRecordType name='QTY'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C186">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C186">
                    <sx:delimitedField name="DE6063"/>
                    <sx:delimitedField name="DE6060"/>
                    <sx:delimitedField name="DE6411"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='PRI'">
              <sx:flatRecordType name='PRI'>
                <sx:delimitedField name="segmentType"/>
                <sx:nonrepeatingGroup name="C509">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C509">
                    <sx:delimitedField name="DE5125"/>
                    <sx:delimitedField name="DE5118"/>
                    <sx:delimitedField name="DE5375"/>
                    <sx:delimitedField name="DE5387"/>
                    <sx:delimitedField name="DE5284"/>
                    <sx:delimitedField name="DE6411"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE5213"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='TAX'">
              <sx:flatRecordType name='TAX'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE5283"/>
                <sx:nonrepeatingGroup name="C241">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C241">
                    <sx:delimitedField name="DE5153"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                    <sx:delimitedField name="DE5152"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:nonrepeatingGroup name="C533">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C533">
                    <sx:delimitedField name="DE5289"/>
                    <sx:delimitedField name="DE1131"/>
                    <sx:delimitedField name="DE3055"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE5286"/>
                <sx:nonrepeatingGroup name="C243">
                  <sx:segmentDelimiter value="+" escapeCharacter="?"/>
                  <sx:fieldDelimiter value=":" escapeCharacter="?"/>
                  <sx:flatRecordType name="C243">
                    <sx:delimitedField name="DE5279"/>
                    <sx:delimitedField name="DE1131-1"/>
                    <sx:delimitedField name="DE3055-1"/>
                    <sx:delimitedField name="DE5278"/>
                    <sx:delimitedField name="DE5273"/>
                    <sx:delimitedField name="DE1131-2"/>
                    <sx:delimitedField name="DE3055-2"/>
                  </sx:flatRecordType>
                </sx:nonrepeatingGroup>
                <sx:delimitedField name="DE5305"/>
                <sx:delimitedField name="DE3446"/>
              </sx:flatRecordType>
            </sx:when>

            <!-- Trailer segments -->
            <sx:when test="segmentType='UNS'">
              <sx:flatRecordType name='UNS'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE0081"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='UNT'">
              <sx:flatRecordType name='UNT'>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="DE0074"/>
                <sx:delimitedField name="DE0062"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="segmentType='UNZ'">
              <sx:flatRecordType name='UNZ'>
                <sx:fieldDelimiter value="+" escapeCharacter="?"/>
                <sx:delimitedField name="segmentType"/>
                <sx:delimitedField name="F0036"/>
                <sx:delimitedField name="F0020"/>
              </sx:flatRecordType>
            </sx:when>

            <sx:otherwise>
              <sx:flatRecordType name='ERROR'>
                <sx:delimitedField name="segmentType"/>
              </sx:flatRecordType>
            </sx:otherwise>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

Note the following points about this script.

- Record delimiters delimit the boundaries of a record, segment delimiters delimit areas within a record, and repeat delimiters separate subrecords from each other within segments. An sx:repeatingGroup reads the subrecords separated by repeat delimiters within a segment. A special case is when there are a number of segments, but at most one subrecord per segment, a nonrepeating group. An sx:nonrepeatingGroup reads the single subrecord, if any.

- The continuation sequence for a linefeed is defined be the linefeed itself, e.g.

          <sx:recordDelimiter value="\r\n" continuationSequence="\r\n"/>
          <sx:recordDelimiter value="\n" continuationSequence="\n"/>

  This has the effect of stripping out the linefeed characters.

You can run this example on the command line by entering

    servingxml -r resources-invoic96A.xml -i data/invoic96A.txt
        -o output/invoic96A.xml invoic96A-to-xml

### FpML Examples

#### FRA to FpML

Suppose "Bank A" has the following CSV file of FRA trades. Each FRA trade is represented by one record in the file.

Figure 41. Input flat file fra.csv

    product,trade_id,client_id,client_name,trade_date,buy_sell_code,currency,index,tenor,day_count_basis,fixed_rate,start_date,maturity_date,payment_date,notional
    FRA,1001,party1,Party 1,2006/12/07,BUY,CAD,BA,3M,ACT/365,0.04,   2008/09/08,2008/12/08,2008/09/08,250000000
    FRA,1002,party1,Party 1,2007/03/19,B,CAD,BA,3M,ACT/365,0.04,   2008/09/15,2008/12/15,2008/09/15,110000000
    FRA,1003,party1,Party 1,2007/03/19,SELL,CAD,BA,3M,ACT/365,0.04,  2008/12/15,2009/03/16,2008/12/15,70000000
    FRA,1004,party2,Party 2,2007/05/02,BUY,USD,LIBOR,3M,ACT/360,0.04,2009/02/02,2009/05/04,2009/02/02,100000000
    FRA,1005,party2,Party 2,2007/05/02,BUY,USD,LIBOR,3M,ACT/360,0.04,2009/02/02,2009/05/04,2009/02/02,100000000
    FRA,1006,party3,Party 3,2007/08/01,B,USD,LIBOR,3M,ACT/360,0.05,09/05/01,2009/08/04,2009/05/01,50000000

Note that two of the FRA trades have errors. The trade with trade_id 1002 has an invalid buy_sell_code, "B" instead of "BUY". The trade with trade id 1006 has the same problem with the buy_sell_code, and also has an invalid start_date, with a two digit year instead of a four digit year. Bank A wants to reject the trades with errors and write them along with an error message to a discard file.

Bank A wants the XML output to look as follows.

Figure 42. Output XML file fra.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <dataDocument xmlns="http://www.fpml.org/FpML-5-0/reporting"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" fpmlVersion="5-0"
                  xsi:schemaLocation="http://www.fpml.org/FpML-5-0/reporting ../fpml-main-5-0.xsd">
      <trade>
        <tradeHeader>
          <partyTradeIdentifier>
            <tradeId tradeIdScheme="http://www.banka.com/ird/trade-id">1001</tradeId>
          </partyTradeIdentifier>
          <tradeDate>2006-12-07</tradeDate>
        </tradeHeader>
        <fra>
          <sellerPartyReference>party1</sellerPartyReference>
          <adjustedEffectiveDate>2008-09-08</adjustedEffectiveDate>
          <adjustedTerminationDate>2008-12-08</adjustedTerminationDate>
          <paymentDate>
            <adjustedDate>2008-09-08</adjustedDate>
          </paymentDate>
          <dayCountFraction>ACT/365</dayCountFraction>
          <notional>
            <currency>CAD</currency>
            <amount>250000000</amount>
          </notional>
          <fixedRate>0.04</fixedRate>
          <floatingRateIndex>BA</floatingRateIndex>
          <indexTenor>
            <periodMultiplier>3</periodMultiplier>
            <period>M</period>
          </indexTenor>
        </fra>
      </trade>
      <trade>
        <tradeHeader>
          <partyTradeIdentifier>
            <tradeId tradeIdScheme="http://www.banka.com/ird/trade-id">1003</tradeId>
          </partyTradeIdentifier>
          <tradeDate>2007-03-19</tradeDate>
        </tradeHeader>
        <fra>
          <buyerPartyReference>party1</buyerPartyReference>
          <adjustedEffectiveDate>2008-12-15</adjustedEffectiveDate>
          <adjustedTerminationDate>2009-03-16</adjustedTerminationDate>
          <paymentDate>
            <adjustedDate>2008-12-15</adjustedDate>
          </paymentDate>
          <dayCountFraction>ACT/365</dayCountFraction>
          <notional>
            <currency>CAD</currency>
            <amount>70000000</amount>
          </notional>
          <fixedRate>0.04</fixedRate>
          <floatingRateIndex>BA</floatingRateIndex>
          <indexTenor>
            <periodMultiplier>3</periodMultiplier>
            <period>M</period>
          </indexTenor>
        </fra>
      </trade>
      <trade>
        <tradeHeader>
          <partyTradeIdentifier>
            <tradeId tradeIdScheme="http://www.banka.com/ird/trade-id">1004</tradeId>
          </partyTradeIdentifier>
          <tradeDate>2007-05-02</tradeDate>
        </tradeHeader>
        <fra>
          <sellerPartyReference>party2</sellerPartyReference>
          <adjustedEffectiveDate>2009-02-02</adjustedEffectiveDate>
          <adjustedTerminationDate>2009-05-04</adjustedTerminationDate>
          <paymentDate>
            <adjustedDate>2009-02-02</adjustedDate>
          </paymentDate>
          <dayCountFraction>ACT/360</dayCountFraction>
          <notional>
            <currency>USD</currency>
            <amount>100000000</amount>
          </notional>
          <fixedRate>0.04</fixedRate>
          <floatingRateIndex>LIBOR</floatingRateIndex>
          <indexTenor>
            <periodMultiplier>3</periodMultiplier>
            <period>M</period>
          </indexTenor>
        </fra>
      </trade>
      <trade>
        <tradeHeader>
          <partyTradeIdentifier>
            <tradeId tradeIdScheme="http://www.banka.com/ird/trade-id">1005</tradeId>
          </partyTradeIdentifier>
          <tradeDate>2007-05-02</tradeDate>
        </tradeHeader>
        <fra>
          <sellerPartyReference>party2</sellerPartyReference>
          <adjustedEffectiveDate>2009-02-02</adjustedEffectiveDate>
          <adjustedTerminationDate>2009-05-04</adjustedTerminationDate>
          <paymentDate>
            <adjustedDate>2009-02-02</adjustedDate>
          </paymentDate>
          <dayCountFraction>ACT/360</dayCountFraction>
          <notional>
            <currency>USD</currency>
            <amount>100000000</amount>
          </notional>
          <fixedRate>0.04</fixedRate>
          <floatingRateIndex>LIBOR</floatingRateIndex>
          <indexTenor>
            <periodMultiplier>3</periodMultiplier>
            <period>M</period>
          </indexTenor>
        </fra>
      </trade>
      <party id="bankA">
        <partyId>Bank A</partyId>
      </party>
      <party id="party1">
        <partyId>Party 1</partyId>
      </party>
      <party id="party3">
        <partyId>Party 3</partyId>
      </party>
      <party id="party2">
        <partyId>Party 2</partyId>
      </party>
    </dataDocument>

Note that the document concludes with a unique list of parties that are referenced by `id` in the `buyerPartyReference` and `sellerPartyReference` elements.

The required resources script is

Figure 43. Resources script resources-fra.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="fra-to-xml">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="fra-xml"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="fra-xml">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="fra-flat-file"/>
          </sx:flatFileReader>
          <sx:recordValidator message="Error in trade {trade_id}.">
            <sx:fieldValidator field="buy_sell_code"
                              message="Invalid buy_sell_code {buy_sell_code}.">
              <sx:or>
                <sx:valueRestriction pattern="BUY"/>
                <sx:valueRestriction pattern="SELL"/>
              </sx:or>
            </sx:fieldValidator>
            <sx:fieldValidator field="start_date"
                               message="Invalid start_date {start_date}.">
              <sx:valueRestriction pattern="(19|20)\d\d[/](0[1-9]|1[012])[/](0[1-9]|[12][0-9]|3[01])"/>
            </sx:fieldValidator>
            <sx:fieldValidator field="maturity_date"
                               message="Invalid maturity_date {maturity_date}.">
              <sx:valueRestriction pattern="(20|21)\d\d[/](0[1-9]|1[012])[/](0[1-9]|[12][0-9]|3[01])"/>
            </sx:fieldValidator>
          </sx:recordValidator>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <sx:modifyRecord>
              <sx:newField name="message" value="{$sx:message}"/>
            </sx:modifyRecord>
            <sx:flatFileWriter>
              <sx:fileSink file="output/fra-error.csv"/>
            </sx:flatFileWriter>
          </sx:discardHandler>
        </sx:recordStream>
        <sx:recordMapping ref="fra-to-xml-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="fra-flat-file">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="record_type"/>
            <sx:when test="record_type='FRA'">
              <sx:flatRecordType name='fra'>
                <sx:fieldDelimiter value=","/>
                <sx:delimitedField name="record_type"/>
                <sx:delimitedField name="trade_id"/>
                <sx:delimitedField name="client_id"/>
                <sx:delimitedField name="client_name"/>
                <sx:delimitedField name="trade_date"/>
                <sx:delimitedField name="buy_sell_code"/>
                <sx:delimitedField name="currency"/>
                <sx:delimitedField name="index"/>
                <sx:delimitedField name="tenor"/>
                <sx:delimitedField name="day_count_basis"/>
                <sx:delimitedField name="fixed_rate"/>
                <sx:delimitedField name="start_date"/>
                <sx:delimitedField name="maturity_date"/>
                <sx:delimitedField name="payment_date"/>
                <sx:delimitedField name="notional"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="fra-to-xml-mapping">
        <dataDocument xmlns="http://www.fpml.org/FpML-5-0/reporting"
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      fpmlVersion="5-0"
                      xsi:schemaLocation="http://www.fpml.org/FpML-5-0/reporting ../fpml-main-5-0.xsd">
          <sx:groupBy fields="trade_id">
            <trade>
              <tradeHeader>
                <partyTradeIdentifier>
                  <sx:fieldElementMap field="trade_id"
                                      element="tradeId">
                    <sx:fieldAttributeMap value="http://www.banka.com/ird/trade-id"
                                          attribute="tradeIdScheme"/>
                  </sx:fieldElementMap>
                </partyTradeIdentifier>
                <tradeDate>
                  <sx:toXmlDate fromFormat="yyyy/MM/dd">
                    <sx:toString value="{trade_date}"/>
                  </sx:toXmlDate>
                </tradeDate>
              </tradeHeader>
              <sx:onRecord recordType="fra">
                <fra>
                  <sx:choose>
                    <sx:when test="buy_sell_code='BUY'">
                      <sx:fieldElementMap field="client_id"
                                          element="sellerPartyReference"/>
                    </sx:when>
                    <sx:otherwise>
                      <sx:fieldElementMap field="client_id"
                                          element="buyerPartyReference"/>
                    </sx:otherwise>
                  </sx:choose>
                  <adjustedEffectiveDate>
                    <sx:toXmlDate fromFormat="yyyy/MM/dd">
                      <sx:toString value="{start_date}"/>
                    </sx:toXmlDate>
                  </adjustedEffectiveDate>
                  <adjustedTerminationDate>
                    <sx:toXmlDate fromFormat="yyyy/MM/dd">
                      <sx:toString value="{maturity_date}"/>
                    </sx:toXmlDate>
                  </adjustedTerminationDate>
                  <paymentDate>
                    <adjustedDate>
                      <sx:toXmlDate fromFormat="yyyy/MM/dd">
                        <sx:toString value="{payment_date}"/>
                      </sx:toXmlDate>
                    </adjustedDate>
                  </paymentDate>
                  <sx:fieldElementMap field="day_count_basis"
                                      element="dayCountFraction"/>
                  <notional>
                    <sx:fieldElementMap field="currency"
                                        element="currency"/>
                    <sx:fieldElementMap field="notional"
                                        element="amount"/>
                  </notional>
                  <sx:fieldElementMap field="fixed_rate"
                                      element="fixedRate"/>
                  <sx:fieldElementMap field="index"
                                      element="floatingRateIndex"/>
                  <indexTenor>
                    <sx:choose>
                      <sx:when test="fn:ends-with(tenor,'D')">
                        <sx:fieldElementMap element="periodMultiplier">
                          <sx:toString select="fn:substring-before(tenor,'D')"/>
                        </sx:fieldElementMap>
                        <period>D</period>
                      </sx:when>
                      <sx:when test="fn:ends-with(tenor,'M')">
                        <sx:fieldElementMap element="periodMultiplier">
                          <sx:toString select="fn:substring-before(tenor,'M')"/>
                        </sx:fieldElementMap>
                        <period>M</period>
                      </sx:when>
                      <sx:when test="fn:ends-with(tenor,'Y')">
                        <sx:fieldElementMap element="periodMultiplier">
                          <sx:toString select="fn:substring-before(tenor,'Y')"/>
                        </sx:fieldElementMap>
                        <period>Y</period>
                      </sx:when>
                    </sx:choose>
                  </indexTenor>
                </fra>
              </sx:onRecord>
            </trade>
          </sx:groupBy>

          <sx:nestedContent>
            <sx:recordContent ref="clients"/>
          </sx:nestedContent>
        </dataDocument>
      </sx:recordMapping>

      <sx:recordContent id="clients">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="fra-flat-file"/>
          </sx:flatFileReader>
          <sx:recordProjection fields="client_id client_name"/>
        </sx:recordStream>
        <sx:recordMapping>
          <party id="bankA">
            <partyId>Bank A</partyId>
          </party>
          <sx:onRecord>
            <party>
              <sx:fieldAttributeMap field="client_id" attribute="id"/>
              <sx:fieldElementMap field="client_name" element="partyId"/>
            </party>
          </sx:onRecord>
        </sx:recordMapping>
      </sx:recordContent>

    </sx:resources>

We use the sx:nestedContent element to insert a list of parties from the `swap.csv` file into the output XML, and we use the sx:recordProjection element to eliminate duplicate pairs of client_id and client_name.

You can run this example on the command line by entering

    servingxml -i data/fra.csv -o output/fra.xml -r resources-fra.xml fra-to-xml

The following error messages will be written to the console (or log file.)

    25-Sep-2008 10:10:47 PM com.servingxml.util.system.DefaultLogger error
    SEVERE: Error in record "fra" on line 3.  Error in trade 1002. Invalid buy_sell_code B.
    25-Sep-2008 10:10:47 PM com.servingxml.util.system.DefaultLogger error
    SEVERE: Error in record "fra" on line 7.  Error in trade 1006. Invalid buy_sell_code B. Invalid start_date 09/05/01.

#### Swap to FpML

Suppose "Bank A" has the following CSV file of SWAP trades. Each swap trade is represented by two records in the file, one for the pay (PAY) side, and one for the receive (REC) side.

Figure 44. Input flat file swap.csv

    product,trade_id,client_id,client_name,trade_date,pay_receive_code,currency,index,tenor,day_count_basis,fixed_rate,start_date,maturity_date,reset_date,payment_date,notional,calculation_period
    SWAP,2001,party1,Party 1,2008/05/14,PAY,FIXED,CAD,BA,6M,30/360,0.04,2008/07/17,2009/01/17,2008/07/17,2009/01/17,10000000.00,6M
    SWAP,2001,party1,Party 1,2008/05/14,REC,FLOAT,CAD,BA,6M,30/360,0.04,2008/07/17,2009/01/17,2008/07/17,2009/01/17,10000000.00,6M
    SWAP,2002,party2,Party 2,2008/05/14,PAY,FIXED,CAD,BA,6M,30/360,0.04,2008/07/17,2009/01/17,2008/07/17,2009/01/17,10000000.00,6M
    SWAP,2002,party2,Party 2,2008/05/14,R,FLOAT,CAD,BA,6M,30/360,0.04,2008/07/17,2009/01/17,2008/07/17,2009/01/17,10000000.00,6M

Note that one of the records, the receive side for the trade with trade_id 2002, has an invalid value for pay_receive_code, the value has been mistakenly entered as "R" instead of "REC". Bank A wants to validate this field, and if it is wrong for either side of the swap, Bank A wants to reject both sides, and write them both with an error message to a discard file.

Bank A wants the XML output to look as follows.

Figure 45. Output XML file swap.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <dataDocument xmlns="http://www.fpml.org/FpML-5-0/reporting"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  fpmlVersion="5-0"
                  xsi:schemaLocation="http://www.fpml.org/FpML-5-0/reporting ../fpml-main-5-0.xsd">
      <trade>
        <tradeHeader>
          <partyTradeIdentifier>
            <tradeId tradeIdScheme="http://www.banka.com/ird/trade-id">2001</tradeId>
          </partyTradeIdentifier>
          <tradeDate>2008-05-14</tradeDate>
        </tradeHeader>
        <swap>
          <swapStream>
            <payerPartyReference href="bankA"/>
            <calculationPeriodDates id="floatingCalcPeriodDates">
              <effectiveDate>
                <adjustedDate>2008-07-17</adjustedDate>
              </effectiveDate>
              <terminationDate>
                <adjustedDate>2009-01-17</adjustedDate>
              </terminationDate>
              <calculationPeriodFrequency>
                <periodMultiplier>6</periodMultiplier>
                <period>M</period>
              </calculationPeriodFrequency>
            </calculationPeriodDates>
            <calculationPeriodAmount>
              <calculation>
                <notionalSchedule>
                  <notionalStepSchedule>
                    <initialValue>10000000.00</initialValue>
                    <currency currencyScheme="http://www.fpml.org/ext/iso4217">CAD</currency>
                  </notionalStepSchedule>
                </notionalSchedule>
                <fixedRateSchedule>
                  <initialValue>0.04</initialValue>
                </fixedRateSchedule>
                <dayCountFraction>30/360</dayCountFraction>
              </calculation>
            </calculationPeriodAmount>
          </swapStream>
          <swapStream>
            <payerPartyReference href="party1"/>
            <calculationPeriodDates id="floatingCalcPeriodDates">
              <effectiveDate>
                <adjustedDate>2008-07-17</adjustedDate>
              </effectiveDate>
              <terminationDate>
                <adjustedDate>2009-01-17</adjustedDate>
              </terminationDate>
              <calculationPeriodFrequency>
                <periodMultiplier>6</periodMultiplier>
                <period>M</period>
              </calculationPeriodFrequency>
            </calculationPeriodDates>
            <calculationPeriodAmount>
              <calculation>
                <notionalSchedule>
                  <notionalStepSchedule>
                    <initialValue>10000000.00</initialValue>
                    <currency currencyScheme="http://www.fpml.org/ext/iso4217">CAD</currency>
                  </notionalStepSchedule>
                </notionalSchedule>
                <floatingRateCalculation>
                  <floatingRateIndex>BA</floatingRateIndex>
                  <indexTenor>
                    <periodMultiplier>6</periodMultiplier>
                    <period>M</period>
                  </indexTenor>
                </floatingRateCalculation>
                <dayCountFraction>30/360</dayCountFraction>
              </calculation>
            </calculationPeriodAmount>
          </swapStream>
        </swap>
      </trade>
      <party id="bankA">
        <partyId>Bank A</partyId>
      </party>
      <party id="party1">
        <partyId>Party 1</partyId>
      </party>
      <party id="party2">
        <partyId>Party 2</partyId>
      </party>
    </dataDocument>

Note that the document concludes with a unique list of parties that are referenced by `id` in the `payerPartyReference` elements.

The required resources script is

Figure 46. Resources script resources-swap.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="swap-to-xml">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="swap-xml"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="swap-xml">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="swap-flat-file"/>
          </sx:flatFileReader>
          <sx:modifyRecord>
            <sx:newField name="trade_id" select="legs/child::node()[1]/trade_id"/>
            <sx:newField name="trade_date" select="legs/child::node()[1]/trade_date"/>
          </sx:modifyRecord>
          <sx:recordValidator ref="swap-validator"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <sx:decomposeRecord recordType="swap" repeatingGroup="legs"/>
            <sx:flatFileWriter>
              <sx:fileSink file="output/swap-error.csv"/>
            </sx:flatFileWriter>
          </sx:discardHandler>
        </sx:recordStream>
        <sx:recordMapping ref="swap-to-xml-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="swap-flat-file">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:compositeFlatRecordType>
            <sx:delimitedField name="record_type"/>
            <sx:delimitedField name="trade_id"/>
            <sx:recordComposition recordType="swap"
                                  repeatingGroup="legs"
                                  endTest="sx:current//trade_id != sx:previous//trade_id">
              <sx:flatRecordTypeChoice>
                <sx:fieldDelimiter value=","/>
                <sx:delimitedField name="record_type"/>
                <sx:delimitedField name="trade_id"/>
                <sx:delimitedField name="client_id"/>
                <sx:delimitedField name="client_name"/>
                <sx:delimitedField name="trade_date"/>
                <sx:delimitedField name="pay_receive_code"/>
                <sx:delimitedField name="fixed_float_code"/>
                <sx:when test="record_type='SWAP' and fixed_float_code='FLOAT'">
                  <sx:flatRecordType name='swap_floating_leg'>
                    <sx:fieldDelimiter value=","/>
                    <sx:delimitedField name="record_type"/>
                    <sx:delimitedField name="trade_id"/>
                    <sx:delimitedField name="client_id"/>
                    <sx:delimitedField name="client_name"/>
                    <sx:delimitedField name="trade_date"/>
                    <sx:delimitedField name="pay_receive_code"/>
                    <sx:delimitedField name="fixed_float_code"/>
                    <sx:delimitedField name="currency"/>
                    <sx:delimitedField name="index"/>
                    <sx:delimitedField name="tenor"/>
                    <sx:delimitedField name="day_count_basis"/>
                    <sx:delimitedField name="fixed_rate"/>
                    <sx:delimitedField name="start_date"/>
                    <sx:delimitedField name="maturity_date"/>
                    <sx:delimitedField name="reset_date"/>
                    <sx:delimitedField name="payment_date"/>
                    <sx:delimitedField name="notional"/>
                    <sx:delimitedField name="calculation_period"/>
                  </sx:flatRecordType>
                </sx:when>
                <sx:when test="record_type='SWAP' and fixed_float_code='FIXED'">
                  <sx:flatRecordType name='swap_fixed_leg'>
                    <sx:fieldDelimiter value=","/>
                    <sx:delimitedField name="record_type"/>
                    <sx:delimitedField name="trade_id"/>
                    <sx:delimitedField name="client_id"/>
                    <sx:delimitedField name="client_name"/>
                    <sx:delimitedField name="trade_date"/>
                    <sx:delimitedField name="pay_receive_code"/>
                    <sx:delimitedField name="fixed_float_code"/>
                    <sx:delimitedField name="currency"/>
                    <sx:delimitedField name="index"/>
                    <sx:delimitedField name="tenor"/>
                    <sx:delimitedField name="day_count_basis"/>
                    <sx:delimitedField name="fixed_rate"/>
                    <sx:delimitedField name="start_date"/>
                    <sx:delimitedField name="maturity_date"/>
                    <sx:delimitedField name="reset_date"/>
                    <sx:delimitedField name="payment_date"/>
                    <sx:delimitedField name="notional"/>
                    <sx:delimitedField name="calculation_period"/>
                  </sx:flatRecordType>
                </sx:when>
              </sx:flatRecordTypeChoice>
            </sx:recordComposition>
          </sx:compositeFlatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>
      <sx:recordValidator id="swap-validator"
                          recordType="swap"
                          message="Error in trade {trade_id}.">
        <sx:fieldValidator field="legs"
                           message="">
          <sx:recordValidator>
            <sx:fieldValidator field="pay_receive_code"
                               message="Invalid pay_receive_code {pay_receive_code}.">
              <sx:or>
                <sx:valueRestriction pattern="PAY"/>
                <sx:valueRestriction pattern="REC"/>
              </sx:or>
            </sx:fieldValidator>
            <sx:fieldValidator field="fixed_float_code"
                               message="Invalid fixed_float_code {fixed_float_code}.">
              <sx:or>
                <sx:valueRestriction pattern="FIXED"/>
                <sx:valueRestriction pattern="FLOAT"/>
              </sx:or>
            </sx:fieldValidator>
            <sx:fieldValidator field="start_date"
                               message="Invalid start_date {start_date}.">
              <sx:valueRestriction pattern="(19|20)\d\d[/](0[1-9]|1[012])[/](0[1-9]|[12][0-9]|3[01])"/>
            </sx:fieldValidator>
            <sx:fieldValidator field="maturity_date"
                               message="Invalid maturity_date {maturity_date}.">
              <sx:valueRestriction pattern="(20|21)\d\d[/](0[1-9]|1[012])[/](0[1-9]|[12][0-9]|3[01])"/>
            </sx:fieldValidator>
          </sx:recordValidator>
        </sx:fieldValidator>
      </sx:recordValidator>

      <sx:recordMapping id="swap-to-xml-mapping">
        <dataDocument xmlns="http://www.fpml.org/FpML-5-0/reporting"
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      fpmlVersion="5-0"
                      xsi:schemaLocation="http://www.fpml.org/FpML-5-0/reporting ../fpml-main-5-0.xsd">
          <sx:groupBy fields="trade_id">
            <trade>
              <tradeHeader>
                <partyTradeIdentifier>
                  <sx:fieldElementMap field="trade_id" element="tradeId">
                    <sx:fieldAttributeMap value="http://www.banka.com/ird/trade-id"
                                          attribute="tradeIdScheme"/>
                  </sx:fieldElementMap>
                </partyTradeIdentifier>
                <tradeDate>
                  <sx:toXmlDate fromFormat="yyyy/MM/dd">
                    <sx:toString value="{trade_date}"/>
                  </sx:toXmlDate>
                </tradeDate>
              </tradeHeader>
              <swap>
                <sx:subrecordMapping repeatingGroup="legs">
                  <sx:groupBy fields="trade_id pay_receive_code">
                    <swapStream>
                      <sx:choose>
                        <sx:when test="pay_receive_code='PAY'">
                          <payerPartyReference>
                            <sx:fieldAttributeMap value="bankA"
                                                  attribute="href"/>
                          </payerPartyReference>
                        </sx:when>
                        <sx:when test="pay_receive_code='REC'">
                          <payerPartyReference>
                            <sx:fieldAttributeMap field="client_id"
                                                  attribute="href"/>
                          </payerPartyReference>
                        </sx:when>
                      </sx:choose>
                      <calculationPeriodDates id="floatingCalcPeriodDates">
                        <effectiveDate>
                          <adjustedDate>
                            <sx:toXmlDate fromFormat="yyyy/MM/dd">
                              <sx:toString value="{start_date}"/>
                            </sx:toXmlDate>
                          </adjustedDate>
                        </effectiveDate>
                        <terminationDate>
                          <adjustedDate>
                            <sx:toXmlDate fromFormat="yyyy/MM/dd">
                              <sx:toString value="{maturity_date}"/>
                            </sx:toXmlDate>
                          </adjustedDate>
                        </terminationDate>
                        <calculationPeriodFrequency>
                          <sx:parameter name="term" value="{calculation_period}"/>
                          <sx:choose ref="expand-term"/>
                        </calculationPeriodFrequency>
                      </calculationPeriodDates>
                      <calculationPeriodAmount>
                        <calculation>
                          <notionalSchedule>
                            <notionalStepSchedule>
                              <sx:fieldElementMap field="notional" element="initialValue"/>
                              <sx:fieldElementMap field="currency" element="currency">
                                <sx:fieldAttributeMap value="http://www.fpml.org/ext/iso4217"
                                                      attribute="currencyScheme"/>
                              </sx:fieldElementMap>
                            </notionalStepSchedule>
                          </notionalSchedule>
                          <sx:choose>
                            <sx:when test="fixed_float_code='FLOAT'">
                              <floatingRateCalculation>
                                <sx:fieldElementMap field="index"
                                                    element="floatingRateIndex"/>
                                <indexTenor>
                                  <sx:parameter name="term" value="{tenor}"/>
                                  <sx:choose ref="expand-term"/>
                                </indexTenor>
                              </floatingRateCalculation>
                            </sx:when>
                            <sx:when test="fixed_float_code='FIXED'">
                              <fixedRateSchedule>
                                <sx:fieldElementMap field="fixed_rate"
                                                    element="initialValue"/>
                              </fixedRateSchedule>
                            </sx:when>
                          </sx:choose>
                          <sx:fieldElementMap field="day_count_basis"
                                              element="dayCountFraction"/>
                        </calculation>
                      </calculationPeriodAmount>
                    </swapStream>
                  </sx:groupBy>
                </sx:subrecordMapping>
              </swap>
            </trade>
          </sx:groupBy>

          <sx:nestedContent>
            <sx:recordContent ref="clients"/>
          </sx:nestedContent>
        </dataDocument>
      </sx:recordMapping>

      <!-- Expands a parameter named "term" with a value <number><D|M|Y> as
        <periodMultiplier>number</periodMultiplier>
        <period>D|M|Y</period>
      -->
      <sx:choose id="expand-term">
        <sx:when test="fn:ends-with($term,'D')">
          <sx:fieldElementMap element="periodMultiplier">
            <sx:toString select="fn:substring-before($term,'D')"/>
          </sx:fieldElementMap>
          <period>D</period>
        </sx:when>
        <sx:when test="fn:ends-with($term,'M')">
          <sx:fieldElementMap element="periodMultiplier">
            <sx:toString select="fn:substring-before($term,'M')"/>
          </sx:fieldElementMap>
          <period>M</period>
        </sx:when>
        <sx:when test="fn:ends-with($term,'Y')">
          <sx:fieldElementMap element="periodMultiplier">
            <sx:toString select="fn:substring-before($term,'Y')"/>
          </sx:fieldElementMap>
          <period>Y</period>
        </sx:when>
      </sx:choose>

      <sx:recordContent id="clients">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile>
              <sx:flatFileHeader lineCount="1"/>
              <sx:flatFileBody>
                <sx:flatRecordType>
                  <sx:fieldDelimiter value=","/>
                  <sx:delimitedField name="record_type"/>
                  <sx:delimitedField name="trade_id"/>
                  <sx:delimitedField name="client_id"/>
                  <sx:delimitedField name="client_name"/>
                </sx:flatRecordType>
              </sx:flatFileBody>
            </sx:flatFile>
          </sx:flatFileReader>
          <sx:recordProjection fields="client_id client_name"/>
        </sx:recordStream>
        <sx:recordMapping>
          <party id="bankA">
            <partyId>Bank A</partyId>
          </party>
          <sx:onRecord>
            <party>
              <sx:fieldAttributeMap field="client_id" attribute="id"/>
              <sx:fieldElementMap field="client_name" element="partyId"/>
            </party>
          </sx:onRecord>
        </sx:recordMapping>
      </sx:recordContent>

    </sx:resources>

Note the following points about this script.

- We use the sx:composeRecord element to compose a single record from the two legs of the swap. This way, when we validate the composite record, a failure in either of the legs will result in the entire swap being discarded, not just one of the legs.

- We use the sx:nestedContent element to insert a list of parties from the `swap.csv` file into the output XML, and we use the sx:recordProjection element to eliminate duplicate pairs of client_id and client_name.

- Inside the sx:discardHandler element, we use the sx:decomposeRecord element to decompose the composite swap record back into separate leg records. This allows us to easily use the default record writer to write the legs to a discard file.

- We have two fields, `calculation_period` and `tenor` , that have values like `3M` and `30D` , where the numeric value and the trailing letter need to be mapped separately to the elements `periodMultiplier` and `period` . We use a parameterized sx:choose instruction to handle both cases.

You can run this example on the command line by entering

    servingxml -i data/swap.csv -o output/swap.xml -r resources-swap.xml swap-to-xml

The following error message will be written to the console (or log file.)

    25-Sep-2008 10:10:54 PM com.servingxml.util.system.DefaultLogger error
    SEVERE: Error in record "swap" on line 5.  Error in trade 2002.  Invalid pay_receive_code R.

### Fixed Width Fields, Single Record Type, Line Delimited

#### A flat file to XML mapping with field substitutions (abtest)

The example below shows how to convert a flat file to XML.

The input file is a positional file.

Figure 47. Input flat file abtest.txt

    ABTESTCD10
    ACTESTCD12
    ABTESTCC80

The requirements are as follows:

- First 2 chars: if AB the output will be SOUTH, if AC then output is WEST.
- For the 7th and 9th chars: if CD, then output change to 01, otherwise 02

The desired output is shown below.

Figure 48. Output XML file Recs.xml

    <?xml version="1.0" encoding="utf-8"?>
    <Recs>
      <Rec>
        <First>SOUTH</First>
        <Second>TEST</Second>
        <Third>01</Third>
        <Last>10</Last>
      </Rec>
      <Rec>
        <First>WEST</First>
        <Second>TEST</Second>
        <Third>01</Third>
        <Last>12</Last>
      </Rec>
      <Rec>
        <First>SOUTH</First>
        <Second>TEST</Second>
        <Third>02</Third>
        <Last>80</Last>
      </Rec>
    </Recs>

The following resources script does the transformation.

Figure 49. Resources script resources-abtest.xml

    <?xml version="1.0"?>
    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="abtest">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="data"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="data">
        <sx:flatFileReader>
          <sx:urlSource url="data/abtest.txt"/>
          <sx:flatFile ref="flatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="abtestToXml"/>
      </sx:recordContent>

      <sx:flatFile id="flatFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="match">
            <sx:positionalField name="first" width="2"/>
            <sx:positionalField name="second" width="4"/>
            <sx:positionalField name="third" width="2"/>
            <sx:positionalField name="fourth" width="2"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="abtestToXml">
        <Recs>
          <sx:onRecord>
            <Rec>
              <sx:fieldElementMap match="first[text()='AB']" select="'SOUTH'" element="First"/>
              <sx:fieldElementMap match="first[text()='AC']" select="'WEST'" element="First"/>
              <sx:fieldElementMap field="second" element="Second"/>
              <sx:fieldElementMap match="third[text()='CD']" select="'01'" element="Third"/>
              <sx:fieldElementMap match="third[text()!='CD']" select="'02'" element="Third"/>
              <sx:fieldElementMap field="fourth" element="Last"/>
            </Rec>
          </sx:onRecord>
        </Recs>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-abtest.xml -o output/abtest.xml abtest

#### Mapping a record to separate top level sections in the XML (persons)

Suppose you have the following positional file of persons and their addresses.

Figure 50. Input flat file persons.txt

    PersonId Name       FirstName       Street              Postcode    CityTown
    1        Jones      Joe             215 Walmer Rd       M5R3P7      Toronto
    1        Jones      Joe             180 Redwood ST      94102-3280  San Francisco
    2        Davis      Ken             212 Harbord St      M5S1H6      Toronto
    3        Morris     Jane            1100 Danforth Ave   M4J1N2      Toronto
    3        Morris     Jane            6 Green St          94111-1402  San Francisco

Each line containes information about a person and an address. A person may have multiple lines indicating multiple addresses.

You want the XML output to look as follows.

Figure 51. Output XML file persons.xml

    <?xml version="1.0" encoding="utf-8"?>
    <Persons-Addresses>
      <Persons>
        <Person>
          <PersonId>1</PersonId>
          <Name>Jones</Name>
          <FirstName>Joe</FirstName>
        </Person>
        <Person>
          <PersonId>2</PersonId>
          <Name>Davis</Name>
          <FirstName>Ken</FirstName>
        </Person>
        <Person>
          <PersonId>3</PersonId>
          <Name>Morris</Name>
          <FirstName>Jane</FirstName>
        </Person>
      </Persons>
      <Addresses>
        <Address>
          <PersonId>1</PersonId>
          <Street>215 Walmer Rd</Street>
          <PostCode>M5R3P7</PostCode>
          <CityTown>Toronto</CityTown>
        </Address>
        <Address>
          <PersonId>1</PersonId>
          <Street>180 Redwood ST</Street>
          <PostCode>94102-3280</PostCode>
          <CityTown>San Francisco</CityTown>
        </Address>
        <Address>
          <PersonId>2</PersonId>
          <Street>212 Harbord St</Street>
          <PostCode>M5S1H6</PostCode>
          <CityTown>Toronto</CityTown>
        </Address>
        <Address>
          <PersonId>3</PersonId>
          <Street>1100 Danforth Ave</Street>
          <PostCode>M4J1N2</PostCode>
          <CityTown>Toronto</CityTown>
        </Address>
        <Address>
          <PersonId>3</PersonId>
          <Street>6 Green St</Street>
          <PostCode>94111-1402</PostCode>
          <CityTown>San Francisco</CityTown>
        </Address>
      </Addresses>
    </Persons-Addresses>

There are two ways to do this:

- Use a record mapping section with two sx:groupBy sections. In this case the implementation will buffer the records in memory until the first group is finished, and only then do the second.

- Within a record mapping section, read the flat file twice, first inserting the person data content, and then inserting the address content.

Taking the first approach, the resources script looks like this:

Figure 52. Resources script resources-persons1.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="personsAddresses">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="persons"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:flatFile id="personsData">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType ref="persons"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordContent id="persons">
        <sx:flatFileReader>
          <sx:flatFile ref="personsData"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="personsAddressesMapping"/>
      </sx:recordContent>

      <sx:flatRecordType id="persons" name="persons">
        <sx:positionalField name="PersonId" width="9"/>
        <sx:positionalField name="Name" width="11"/>
        <sx:positionalField name="FirstName" width="16"/>
        <sx:positionalField name="Street" width="20"/>
        <sx:positionalField name="PostCode" width="12"/>
        <sx:positionalField name="CityTown" width="20" />
      </sx:flatRecordType>

      <sx:recordMapping id="personsAddressesMapping">
        <Persons-Addresses>
          <Persons>
            <sx:groupBy fields="PersonId">
              <Person>
                <sx:fieldElementMap field="PersonId" element="PersonId"/>
                <sx:fieldElementMap field="Name" element="Name"/>
                <sx:fieldElementMap field="FirstName" element="FirstName"/>
                <sx:onRecord/>
              </Person>
            </sx:groupBy>
          </Persons>
          <Addresses>
            <sx:onRecord>
              <Address>
                <sx:fieldElementMap field="PersonId" element="PersonId"/>
                <sx:fieldElementMap field="Street" element="Street"/>
                <sx:fieldElementMap field="PostCode" element="PostCode"/>
                <sx:fieldElementMap field="CityTown" element="CityTown"/>
              </Address>
            </sx:onRecord>
          </Addresses>
        </Persons-Addresses>
      </sx:recordMapping>

    </sx:resources>

Taking the second approach, the resources script looks like this:

Figure 53. Resources script resources-persons2.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="personsAddresses">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="persons"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:flatFile id="personsData">
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType ref="persons"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordContent id="persons">
        <sx:recordMapping ref="personsAddressesMapping"/>
      </sx:recordContent>

      <sx:flatRecordType id="persons" name="persons">
        <sx:positionalField name="PersonId" width="9"/>
        <sx:positionalField name="Name" width="11"/>
        <sx:positionalField name="FirstName" width="16"/>
        <sx:positionalField name="Street" width="20"/>
        <sx:positionalField name="PostCode" width="12"/>
        <sx:positionalField name="CityTown" width="20" />
      </sx:flatRecordType>

      <sx:recordMapping id="personsAddressesMapping">
        <Persons-Addresses>
          <sx:nestedContent>
            <sx:recordContent>
              <sx:flatFileReader>
                <sx:flatFile ref="personsData"/>
              </sx:flatFileReader>
              <sx:recordMapping ref="personsMapping"/>
            </sx:recordContent>
          </sx:nestedContent>
          <sx:nestedContent>
            <sx:recordContent>
              <sx:flatFileReader>
                <sx:flatFile ref="personsData"/>
              </sx:flatFileReader>
              <sx:recordMapping ref="addressesMapping"/>
            </sx:recordContent>
          </sx:nestedContent>
        </Persons-Addresses>
      </sx:recordMapping>

      <sx:recordMapping id="personsMapping">
        <Persons>
          <sx:groupBy fields="PersonId">
            <Person>
              <sx:fieldElementMap field="PersonId" element="PersonId"/>
              <sx:fieldElementMap field="Name" element="Name"/>
              <sx:fieldElementMap field="FirstName" element="FirstName"/>
              <sx:onRecord/>
            </Person>
          </sx:groupBy>
        </Persons>
      </sx:recordMapping>

      <sx:recordMapping id="addressesMapping">
        <Addresses>
          <sx:onRecord>
            <Address>
              <sx:fieldElementMap field="PersonId" element="PersonId"/>
              <sx:fieldElementMap field="Street" element="Street"/>
              <sx:fieldElementMap field="PostCode" element="PostCode"/>
              <sx:fieldElementMap field="CityTown" element="CityTown"/>
            </Address>
          </sx:onRecord>
        </Addresses>
      </sx:recordMapping>

    </sx:resources>

Note the following:

- Since you only want to show distinct persons in the persons section, group by PersonId.

You can run the two examples on the command line by entering

    servingxml -i data/persons.txt -o output/persons1.xml -r resources-persons1.xml
        personsAddresses

    servingxml -i data/persons.txt -o output/persons2.xml -r resources-persons2.xml
        personsAddresses

#### Converting a directory of books positional files to XML files (all books)

The example below shows how to process all the files in the `data` directory matching the regular expression "books.*[.]txt", convert them from positional flat file formats to XML files, and write them out to the `output` directory.

As before, the input is a directory of files.

Figure 54. Directory containing input files

    [.]                     countries.csv           multivalued-field.csv
    [..]                    countries.xsd           plans.txt
    3545_JH4DA3_4_H_.xml    country-record.xsd      tasks.csv
    bad-countries.csv       exotics.txt             timesheets.csv
    books.20040613.txt      hot-record.xsd          trades.txt
    books.20040802.txt      hot-record.xsx
    books.txt               hot.txt
    books.xml               messages.properties

We want to take the positional files whose names match the regular expression `"(books.*)[.]txt"` and convert them all to XML files.

Figure 55. Books XML output files.

    books.xml            books.20040802.xml
    books.20040613.xml

The following resources script does the transformation.

Figure 56. Resources script resources-allbooks.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="all-books">
        <sx:recordStream>
          <sx:directoryReader directory="data">
            <sx:fileFilter pattern="books.*[.]txt"/>
          </sx:directoryReader>
          <sx:processRecord>
            <sx:parameter name="output-file">
              <sx:findAndReplace searchFor ="(books.*)[.]txt" replaceWith ="$1.xml">
                <sx:toString value="{name}"/>
              </sx:findAndReplace>
            </sx:parameter>
            <sx:serialize>
              <sx:xsltSerializer>
                <sx:fileSink directory="output" file="{$output-file}"/>
              </sx:xsltSerializer>
              <sx:transform>
                <sx:content ref="books"/>
              </sx:transform>
            </sx:serialize>
          </sx:processRecord>
        </sx:recordStream>
      </sx:service>

      <sx:recordContent id="books">
        <sx:flatFileReader>
          <sx:fileSource directory="{parentDirectory}" file="{name}"/>
          <sx:flatFile ref="booksFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="booksToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
      </sx:flatRecordType>

      <sx:recordMapping id="booksToXmlMapping">
        <myns:books xmlns:myns="http://mycompany.com/mynames/"
                         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                         xsi:schemaLocation="url2">
          <sx:onRecord>
            <myns:book>
              <sx:fieldAttributeMap field="category" attribute="categoryCode"/>
              <sx:fieldElementMap field="title" element="myns:title"/>
              <sx:fieldElementMap field="author" element="myns:author"/>
              <sx:fieldElementMap field="price" element="myns:price"/>
            </myns:book>
          </sx:onRecord>
        </myns:books>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-allbooks.xml all-books

#### Converting a flat file to multiple XML book files (multiple books)

The example below shows how to process one flat file and produce multiple XML output files.

The input file is shown below.

Figure 57. Books input file

    CAuthor                        Title                              Price ISBN

    FCharles Bukowski              Factotum                           22.95 0876852630
    FSergei Lukyanenko             The Night Watch                    17.99 0434014125
    FAndrew Crumey                 Mr Mee                             22.00 0312282354
    CSteven John Metsker           Building Parsers with Java         39.95 0201719622

    This is a trailer record

We want to produce multiple files like the one shown below, with filenames differentiated by the isbn number.

Figure 58. Book XML output file book-0876852630.xml.

    <?xml version="1.0" encoding="UTF-8"?>
    <myns:book xmlns:myns="http://mycompany.com/mynames/"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               categoryCode="F">
      <myns:title>Factotum</myns:title>
      <myns:author>Charles Bukowski</myns:author>
      <myns:price>22.95</myns:price>
      <myns:isbn>0876852630</myns:isbn>
    </myns:book>

The following resources script does the transformation.

Figure 59. Resources script resources-books_multiple.xml

    <?xml version="1.0"?>

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                         xmlns:myns="http://mycompany.com/mynames/" >

      <sx:service id="books">
        <sx:transform>
          <sx:content ref="books"/>
          <sx:processSubtree path="/myns:books/myns:book">
            <sx:parameter name="isbn" select="myns:isbn"/>
            <sx:serialize>
              <sx:xsltSerializer>
                <sx:fileSink directory="output" file="book-{$isbn}.xml"/>
              </sx:xsltSerializer>
            </sx:serialize>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

      <!--
      This sx:flatFileReader element does not specify a stream source, so
      the source will default to the file specified with the -i option on the command line.
      -->
      <sx:recordContent id="books">
        <sx:flatFileReader>
          <sx:flatFile ref="booksFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="booksToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
        <sx:positionalField name="isbn" label="ISBN" start="73" width="10"/>
      </sx:flatRecordType>

      <sx:recordMapping id="booksToXmlMapping">
        <myns:books xmlns:myns="http://mycompany.com/mynames/"
                         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                         xsi:schemaLocation="url2">
          <sx:onRecord>
            <myns:book>
              <sx:fieldAttributeMap field="category" attribute="categoryCode"/>
              <sx:fieldElementMap field="title" element="myns:title"/>
              <sx:fieldElementMap field="author" element="myns:author"/>
              <sx:fieldElementMap field="price" element="myns:price"/>
              <sx:fieldElementMap field="isbn" element="myns:isbn"/>
            </myns:book>
          </sx:onRecord>
        </myns:books>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-books_multiple.xml -i data/books.txt books

#### Converting a flat file to multiple XML documents with default namespace prefixes (multiple default ns)

The example below shows how to process one flat file and produce multiple XML documents with default namespace prefixes.

The input file is the same as in the previous example.

Figure 60. Books input file

    CAuthor                        Title                              Price ISBN

    FCharles Bukowski              Factotum                           22.95 0876852630
    FSergei Lukyanenko             The Night Watch                    17.99 0434014125
    FAndrew Crumey                 Mr Mee                             22.00 0312282354
    CSteven John Metsker           Building Parsers with Java         39.95 0201719622

    This is a trailer record

This time we want to produce multiple files like the one shown below, with a default namespace prefix.

Figure 61. Book XML output file book-with-default-ns-0156028972.xml.

    <?xml version="1.0" encoding="UTF-8"?>
    <book xmlns="http://mycompany.com/mynames/"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          categoryCode="F">
       <title>Gun, with Occasional Music</title>
       <author>Jonathan Lethem</author>
       <price>17.99</price>
       <isbn>0156028972</isbn>
    </book>

The following resources script does the transformation.

Figure 62. Resources script resources-multiple_default_ns.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="books">
        <sx:transform>
          <sx:content ref="books"/>
          <sx:processSubtree path="/myns:books/myns:book">
            <sx:parameter name="isbn" select="myns:isbn"/>
            <sx:serialize>
              <sx:xsltSerializer>
                <sx:fileSink directory="output" file="book-with-default-ns-{$isbn}.xml"/>
                <sx:outputProperty name="indent" value="yes"/>
              </sx:xsltSerializer>
            </sx:serialize>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

      <!--
      This sx:flatFileReader element does not specify a stream source, so
      the source will default to the file specified with the -i option on the command line.
      -->
      <sx:recordContent id="books">
        <sx:flatFileReader>
          <sx:flatFile ref="booksFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="booksToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
        <sx:positionalField name="isbn" label="ISBN" start="73" width="10"/>
      </sx:flatRecordType>

      <sx:recordMapping id="booksToXmlMapping">
        <books xmlns="http://mycompany.com/mynames/"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="url2">
          <sx:onRecord>
            <book>
              <sx:fieldAttributeMap field="category" attribute="myns:categoryCode"/>
              <sx:fieldElementMap field="title" element="myns:title"/>
              <sx:fieldElementMap field="author" element="myns:author"/>
              <sx:fieldElementMap field="price" element="myns:price"/>
              <sx:fieldElementMap field="isbn" element="myns:isbn"/>
            </book>
          </sx:onRecord>
        </books>
      </sx:recordMapping>

    </sx:resources>

##### Remarks

- Note that the root element in the sx:recordMapping section contains a default namespace declaration, which will appear in the output XML, and which will have the effect that XML names belonging to the namespace "http://mycompany.com/mynames/" will be output without an explicit prefix.

- Note that the sx:fieldAttributeMap and sx:fieldElementMap elements use the `myns` prefix to associate attribute and element names with the namespace "http://mycompany.com/mynames/". The prefix is resolved against the nearest in-scope prefix declaration for `myns` , which happens to be the one in the sx:resources element. Since this declaration does not appear inside the record mapping section, it is used only to associate a name with a namespace, not for serialization.

- If a prefix were omitted in an attribute or element mapping, say on `title` , that element would belong to no namespace, all others would belong to the default namespace, and the output would become

      <book xmlns="http://mycompany.com/mynames/"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            categoryCode="F">
         <title xmlns="">Gun, with Occasional Music</title>
         <author>Jonathan Lethem</author>
         <price>17.99</price>
         <isbn>0156028972</isbn>
      </book>

You can run this example on the command line by entering

    servingxml -r resources-multiple_default_ns.xml -i data/books.txt books

### Flat Files with Multibyte Characters

#### A UTF-8 positional flat file with double byte characters

The example below shows how to convert a positional flat file with multi-byte encoding to XML.

The input file is a positional flat file containing the following hex UTF-8 byte input.

Figure 63. Input flat file doubleByte.txt

    41414242 43430A41 C3964242 43430A

The two byte sequence C396 represents the single character Ö.

The desired output is shown below.

Figure 64. Output XML file doubleByte.xml

    <?xml version="1.0" encoding="UTF-8" ?>
    <document>
      <documentData>
        <param1>AA</param1>
        <param2>BB</param2>
        <param3>CC</param3>
      </documentData>
      <documentData>
        <param1>AÖ</param1>
        <param2>BB</param2>
        <param3>CC</param3>
      </documentData>
    </document>

The following resources script does the transformation.

Figure 65. Resources script resources-doubleByte.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="convert">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="data"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="data">
        <sx:flatFileReader>
          <sx:flatFile ref="flatfile"/>
          <sx:defaultStreamSource encoding="UTF-8"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="recordmap"/>
      </sx:recordContent>

      <sx:flatFile id="flatfile">
        <sx:flatFileBody>
          <sx:flatRecordType ref="recordtype"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="recordtype">
        <sx:positionalField name="param1" width="2"/>
        <sx:positionalField name="param2" width="2"/>
        <sx:positionalField name="param3" width="2"/>
      </sx:flatRecordType>

      <sx:recordMapping id="recordmap">
        <document>
          <sx:onRecord>
            <documentData>
              <sx:fieldElementMap element="param1" field="param1"/>
              <sx:fieldElementMap element="param2" field="param2"/>
              <sx:fieldElementMap element="param3" field="param3"/>
            </documentData>
          </sx:onRecord>
        </document>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-doubleByte.xml -i data/doubleByte.dat
        -o output/doubleByte.xml convert

### Fixed Width Fields, Multiple Record Types, Line Delimited

#### Converting a flat file with multiple record formats to XML (trades)

The example below shows how to convert a flat file with multiple record formats to XML.

The input file is a positional flat file.

Figure 66. Input flat file trades.txt

    TR000103/25/2005 1:50:00This is a trade record
    TN0002X1234A child transaction
    TN0003X1235Another child transaction

The desired output is shown below.

Figure 67. Output XML file trades.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <trades feed="LONDON">
      <trade id="0001">
        <trade-date>2005-03-25T01:50:00.000-05:00</trade-date>
        <description>This is a trade record</description>
        <transaction id="0002">
          <reference>X1234</reference>
          <description>A child transaction</description>
        </transaction>
        <transaction id="0003">
          <reference>X1235</reference>
          <description>Another child transaction</description>
        </transaction>
      </trade>
    </trades>

The following resources script does the transformation.

Figure 68. Resources script resources-trades.xml

    <?xml version="1.0"?>
    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="trades">
        <sx:parameter name="feed">
          <sx:defaultValue>NY</sx:defaultValue>
        </sx:parameter>
        <sx:serialize>
          <sx:transform>
            <sx:content ref="trades"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="trades">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:urlSource url="data/trades.txt"/>
            <sx:flatFile ref="tradesFlatFile"/>
          </sx:flatFileReader>
          <sx:composeRecord recordType="composite" repeatingGroup="detail"
                                                 startTest="sx:current//record_type='TR'"
                                                 endTest="sx:current//record_type='TR'">
            <sx:newField name="id" select="detail/trade/id"/>
          </sx:composeRecord>
        </sx:recordStream>
        <sx:recordMapping ref="tradesToXmlMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="tradesToXmlMapping">
        <trades>
          <sx:fieldAttributeMap value="{$feed}" attribute="feed"/>
          <sx:onRecord recordType="composite">
            <trade>
              <sx:fieldAttributeMap field="id" attribute="id"/>
              <sx:subrecordMapping repeatingGroup="detail">
                <sx:onRecord recordType="trade">
                  <sx:elementMap element="trade-date">
                    <sx:convertToDateTime format="MM/dd/yyyy H:mm:ss">
                      <sx:concat separator=" ">
                        <sx:toString value="{trade_date}"/>
                        <sx:toString value="{trade_time}"/>
                      </sx:concat>
                    </sx:convertToDateTime>
                  </sx:elementMap>
                  <sx:fieldElementMap field="description" element="description"/>
                </sx:onRecord>
                <sx:onRecord recordType="transaction">
                  <transaction>
                    <sx:fieldAttributeMap field="id" attribute="id"/>
                    <sx:fieldElementMap field="reference" element="reference"/>
                    <sx:fieldElementMap field="description" element="description"/>
                  </transaction>
                </sx:onRecord>
              </sx:subrecordMapping>
            </trade>
          </sx:onRecord>
        </trades>
      </sx:recordMapping>

      <sx:flatFile id="tradesFlatFile">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="record_type" width="2"/>
            <sx:when test="record_type='TR'">
              <sx:flatRecordType name="trade">
                <sx:positionalField name="record_type" width="2"/>
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="trade_date" width="10"/>
                <sx:positionalField name="trade_time" width="8"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="record_type='TN'">
              <sx:flatRecordType name="transaction">
                <sx:positionalField name="record_type" width="2"/>
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="reference" width="5"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

Note the following points about the content of the<sx:elementMap element="trade-date"> element.

- The inner sx:concat element concatenates the date and time fields from the record, separating them with a space.
- The sx:convertDate element converts the datetime value into something close to an ISO 8601 date, where the input and output formats follow the syntax specified for the JDK SimpleDateFormat class.
- The outer sx:findAndReplace element inserts a colon into the time format.

You can run this example on the command line by entering

    servingxml -r resources-trades.xml -o output/trades.xml trades feed=LONDON

#### Converting a flat file to XML with header, detail, and summary records (CONV)

The example below shows how to convert a flat file with header, detail, and summary records to XML.

The input file is a positional flat file with variable record layouts.

Figure 69. Input flat file CONV_in.dat

    90112345678abcdefg1000001
    100SURNAME1 GIVEN1 A
    100SURNAME2 GIVEN2 B
    3011111111AB111111

The desired output is shown below.

Figure 70. Output XML file CONV.xml

    <?xml version="1.0" encoding="utf-8"?>
    <Submission>
      <Header>
        <record_type>901</record_type>
        <sbmt_ref_id>abcdefg</sbmt_ref_id>
        <trnmtr_tcd>1</trnmtr_tcd>
        <summ_cnt>000001</summ_cnt>
      </Header>
      <All>
        <Level1>
          <Detail>
            <snm>SURNAME1</snm>
            <gvn_nm>GIVEN1</gvn_nm>
            <init>A</init>
          </Detail>
          <Detail>
            <snm>SURNAME2</snm>
            <gvn_nm>GIVEN2</gvn_nm>
            <init>B</init>
          </Detail>
          <Summary>
            <bn>1111111AB111111</bn>
          </Summary>
        </Level1>
      </All>
    </Submission>

The following resources script does the transformation.

Figure 71. Resources script resources-CONV.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="CONV">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="CONV"/>
            <sx:removeEmptyElements elements="*"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="CONV">
        <sx:flatFileReader>
          <sx:urlSource url="data/CONV_in.dat"/>
          <sx:flatFile ref="CONVFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="CONVToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="CONVFlatFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="record_type" width="3"/>
            <sx:when test="record_type='901'">
              <sx:flatRecordType name="header">
                <sx:positionalField name="record_type" width="3"/>
                <sx:positionalField name="trnmtr_nbr" width="8"/>
                <sx:positionalField name="sbmt_ref_id" width="7"/>
                <sx:positionalField name="trnmtr_tcd" width="1"/>
                <sx:positionalField name="summ_cnt" width="6"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="record_type='100'">
              <sx:flatRecordType name="detail">
                <sx:positionalField name="record_type" width="3"/>
                <sx:positionalField name="snm" width="9"/>
                <sx:positionalField name="gvn_nm" width="7"/>
                <sx:positionalField name="init" width="1"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="record_type='301'">
              <sx:flatRecordType name="summary">
                <sx:positionalField name="record_type" width="3"/>
                <sx:positionalField name="bn" width="15"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="CONVToXmlMapping">
        <Submission>
          <sx:onRecord recordType="header">
            <Header>
              <sx:fieldElementMap field="record_type" element="record_type"/>
              <sx:fieldElementMap field="sbmt_ref_id" element="sbmt_ref_id"/>
              <sx:fieldElementMap field="trnmtr_tcd" element="trnmtr_tcd"/>
              <sx:fieldElementMap field="summ_cnt" element="summ_cnt"/>
            </Header>
          </sx:onRecord>
          <sx:innerGroup startTest="sx:current/detail or sx:current/summary"
                         endTest="not(sx:current/detail or sx:current/summary)">
            <All>
              <Level1>
                <sx:onRecord recordType='detail'>
                  <Detail>
                    <sx:fieldElementMap field="snm" element="snm"/>
                    <sx:fieldElementMap field="gvn_nm" element="gvn_nm"/>
                    <sx:fieldElementMap field="init" element="init"/>
                  </Detail>
                </sx:onRecord>
                <sx:onRecord recordType="summary">
                  <Summary>
                    <sx:fieldElementMap field="bn" element="bn"/>
                  </Summary>
                </sx:onRecord>
              </Level1>
            </All>
          </sx:innerGroup>
        </Submission>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -o output/CONV.xml -r resources-CONV.xml CONV

#### Converting a positional file to XML using a record filter (hot 1)

This is the first of two examples that map the `hot.txt` flat file to XML.

The input file is a variable record positional file.

Figure 72. Input flat file hot.txt

    BFH01value01
    BCH02value02
    BOH03value03
    BKT06value04issue
    BKP84value05A6USD2
    BOT93value06
    BOT94value07
    BOH03value08
    BKT06value09issue
    BKP84value10A6CAD2
    BOT93value11
    BKT06value12refund
    BKP84value13A6USD2
    BOT93value14
    BOT94value15
    BCT95value16
    BFT99value17

This file has variant record types. The first five characters identify the record type, and the next seven characters represent a hypothetical attribute. The BKT06 and BKP84 records have additional fields.

The desired output is shown below.

Figure 73. Output XML file hot1.xml

    <?xml version="1.0" encoding="utf-8" ?>
    <hot>
       <BFH01 attr1="value01"/>
       <BCH02 attr1="value02"/>
       <BOH03 attr1="value03"/>
       <BKT06 attr1="value04"/>
       <BKP84 attr1="value05" amount="1.66" currency="USD"/>
       <BOT93 attr1="value06"/>
       <BOT94 attr1="value07"/>
       <BOH03 attr1="value08"/>
       <BKT06 attr1="value09"/>
       <BKP84 attr1="value10" amount="2.46" currency="CAD"/>
       <BOT93 attr1="value11"/>
       <BKT06 attr1="value12"/>
       <BKP84 attr1="value13" amount="2.27" currency="USD"/>
       <BOT93 attr1="value14"/>
       <BOT94 attr1="value15"/>
       <BCT95 attr1="value16"/>
       <BFT99 attr1="value17"/>
    </hot>

The following resources script does the transformation.

Figure 74. flatfile-hot.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:flatFile id="hotFlatFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="record-type" width="5"/>

            <sx:when  test="record-type='BKP84'">
              <sx:flatRecordType name="bkp84">
                <sx:positionalField name="record-type-prefix"  width="3"/>
                <sx:positionalField name="record-type"  start="1" width="5"/>
                <sx:positionalField name="value" width="7"/>
                <sx:positionalField name="amount" width="2"/>
                <sx:positionalField name="currency" width="3"/>
                <sx:positionalField name="precision" width="1"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="record-type='BKT06'">
              <sx:flatRecordType name="bkt06">
                <sx:positionalField name="record-type-prefix"  width="3"/>
                <sx:positionalField name="record-type"  start="1" width="5"/>
                <sx:positionalField name="value" width="7"/>
                <sx:positionalField name="type" width="6"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:otherwise>
              <sx:flatRecordType name="other">
                <sx:positionalField name="record-type-prefix"  width="3"/>
                <sx:positionalField name="record-type" start="1" width="5"/>
                <sx:positionalField name="value" width="7"/>
              </sx:flatRecordType>
            </sx:otherwise>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

The values in the `bkp84` record need to be transformed as follows:

- The amount field is encoded as a hexadecimal value and needs to be converted to a decimal value.

- The resulting decimal value needs to be converted to the specified precision.

In the hot 1 example in this section, we transform the values using a sx:customRecordFilter element to define a custom record filter written in Java. In the hot 2 example in the next section, we take a different approach, transforming the values within an XPath select expression against the record fields, making use of extension calls to static Java functions.

Figure 75. XML Schema file hot-record.xsd

    <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <!-- This element's name matches the value of the name attribute in the "bkp84" sx:flatFileRecordType. -->
      <xs:element name="bkp84" type="bkp84Type"/>

      <!-- This element's name matches the value of the name attribute in the "bkt06" sx:flatFileRecordType. -->
      <xs:element name="bkt06" type="bkt06Type"/>

      <!-- This element's name matches the value of the name attribute in the "other" sx:flatFileRecordType. -->
      <xs:element name="other" type="otherType"/>

      <xs:complexType name="bkp84Type">
        <xs:sequence>
          <xs:element name="record-type-prefix" type="xs:string"/>
          <xs:element name="record-type" type="xs:string"/>
          <xs:element name="value" type="xs:string"/>
          <xs:element name="amount" type="xs:hexBinary"/>
          <xs:element name="currency" type="xs:string"/>
          <xs:element name="precision" type="xs:nonNegativeInteger"/>
        </xs:sequence>
      </xs:complexType>

      <xs:complexType name="bkt06Type">
        <xs:sequence>
          <xs:element name="record-type-prefix" type="xs:string"/>
          <xs:element name="record-type" type="xs:string"/>
          <xs:element name="value" type="xs:string"/>
          <xs:element name="type" type="xs:string"/>
        </xs:sequence>
      </xs:complexType>

      <xs:complexType name="otherType">
        <xs:sequence>
          <xs:element name="record-type-prefix" type="xs:string"/>
          <xs:element name="record-type" type="xs:string"/>
          <xs:element name="value" type="xs:string"/>
        </xs:sequence>
      </xs:complexType>
    </xs:schema>

Figure 76. Resources script resources-hot1.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:include href="flatfile-hot.xml"/>

      <sx:service id="hot1">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="hot1"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="hot1">
        <sx:flatFileReader>
          <sx:flatFile ref="hotFlatFile"/>
        </sx:flatFileReader>
        <msv:recordValidator>
          <sx:urlSource url="data/hot-record.xsd"/>
        </msv:recordValidator>
        <sx:customRecordFilter class="flat2xml.HotRecordFilter"/>
        <sx:discardHandler>
          <sx:log message="{$sx:message}"/>
        </sx:discardHandler>
        <sx:recordMapping ref="hot1ToXmlMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="hot1ToXmlMapping">
        <hot>
          <sx:onRecord recordType="bkp84">
            <sx:elementMap element="{record-type}">
              <sx:fieldAttributeMap field="value" attribute="attr1"/>
              <sx:fieldAttributeMap field="calculatedAmount"  attribute="amount"/>
              <sx:fieldAttributeMap field="currency" attribute="currency"/>
            </sx:elementMap>
          </sx:onRecord>
          <sx:onRecord recordType="bkt06">
            <sx:elementMap element="{record-type}">
              <sx:fieldAttributeMap field="value" attribute="attr1"/>
            </sx:elementMap>
          </sx:onRecord>
          <sx:onRecord recordType="other">
            <sx:elementMap element="{record-type}">
              <sx:fieldAttributeMap field="value" attribute="attr1"/>
            </sx:elementMap>
          </sx:onRecord>
        </hot>
      </sx:recordMapping>

    </sx:resources>

Figure 77. HotRecordFilter class

    package flat2xml;

    import com.servingxml.app.ServiceContext;
    import com.servingxml.components.recordio.AbstractRecordFilter;
    import com.servingxml.app.Flow;
    import com.servingxml.util.Name;
    import com.servingxml.util.QualifiedName;
    import com.servingxml.util.ServingXmlException;
    import com.servingxml.util.record.Record;
    import com.servingxml.util.record.RecordBuilder;

    public class HotRecordFilter extends AbstractRecordFilter {
      private static final Name BKP84_RECORD_TYPE = new QualifiedName("bkp84");
      private static final Name AMOUNT_NAME = new QualifiedName("amount");
      private static final Name PRECISION_NAME = new QualifiedName("precision");
      private static final Name CALCULATED_AMOUNT_NAME = new QualifiedName("calculatedAmount");

      public void writeRecord(ServiceContext context, Flow flow)
       {

        Record record = flow.getRecord();
        Flow newFlow = flow;
        if (record.getRecordType().getName().equals(BKP84_RECORD_TYPE)) {
          RecordBuilder recordBuilder = new RecordBuilder(record);
          String amountString = record.getString(AMOUNT_NAME);
          if (amountString == null) {
            throw new ServingXmlException("amount is NULL");
          }
          String precisionString = record.getString(PRECISION_NAME);
          if (precisionString == null) {
            throw new ServingXmlException("precision is NULL");
          }
          int amount = Integer.parseInt(amountString,16);
          int precision = Integer.parseInt(precisionString);
          double calculatedAmount = (double)amount/Math.pow(10.0,(double)precision);
          recordBuilder.setDouble(CALCULATED_AMOUNT_NAME,calculatedAmount);
          record = recordBuilder.toRecord();
          newFlow = flow.replaceRecord(context, record);
        }

        super.writeRecord(context, newFlow);
      }
    }

You can run this example on the command line by

- Compiling the Java class `HotRecordFilter` and copying the resulting `.class` file into the `dir/classes` directory
- Entering the command

      servingxml -r resources-hot1.xml -i data/hot.txt -o output/hot1.xml hot1

#### Converting a positional file to XML with outer and inner grouping (hot 2)

This version of the hot example shows how to convert a flat file to XML document with outer and inner grouping. Each row in the flat file is validated with Sun's multi schema validator, and the row is discarded if an error is encountered.

This examples illustrates the sx:innerGroup and sx:outerGroup elements. Both elements use the `startTest` and `endTest` attributes to define the beginning and end of a group through XPath boolean expressions applied to records. They differ in that sx:innerGroup will never allow any descendent elements to be produced unless a group is recognized at its level, whereas a sx:outerGroup will only suppress content between this element and the next succeeding grouping element.

The `resources-hot2.xml` file defines the record mapping for this version of the hot example. The `BFH01` record type marks the beginning of a group of level 1, the `BFT99` s mark the end. The `BCH02` s mark the beginning of a group of level 2, the `BCT95` s mark the end. The `BOH03` marks the beginning of a group of level 3, the `BOT94s` mark the end.

The desired output is shown below.

Figure 78. Output XML file hot2.xml

    <?xml version="1.0" encoding="utf-8" ?>
    <hot>
       <BFH01 attr1="value01"/>
       <periods>
          <period>
             <BCH02s>
                <BCH02 attr1="value02"/>
             </BCH02s>
             <agents>
                <agent>
                   <BOH03 attr1="value03"/>
                   <issues>
                      <issue>
                         <BKT06 attr1="value04"/>
                         <BKP84 attr1="value05" amount="1.66" currency="USD"/>
                      </issue>
                      <BOT93s>
                         <BOT93 attr1="value06"/>
                      </BOT93s>
                   </issues>
                   <BOT94s>
                      <BOT94 attr1="value07"/>
                   </BOT94s>
                </agent>
                <agent>
                   <BOH03 attr1="value08"/>
                   <issues>
                      <issue>
                         <BKT06 attr1="value09"/>
                         <BKP84 attr1="value10" amount="2.46" currency="CAD"/>
                      </issue>
                      <BOT93s>
                         <BOT93 attr1="value11"/>
                      </BOT93s>
                   </issues>
                   <refunds>
                      <refund>
                         <BKT06 attr1="value12"/>
                         <BKP84 attr1="value13" amount="2.27" currency="USD"/>
                      </refund>
                      <BOT93s>
                         <BOT93 attr1="value14"/>
                      </BOT93s>
                   </refunds>
                   <BOT94s>
                      <BOT94 attr1="value15"/>
                   </BOT94s>
                </agent>
             </agents>
             <BCT95s>
                <BCT95 attr1="value16"/>
             </BCT95s>
          </period>
       </periods>
       <BFT99s>
          <BFT99 attr1="value17"/>
       </BFT99s>
    </hot>

The following resources script does the transformation.

Figure 79. Resources script resources-hot2.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv"
                  xmlns:integer="java.lang.Integer" xmlns:math="java.lang.Math">

      <sx:include href="flatfile-hot.xml"/>

      <sx:service id="hot2">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="hot2"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="hot2">
        <sx:flatFileReader>
          <sx:flatFile ref="hotFlatFile"/>
        </sx:flatFileReader>
        <msv:recordValidator>
          <sx:urlSource url="data/hot-record.xsd"/>
        </msv:recordValidator>
        <sx:discardHandler>
          <sx:log message="{$sx:message}"/>
        </sx:discardHandler>
        <sx:recordMapping ref="hot2ToXmlMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="hot2ToXmlMapping">
        <hot>
          <sx:outerGroup startTest="sx:previous//record-type='BFH01'"
                         endTest="sx:current//record-type='BFT99'">
            <periods>
              <sx:outerGroup startTest="sx:current//record-type='BCH02'"
                             endTest="sx:previous//record-type='BCT95'">
                <period>
                  <sx:outerGroup startTest="sx:previous//record-type='BCH02'"
                                 endTest="sx:current//record-type='BCT95'">
                    <agents>
                      <sx:outerGroup startTest="sx:current//record-type='BOH03'"
                                     endTest="sx:previous//record-type='BOT94'">
                        <agent>
                          <sx:groupChoice>
                            <sx:innerGroup startTest="sx:current//record-type-prefix='BFH'"
                                           endTest="sx:previous//record-type-prefix='BFH'">
                              <sx:onRecord>
                                <sx:elementMap element="{record-type}">
                                  <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                </sx:elementMap>
                              </sx:onRecord>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type-prefix='BOH'"
                                           endTest="sx:previous//record-type-prefix='BOH'">
                              <sx:onRecord>
                                <sx:elementMap element="{record-type}">
                                  <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                </sx:elementMap>
                              </sx:onRecord>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type-prefix='BFT'"
                                           endTest="sx:previous//record-type-prefix='BFT'">
                              <sx:elementMap element="{record-type}s">
                                <sx:onRecord>
                                  <sx:elementMap element="{record-type}">
                                    <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                  </sx:elementMap>
                                </sx:onRecord>
                              </sx:elementMap>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type-prefix='BCT'"
                                           endTest="sx:previous//record-type-prefix='BCT'">
                              <sx:elementMap element="{record-type}s">
                                <sx:onRecord>
                                  <sx:elementMap element="{record-type}">
                                    <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                  </sx:elementMap>
                                </sx:onRecord>
                              </sx:elementMap>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type='BOT94'"
                                           endTest="sx:previous//record-type='BOT94'">
                              <sx:elementMap element="{record-type}s">
                                <sx:onRecord>
                                  <sx:elementMap element="{record-type}">
                                    <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                  </sx:elementMap>
                                </sx:onRecord>
                              </sx:elementMap>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type='BCH02'"
                                           endTest="sx:previous//record-type='BOH03'">
                              <sx:elementMap element="{record-type}s">
                                <sx:onRecord>
                                  <sx:elementMap element="{record-type}">
                                    <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                  </sx:elementMap>
                                </sx:onRecord>
                              </sx:elementMap>
                            </sx:innerGroup>
                            <sx:innerGroup startTest="sx:current//record-type='BKT06'"
                                           endTest="sx:previous//record-type='BOT93'">
                              <sx:elementMap element="{type}s">
                                <sx:groupChoice>
                                  <sx:innerGroup startTest="sx:current//record-type='BKT06'"
                                                 endTest="sx:previous//record-type='BKP84'">
                                    <sx:elementMap element="{type}">
                                      <sx:onRecord recordType="bkt06">
                                        <sx:elementMap element="{record-type}">
                                          <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                        </sx:elementMap>
                                      </sx:onRecord>
                                      <sx:onRecord recordType="bkp84">
                                        <sx:elementMap element="{record-type}">
                                          <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                          <sx:fieldAttributeMap select="integer:valueOf(amount,16) div math:pow(10,precision)"  attribute="amount"/>
                                          <sx:fieldAttributeMap field="currency" attribute="currency"/>
                                        </sx:elementMap>
                                      </sx:onRecord>
                                    </sx:elementMap>
                                  </sx:innerGroup>
                                  <sx:innerGroup startTest="sx:current//record-type-prefix='BOT'"
                                                 endTest="sx:previous//record-type-prefix='BOT'">
                                    <sx:elementMap element="{record-type}s">
                                      <sx:onRecord>
                                        <sx:elementMap element="{record-type}">
                                          <sx:fieldAttributeMap field="value" attribute="attr1"/>
                                        </sx:elementMap>
                                      </sx:onRecord>
                                    </sx:elementMap>
                                  </sx:innerGroup>
                                </sx:groupChoice>
                              </sx:elementMap>
                            </sx:innerGroup>
                          </sx:groupChoice>
                        </agent>
                      </sx:outerGroup>
                    </agents>
                  </sx:outerGroup>
                </period>
              </sx:outerGroup>
            </periods>
          </sx:outerGroup>
        </hot>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-hot2.xml -i data/hot.txt -o output/hot2.xml hot2

#### Converting a positional flat file with variable field widths to XML

The example below shows how to convert a flat file with variable field widths to XML.

The input file is a positional flat file.

Figure 80. Input flat file testWidth.txt

    TX#00000000010016This is a remarkXXXXXXXXXX
    TX#0000000002
    TX#0000000003    YYYYYYYYYY
    TX#00000000030004TestZZZZZZZZZZ

This file contains an optional positional field called "remarksText". The field size varies and when the field exists it's width is specified using the value of its preceding field called "remarksSize". When no "remarksText" exists, 4 space characters are specified as "remarksSize" in the file directly followed by the value of "followingField".

The desired output is shown below.

Figure 81. Output XML file testWidth.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Doc>
      <Transaction>
        <Reference>0000000001</Reference>
        <Remarks>This is a remark</Remarks>
        <FollowingField>XXXXXXXXXX</FollowingField>
      </Transaction>
      <Transaction>
        <Reference>0000000002</Reference>
        <Remarks/>
        <FollowingField/>
      </Transaction>
      <Transaction>
        <Reference>0000000003</Reference>
        <Remarks/>
        <FollowingField>YYYYYYYYYY</FollowingField>
      </Transaction>
      <Transaction>
        <Reference>0000000003</Reference>
        <Remarks>Test</Remarks>
        <FollowingField>ZZZZZZZZZZ</FollowingField>
      </Transaction>
    </Doc>

The following resources script does the transformation.

Figure 82. Resources script resources-testWidth.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="testWidth">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="data"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="data">
        <sx:flatFileReader>
          <sx:flatFile ref="flatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="segmentsToXml"/>
      </sx:recordContent>

      <sx:flatFile id="flatFile">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="tag" width="3"/>
            <sx:when test="tag='TX#'">
              <sx:flatRecordType name="transaction">
                <sx:positionalField name="recordType" width="3"/>
                <sx:positionalField name="reference" width="10"/>
                <sx:positionalField name="remarksSize" width="4">
                  <sx:defaultValue value="0"/>
                </sx:positionalField>
                <sx:positionalField name="remarksText" width="{remarksSize}"/>
                <sx:positionalField name="followingField" width="10"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="segmentsToXml">
        <Doc>
          <sx:innerGroup startTest="sx:current/transaction">
            <Transaction>
              <sx:fieldElementMap field="reference"  element="Reference"/>
              <sx:fieldElementMap field="remarksText" element="Remarks"/>
              <sx:fieldElementMap field="followingField" element="FollowingField"/>
            </Transaction>
          </sx:innerGroup>
        </Doc>
      </sx:recordMapping>
    </sx:resources>

Note the following.

- When the "remarksSize" is not specified as shown in row 2 and 3 of the flat file, the value for {remarksSize} must default to `0` . The "remarksSize" positional field definition therefore contains a sx:defaultValue element, which supplies the default.

You can run this example on the command line by entering

    servingxml -r resources-testWidth.xml -i testWidth.txt -o output/testWidth.xml testWidth

#### Repeat Header Information in the XML

The example below shows how to convert a flat file to XML with header information repeated when processing the other records.

The record input is

Figure 83. Record Input

    1A
    2B
    3C
    2D
    3E

The desired output is shown below.

Figure 84. Output XML file repeated_header_info.xml

    <doc>
      <b attr="B">
        <c attr="C">A</c>
      </b>
      <b attr="D">
        <c attr="E">A</c>
      </b>
    </doc>

The following resources script does the transformation.

Figure 85. Resources script resources-repeatHeader.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="hut">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="HutContent"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="HutContent">
        <sx:flatFileReader ref="HutFlatFileReader"/>
        <sx:recordMapping ref="hut2ToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="HutFlatFile">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="record-type" start="1" width="1"/>
            <sx:when test='record-type="1"'>
              <sx:flatRecordType id="R1" name="R1">
                <sx:positionalField name="TOTO" start="2" width="1"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test='record-type="2"'>
              <sx:flatRecordType id="R2" name="R2">
                <sx:positionalField name="TATA" start="2" width="1"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test='record-type="3"'>
              <sx:flatRecordType id="R3" name="R3">
                <sx:positionalField name="TITI" start="2" width="1"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatFileReader id="HutFlatFileReader">
        <sx:inlineSource>1A
    2B
    3C
    2D
    3E</sx:inlineSource>
        <sx:flatFile ref="HutFlatFile"/>
      </sx:flatFileReader>

      <sx:recordMapping id="hut2ToXmlMapping">
        <doc>
          <sx:parameter name="temp" value="{TOTO}"/>
          <sx:outerGroup startTest="sx:current/R2">
            <b>
              <sx:fieldAttributeMap value="{TATA}" attribute="attr"/>
              <sx:onRecord recordType="R3">
                <sx:fieldElementMap value="{$temp}" element="c">
                  <sx:fieldAttributeMap value="{TITI}" attribute="attr"/>
                </sx:fieldElementMap>
              </sx:onRecord>
            </b>
          </sx:outerGroup>
        </doc>
      </sx:recordMapping>

    </sx:resources>

Note the parameter declaration for "temp" in the record mapping section. This parameter declaration occurs immediately below the document element, which represents a grouping of all records. Within a group, `ServingXML` ServingXML binds the parameter name to the record that begins the grouping section, in this case the first header record in the input.

You can run this example on the command line by entering

    servingxml -r resources-repeatheader.xml -o output/repeated_header_info.xml hut

### Fixed Width Fields, Multiple Record Types, Repeating Groups, Line Delimited

#### Converting fixed position records with repeating groups to XML (segments)

This shows how to convert fixed position records that contain repeatable/optional fixed parts identified by tags.

The input file is a positional file.

Figure 86. Input flat file segments.txt

    Z01xxxyyyZ02aaaabbccZ02aaaabbccZ01xxxyyy

Note the following:

- The whole record is positional, but contains "segments" that do not have a terminator but a starting tag and a fixed length.
- Z01 and Z02 are "segment"-tags and a,b,x and y are positional fields within the "segment"

The desired output is shown below.

Figure 87. Output XML file segments.xml

    <?xml version="1.0" encoding="utf-8"?>
    <Recs>
      <Rec>
        <Z01>
          <xxx>xxx</xxx>
          <yyy>yyy</yyy>
        </Z01>
        <Z02>
          <aaaa>aaaa</aaaa>
          <bb>bb</bb>
          <cc>cc</cc>
        </Z02>
        <Z02>
          <aaaa>aaaa</aaaa>
          <bb>bb</bb>
          <cc>cc</cc>
        </Z02>
        <Z01>
          <xxx>xxx</xxx>
          <yyy>yyy</yyy>
        </Z01>
      </Rec>
    </Recs>

The following resources script does the transformation.

Figure 88. Resources script resources-segments.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="segments">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="data"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="data">
        <sx:flatFileReader>
          <sx:urlSource url="data/segments.txt"/>
          <sx:flatFile ref="flatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="segmentsToXml"/>
      </sx:recordContent>

      <sx:flatFile id="flatFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="rec">
            <sx:repeatingGroup name="segments">
              <sx:flatRecordTypeChoice>
                <sx:positionalField name="tag" width="3"/>
                <sx:when test="tag='Z01'">
                  <sx:flatRecordType name="Z01">
                    <sx:positionalField name="tag" width="3"/>
                    <sx:positionalField name="xxx" width="3"/>
                    <sx:positionalField name="yyy" width="3"/>
                  </sx:flatRecordType>
                </sx:when>
                <sx:otherwise>
                  <sx:flatRecordType name="Z02">
                    <sx:positionalField name="tag" width="3"/>
                    <sx:positionalField name="aaaa" width="4"/>
                    <sx:positionalField name="bb" width="2"/>
                    <sx:positionalField name="cc" width="2"/>
                  </sx:flatRecordType>
                </sx:otherwise>
              </sx:flatRecordTypeChoice>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="segmentsToXml">
        <Recs>
          <sx:onRecord>
            <Rec>
              <sx:subrecordMapping repeatingGroup="segments">
                <sx:onRecord recordType="Z01">
                  <Z01>
                    <sx:fieldElementMap field="xxx" element="xxx"/>
                    <sx:fieldElementMap field="yyy"  element="yyy"/>
                  </Z01>
                </sx:onRecord>
                <sx:onRecord recordType="Z02">
                  <Z02>
                    <sx:fieldElementMap field="aaaa" element="aaaa"/>
                    <sx:fieldElementMap field="bb" element="bb"/>
                    <sx:fieldElementMap field="cc" element="cc"/>
                  </Z02>
                </sx:onRecord>
              </sx:subrecordMapping>
            </Rec>
          </sx:onRecord>
        </Recs>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -o output/segments.xml -r resources-segments.xml
        segments

#### Mapping multiple repeating groups to XML (students-fix)

Suppose you have the following positional file of students, their course grades, and their addresses.

Figure 89. Input flat file students-fix.txt

    JANEENGLC-MATHA+1972BLUECHICAGOILATLANTAGA

The file has the following layout.

name | 4 characters
subject-grade | repeating group, repeats twice
year-born | 4 characters
favorite-color | 4 characters
address | repeating group, repeats twice

You want the XML output to look as follows.

Figure 90. Output XML file students-fix.xml

    <?xml version="1.0" encoding="utf-8"?>
    <StudentGrades>
      <StudentGrade>
        <Name>JANE</Name>
        <SubjectGrade>
          <Subject>ENGL</Subject>
          <Grade>C-</Grade>
        </SubjectGrade>
        <SubjectGrade>
          <Subject>MATH</Subject>
          <Grade>A+</Grade>
        </SubjectGrade>
        <YearBorn>1972</YearBorn>
        <FavoriteColor>BLUE</FavoriteColor>
        <Address>
          <City>CHICAGO</City>
          <State>IL</State>
        </Address>
        <Address>
          <City>ATLANTA</City>
          <State>GA</State>
        </Address>
      </StudentGrade>
    </StudentGrades>

The required resources script is

Figure 91. Resources script resources-students-fix.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="students">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="students-content"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="students-content">
        <sx:flatFileReader>
          <sx:flatFile ref="students-file"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="students-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="students-file">
        <sx:flatFileBody>
          <sx:flatRecordType name="student">
            <sx:positionalField name="name" width="4"/>
            <sx:repeatingGroup name="grades" count="2">
              <sx:flatRecordType name="subject-grade">
                <sx:positionalField name="subject" width="4"/>
                <sx:positionalField name="grade" width="2"/>
              </sx:flatRecordType>
            </sx:repeatingGroup>
            <sx:positionalField name="year-born" width="4"/>
            <sx:positionalField name="favorite-color" width="4"/>
            <sx:repeatingGroup name="addresses" count="2">
              <sx:flatRecordType name="address">
                <sx:positionalField name="city" width="7"/>
                <sx:positionalField name="state" width="2"/>
              </sx:flatRecordType>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="students-mapping">
        <StudentGrades>
          <sx:onRecord>
            <StudentGrade>
              <sx:fieldElementMap field="name" element="Name"/>
              <sx:subrecordMapping repeatingGroup="grades">
                <sx:onRecord>
                  <SubjectGrade>
                    <sx:fieldElementMap field="subject" element="Subject"/>
                    <sx:fieldElementMap field="grade" element="Grade"/>
                  </SubjectGrade>
                </sx:onRecord>
              </sx:subrecordMapping>
              <sx:fieldElementMap field="year-born" element="YearBorn"/>
              <sx:fieldElementMap field="favorite-color" element="FavoriteColor"/>
              <sx:subrecordMapping repeatingGroup="addresses">
                <sx:onRecord>
                  <Address>
                    <sx:fieldElementMap field="city" element="City"/>
                    <sx:fieldElementMap field="state" element="State"/>
                  </Address>
                </sx:onRecord>
              </sx:subrecordMapping>
            </StudentGrade>
          </sx:onRecord>
        </StudentGrades>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-students-fix.xml -i data/students-fix.txt -o output/students-fix.xml
        students

### Fixed Width Fields, Single Record Type, No End-of-Line Delimiters

#### Converting a flat file with a single record type, but no end-of-line delimiters, to XML (non-delimited book orders)

The example below shows how to convert a flat file with a single record format, but no end-of-line delimiters, to XML.

The input file is a fixed field width flat file.

Figure 92. Input flat file non_delimited_book_orders.txt

    CAuthor                        Title                              Price InvoiceNo InvoiceDate       FCharles Bukowski              Factotum                           22.95 001  12/Mar/2005            FSergei Lukyanenko             The Night Watch                    17.99 002  13/Mar/2005            FAndrew Crumey                 Mr Mee                             22.00 003 14/June/2005            CSteven John Metsker           Building Parsers with Java         39.95 004 15/June/2005            This is a trailer record

The desired output is shown below.

Figure 93. Output XML file book_orders.xml

    <?xml version="1.0" encoding="utf-8"?>
    <book-orders>
      <book-type>
        <category>F</category>
        <author>Charles Bukowski</author>
        <title>Factotum</title>
        <price>22.95</price>
        <invoiceno>001</invoiceno>
        <invoicedate>12/Mar/2005</invoicedate>
        <filler/>
      </book-type>
      <book-type>
        <category>F</category>
        <author>Sergei Lukyanenko</author>
        <title>The Night Watch</title>
        <price>17.99</price>
        <invoiceno>002</invoiceno>
        <invoicedate>13/Mar/2005</invoicedate>
        <filler/>
      </book-type>
      <book-type>
        <category>F</category>
        <author>Andrew Crumey</author>
        <title>Mr Mee</title>
        <price>22.00</price>
        <invoiceno>003</invoiceno>
        <invoicedate>14/June/2005</invoicedate>
        <filler/>
      </book-type>
      <book-type>
        <category>C</category>
        <author>Steven John Metsker</author>
        <title>Building Parsers with Java</title>
        <price>39.95</price>
        <invoiceno>004</invoiceno>
        <invoicedate>15/June/2005</invoicedate>
        <filler/>
      </book-type>
    </book-orders>

The following resources script does the transformation.

Figure 94. Resources script resources-non_delimited_book_orders.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="book-orders">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="book-orders"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="book-orders" name="book-orders">
        <sx:flatFileReader>
          <sx:flatFile ref="books-flatfile"/>
        </sx:flatFileReader>
      </sx:recordContent>

      <sx:flatFile id="books-flatfile" lineDelimited="false">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="book-type"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="book-type"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord width="100">This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="book-type" name="book-type">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
        <sx:positionalField name="invoiceno" width="4" justify="right"/>
        <sx:positionalField name="invoicedate" width="13" justify="right"/>
        <sx:positionalField name="filler" width="12"/>
      </sx:flatRecordType>

    </sx:resources>

Note the following points.

- The sx:recordContent element has a `lineDelimited` attribute set to `false` .

You can run this example on the command line by entering

    servingxml -r resources-non_delimited_book_orders.xml
      -i data/non_delimited_book_orders.txt
      -o output/book_orders.xml  book-orders

### Fixed Width Fields, Multiple Record Types, No End-of-Line Delimiters

#### Converting a flat file with multiple record formats, but no end-of-line delimiters, to XML (non-delimited trades)

The example below shows how to convert a flat file with multiple record formats, but no end-of-line delimiters, to XML.

The input file is a fixed field width flat file.

Figure 95. Input flat file non_delimited_variant_trades.txt

    TR000103/25/2005 1:50:00This is a trade record        TN0002X1234A child transaction           TN0003X1235Another child transaction

The desired output is shown below.

Figure 96. Output XML file trades.xml

    <?xml version="1.0" encoding="utf-8"?>
    <trades>
      <trade id="0001">
        <trade-date>2005-03-25T01:50:00.000-05:00</trade-date>
        <description>This is a trade record</description>
      </trade>
      <transaction id="0002">
        <reference>X1234</reference>
        <description>A child transaction</description>
      </transaction>
      <transaction id="0003">
        <reference>X1235</reference>
        <description>Another child transaction</description>
      </transaction>
    </trades>

The following resources script does the transformation.

Figure 97. Resources script resources-non_delimited_variant_trades.xm

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="trades">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="trades"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="trades">
        <sx:flatFileReader>
          <sx:urlSource url="data/non_delimited_variant_trades.txt"/>
          <sx:flatFile ref="trades-flatfile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="trades-xml-mapping"/>
      </sx:recordContent>

      <sx:flatFile id="trades-flatfile" name="trades" lineDelimited="false">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="record_type" width="2"/>
            <sx:when test="record_type='TR'">
              <sx:flatRecordType name="trade">
                <sx:positionalField name="record_type" width="2"/>
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="trade_date" width="10"/>
                <sx:positionalField name="trade_time" width="8"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="record_type='TN'">
              <sx:flatRecordType name="transaction">
                <sx:positionalField name="record_type" width="2"/>
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="reference" width="5"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="trades-xml-mapping">
        <trades>
          <sx:onRecord recordType="trade">
            <trade>
              <sx:fieldAttributeMap field="id" attribute="id"/>
              <sx:elementMap element="trade-date">
                <sx:convertToDateTime format="MM/dd/yyyy H:mm:ss">
                  <sx:concat separator=" ">
                    <sx:toString value="{trade_date}"/>
                    <sx:toString value="{trade_time}"/>
                  </sx:concat>
                </sx:convertToDateTime>
              </sx:elementMap>
              <sx:fieldElementMap field="description" element="description"/>
            </trade>
          </sx:onRecord>
          <sx:onRecord recordType="transaction">
            <transaction>
              <sx:fieldAttributeMap field="id" attribute="id"/>
              <sx:fieldElementMap field="reference" element="reference"/>
              <sx:fieldElementMap field="description" element="description"/>
            </transaction>
          </sx:onRecord>
        </trades>
      </sx:recordMapping>

    </sx:resources>

Note the following points.

- The sx:recordContent element has a `lineDelimited` attribute set to `false` .

You can run this example on the command line by entering

    servingxml -r resources-non_delimited_variant_trades.xml
      -o output/trades.xml trades

#### Converting a flat file with special characters to XML (special char)

The example below shows how to convert a flat file with special characters to XML.

The input file is a flat file with two adjacent records (no line delimiters):

- A record beginning with the hexadecimal value 03, followed by the text "1111111"

- A record beginning with the hexadecimal value 04, followed by the text "John Smith"

The two hexadecimal values are needed to differentiate the record type, but these valued cannot be written as &#x03; and&#x04; in XML, as these are not valid XML characters.

The desired output is shown below.

Figure 98. Output XML file specialChar.xml

    <?xml version="1.0" encoding="utf-8"?>
    <transaction>
      <R03>
        <firstField>03</firstField>
        <CLIENUM>1111111</CLIENUM>
      </R03>
      <R04>
        <firstField>04</firstField>
        <NAME>John Smith</NAME>
      </R04>
    </transaction>

The following resources script does the transformation.

Figure 99. Resources script resources-specialChar.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="transaction">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="transactionDoc"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="transactionDoc" name="transaction">
        <sx:flatFileReader>
          <sx:flatFile ref="transactionFile"/>
        </sx:flatFileReader>
      </sx:recordContent>

      <sx:flatFile id="transactionFile" lineDelimited="false">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:binaryField name="firstField" width="1"/>
            <sx:when test="firstField='03'">
              <sx:flatRecordType id="R03" name="R03">
                <sx:binaryField name="firstField" label="firstField" width="1" />
                <sx:positionalField name="CLIENUM" label="CLIENUM" width="007" />
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="firstField='04'">
              <sx:flatRecordType id="R04" name="R04">
                <sx:binaryField name="firstField" label="firstField" width="1" />
                <sx:positionalField name="NAME" label="NAME" width="020" />
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>
    </sx:resources>

Note that the first field of each record is declared as an sx:binaryField . The string rendering of this field is as an hexidecimal value.

You can run this example on the command line by entering

    servingxml  -r resources-specialChar.xml -i data/specialChar.txt
      -o output/specialChar.xml transaction

#### Converting a flat file containing Cobol packed decimal data to XML (packed decimal to XML)

The example below shows how to convert a non-delimited flat file containing Cobol packed decimal data to XML.

The input file, shown below, is encoded in ebcdic, except for the price field, which is expressed as a Cobol packed decimal number `99999999.99` .

Figure 100. Input positional flat file books.txt

    CAuthor                        Title                              Price

    FCharles Bukowski              Factotum                           22.95
    FSergei Lukyanenko             The Night Watch                    17.99
    FAndrew Crumey                 Mr Mee                             22.00
    CSteven John Metsker           Building Parsers with Java         39.95

    This is a trailer record

The desired output is shown below.

Figure 101. Output XML file books.xml

    <?xml version="1.0" encoding="utf-8"?>
    <myns:books xmlns:myns="http://mycompany.com/mynames/"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="url2">
      <myns:book categoryCode="F">
        <myns:title>Factotum</myns:title>
        <myns:author>Charles Bukowski</myns:author>
        <myns:price>22.95</myns:price>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>The Night Watch</myns:title>
        <myns:author>Sergei Lukyanenko</myns:author>
        <myns:price>17.99</myns:price>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>Mr Mee</myns:title>
        <myns:author>Andrew Crumey</myns:author>
        <myns:price>22.00</myns:price>
      </myns:book>
      <myns:book categoryCode="C">
        <myns:title>Building Parsers with Java</myns:title>
        <myns:author>Steven John Metsker</myns:author>
        <myns:price>39.95</myns:price>
      </myns:book>
    </myns:books>

The following resources script does the transformation.

Figure 102. Resources script resources-books_ebcdic_packed.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="books-ebcdic-packed2xml">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="books"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <!--
      This sx:flatFileReader element does not specify a stream source, so
      the source will default to the file specified with the -i option on the command line.
      -->
      <sx:recordContent id="books">
        <sx:flatFileReader>
          <sx:flatFile ref="booksFile"/>
          <sx:defaultStreamSource encoding="Cp1047"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="booksToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="booksFile" lineDelimited="false">
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:packedDecimalField name="price" label="Price" digitCount="10" decimalPlaces="2"/>
      </sx:flatRecordType>

      <sx:recordMapping id="booksToXmlMapping">
        <myns:books xmlns:myns="http://mycompany.com/mynames/"
                         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                         xsi:schemaLocation="url2">
          <sx:onRecord>
            <myns:book>
              <sx:fieldAttributeMap field="category" attribute="categoryCode"/>
              <sx:fieldElementMap field="title" element="myns:title"/>
              <sx:fieldElementMap field="author" element="myns:author"/>
              <sx:fieldElementMap field="price" element="myns:price"/>
            </myns:book>
          </sx:onRecord>
        </myns:books>
      </sx:recordMapping>

    </sx:resources>

Note that the last field of each record is declared as an sx:packedDecimalField .

You can run this example on the command line by entering

    servingxml  -r resources-books_ebcdic_packed.xml
      -i data/books_ebcdic_packed.dat
      -o  output/books.xml    books-ebcdic-packed2xml

### Logical Record Consists of Multiple Physical Records

#### Converting a System Management Facilities (SMF) file to XML (Segment Concatenation)

The example below shows how to convert a System Management Facilities (SMF) file to XML. The example includes support for variable block spanned (VBS) records. The format of an SMF file is described in SMF records.

The desired output is shown below.

Figure 103. Output XML file smf.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <smf>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>2</type>
       </smfRecord>
       <smfRecord>
          <systemIndicator>DE</systemIndicator>
          <type>110</type>
       </smfRecord>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>64</type>
          <jobName>SMF</jobName>
       </smfRecord>
       <smfRecord>
          <systemIndicator>5E</systemIndicator>
          <type>42</type>
          <subType>6</subType>
       </smfRecord>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>64</type>
          <jobName>GPL1</jobName>
       </smfRecord>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>64</type>
          <jobName>GPL1</jobName>
       </smfRecord>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>64</type>
          <jobName>GPL1</jobName>
       </smfRecord>
       <smfRecord>
          <systemIndicator>1E</systemIndicator>
          <type>3</type>
       </smfRecord>
    </smf>

The following resources script does the transformation.

Figure 104. Resources script resources-smf.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:xs="http://www.w3.org/2001/XMLSchema"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="smf-xml">
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:content ref="smf"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="smf">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="smfFile"/>
            <sx:defaultStreamSource encoding="Cp1047"/>
          </sx:flatFileReader>
        </sx:recordStream>
        <sx:recordMapping ref="smfToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="smfFile" lineDelimited="false" countPositionsInBytes="true">
        <sx:flatFileBody>
          <sx:flatRecordType ref="smfType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:vbsFlatRecordType id="smfType">
        <sx:integerField name="recordLength" width="2"/>
        <sx:integerField name="segmentControlCode" width="1"/>
        <sx:integerField name="reserved" width="1"/>
        <sx:segmentConcatenation startTest = "number(sx:current//segmentControlCode) = 0
                                              or number(sx:current//segmentControlCode) = 1"
                                 segmentLength="{recordLength}">
          <sx:flatRecordTypeChoice>
            <sx:integerField name="recordLength" width="2"/>
            <sx:integerField name="segmentControlCode" width="1"/>
            <sx:integerField name="reserved" width="1"/>
            <sx:binaryField name="systemIndicator" width="1"/>
            <sx:integerField name="recordType" width="1"/>
            <sx:when test="recordType=64">
              <sx:flatRecordType name="smfType64">
                <sx:integerField name="recordLength" width="2"/>
                <sx:integerField name="segmentControlCode" width="1"/>
                <sx:integerField name="reserved" width="1"/>
                <sx:binaryField name="systemIndicator" label="systemIndicator" width="1"/>
                <sx:integerField name="recordType" label="recordType" width="1"/>
                <sx:positionalField name="jobName" label="Job Name" start="19" width="8"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="recordType=42">
              <sx:flatRecordType name="smfType42">
                <sx:integerField name="recordLength" width="2"/>
                <sx:integerField name="segmentControlCode" width="1"/>
                <sx:integerField name="reserved" width="1"/>
                <sx:binaryField name="systemIndicator" label="systemIndicator" width="1"/>
                <sx:integerField name="recordType" label="recordType" width="1"/>
                <sx:integerField name="subType" label="Sub Type" start="23" width="2"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="recordType=110">
              <sx:flatRecordType name="smfType110">
                <sx:integerField name="recordLength" width="2"/>
                <sx:integerField name="segmentControlCode" width="1"/>
                <sx:integerField name="reserved" width="1"/>
                <sx:binaryField name="systemIndicator" label="systemIndicator" width="1"/>
                <sx:integerField name="recordType" label="recordType" width="1"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:otherwise>
              <sx:flatRecordType name="other">
                <sx:integerField name="recordLength" width="2"/>
                <sx:integerField name="segmentControlCode" width="1"/>
                <sx:integerField name="reserved" width="1"/>
                <sx:binaryField name="systemIndicator" label="systemIndicator" width="1"/>
                <sx:integerField name="recordType" label="recordType" width="1"/>
              </sx:flatRecordType>
            </sx:otherwise>
          </sx:flatRecordTypeChoice>
        </sx:segmentConcatenation>
      </sx:vbsFlatRecordType>

      <sx:recordMapping id="smfToXmlMapping">
        <smf>
          <sx:onRecord recordType="smfType64">
            <smfRecord>
              <sx:fieldElementMap field="systemIndicator" element="systemIndicator"/>
              <sx:fieldElementMap field="recordType" element="type"/>
              <sx:fieldElementMap field="jobName" element="jobName"/>
            </smfRecord>
          </sx:onRecord>
          <sx:onRecord recordType="smfType42">
            <smfRecord>
              <sx:fieldElementMap field="systemIndicator" element="systemIndicator"/>
              <sx:fieldElementMap field="recordType" element="type"/>
              <sx:fieldElementMap field="subType" element="subType"/>
            </smfRecord>
          </sx:onRecord>
          <sx:onRecord recordType="smfType110">
            <smfRecord>
              <sx:fieldElementMap field="systemIndicator" element="systemIndicator"/>
              <sx:fieldElementMap field="recordType" element="type"/>
            </smfRecord>
          </sx:onRecord>
          <sx:onRecord recordType="other">
            <smfRecord>
              <sx:fieldElementMap field="systemIndicator" element="systemIndicator"/>
              <sx:fieldElementMap field="recordType" element="type"/>
            </smfRecord>
          </sx:onRecord>
        </smf>
      </sx:recordMapping>

    </sx:resources>

##### Remarks

- The field definitions that appear as children of sx:vbsFlatRecordType comprise the 4-byte segment descriptor word (SDW), which describes the segment. The first 2 bytes contain the length of the segment, including the 4-byte SDW. The third byte of the SDW contains the segment control code, a value of `00` indicates that the segment is a complete logical record, and `01` indicates that the segment is the first segment of a multisegment record. For more details, see segment descriptor word.

- The sx:segmentConcatenation element controls the composition of the mulitple segments into a single logical record. Note that a `segmentControlCode` of either `00` or `01` marks the beginning of a logical record. By default, the first segment descriptor word goes at the front of the logical record, in the place of a record descriptor word (RDW), subsequent segment descriptor words are discarded. If the optional `suppressRDW` attribute is set to "true", the first segment descriptor word is discarded as well, the SDW fields must be removed from the content of sx:segmentConcatenation, and four bytes need to be subtracted from any positions specified in the record definitions.

- Note that ServingXML positions are one-based, while the offsets in the IBM documentation are zero-based.

You can run this example on the command line by entering

    servingxml -r resources-smf.xml -i data/G6744V00.smf -o output/smf.xml smf-xml

#### Aggregating Multiple Physical Records into a Composite Record (Record Composition)

The input file is a fixed field width flat file.

Figure 105. Input flat file composite-records.txt

    ODANIEL    356996699001
    A233ServingXML132433SAS
    B343443DD2322ARDE021101
    CDANIEL    029299839907
    B344233DD2322PREE021100
    D0000040000002300000001
    A233ServingXML132433AGS
    B3434432323000022021101
    CDANIEL    029299839907
    B3433132323000022021101
    CDANIEL    029299839907
    D0000040000002300000002
    E0030000000100009803600

Note the following properties of this data:

- The file begins with an "O" record and ends with an "E" record.
- An "A" record begins a group of records.
- The last three character field of an A record (e.g. "SAS") determines a variant format for a B record.

The desired output is shown below.

Figure 106. Output XML file composite-records.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <document>
      <header>
        <O>
          <recordType>O</recordType>
          <name>DANIEL</name>
          <id>356</id>
          <code>996699</code>
          <tracking>001</tracking>
        </O>
      </header>
      <body>
        <A>
          <recordType>A</recordType>
          <code>233</code>
          <book>ServingXML</book>
          <isbn>132433</isbn>
          <identifier>SAS</identifier>
        </A>
        <B-SAS>
          <recordType>B</recordType>
          <library>343443</library>
          <shelf>DD2322ARDE</shelf>
          <ebook>021</ebook>
          <code>101</code>
        </B-SAS>
        <C>
          <recordType>C</recordType>
          <author>DANIEL</author>
          <publisher>0292998399</publisher>
          <flag>07</flag>
        </C>
        <B-SAS>
          <recordType>B</recordType>
          <library>344233</library>
          <shelf>DD2322PREE</shelf>
          <ebook>021</ebook>
          <code>100</code>
        </B-SAS>
        <D>
          <recordType>D</recordType>
          <count>000004</count>
          <amount>0000002300</amount>
          <tracking>000001</tracking>
        </D>
        <A>
          <recordType>A</recordType>
          <code>233</code>
          <book>ServingXML</book>
          <isbn>132433</isbn>
          <identifier>AGS</identifier>
        </A>
        <B-AGS>
          <recordType>B</recordType>
          <store>34344323</store>
          <cost>23000022</cost>
          <code>021</code>
          <make>101</make>
        </B-AGS>
        <C>
          <recordType>C</recordType>
          <author>DANIEL</author>
          <publisher>0292998399</publisher>
          <flag>07</flag>
        </C>
        <B-AGS>
          <recordType>B</recordType>
          <store>34331323</store>
          <cost>23000022</cost>
          <code>021</code>
          <make>101</make>
        </B-AGS>
        <C>
          <recordType>C</recordType>
          <author>DANIEL</author>
          <publisher>0292998399</publisher>
          <flag>07</flag>
        </C>
        <D>
          <recordType>D</recordType>
          <count>000004</count>
          <amount>0000002300</amount>
          <tracking>000002</tracking>
        </D>
      </body>
      <trailer>
        <E>
          <recordType>E</recordType>
          <total>00300</total>
          <amount>0000010000</amount>
          <count>9803</count>
          <tracking>600</tracking>
        </E>
      </trailer>
    </document>

The following resources script does the transformation.

Figure 107. Resources script resources-recordComposition.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="composite-records-to-xml">
        <sx:task ref="serialized-composite-records"/>
      </sx:service>

      <sx:serialize id="serialized-composite-records">
        <sx:transform>
          <sx:recordContent ref="composite-record-content"/>
        </sx:transform>
      </sx:serialize>

      <sx:recordContent id="composite-record-content">
        <sx:flatFileReader ref="composite-record-reader"/>
        <sx:recordMapping ref="composite-record-mapping"/>
      </sx:recordContent>

      <sx:flatFileReader id="composite-record-reader">
        <sx:flatFile>
          <sx:flatFileBody>
            <sx:flatRecordTypeChoice>
              <sx:positionalField name="recordType" width="1"/>
              <sx:when test="recordType='O'">
                <sx:flatRecordType ref="O"/>
              </sx:when>
              <sx:when test="recordType='E'">
                <sx:flatRecordType ref="E"/>
              </sx:when>
              <sx:otherwise>
                <sx:compositeFlatRecordType>
                  <sx:positionalField name="recordType" width="1"/>
                  <sx:positionalField name="code" width="3"/>
                  <sx:positionalField name="book" width="10"/>
                  <sx:positionalField name="isbn" width="6"/>
                  <sx:positionalField name="identifier" width="3"/>
                  <sx:recordComposition recordType="composite" repeatingGroup="group"
                                        startTest = "sx:current//recordType = 'A'
                                                     and sx:current//identifier = 'SAS'"
                                        endTest = "sx:current//recordType = 'A'">
                    <sx:flatRecordTypeChoice>
                      <sx:positionalField name="recordType" width="1"/>
                      <sx:when test="recordType='A'">
                        <sx:flatRecordType ref="A"/>
                      </sx:when>
                      <sx:when test="recordType='B'">
                        <sx:flatRecordType ref="B-SAS"/>
                      </sx:when>
                      <sx:when test="recordType='C'">
                        <sx:flatRecordType ref="C"/>
                      </sx:when>
                      <sx:when test="recordType='D'">
                        <sx:flatRecordType ref="D"/>
                      </sx:when>
                    </sx:flatRecordTypeChoice>
                  </sx:recordComposition>
                  <sx:recordComposition recordType="composite" repeatingGroup="group"
                                        startTest = "sx:current//recordType = 'A'
                                                     and sx:current//identifier = 'AGS'"
                                        endTest = "sx:current//recordType = 'A'">
                    <sx:flatRecordTypeChoice>
                      <sx:positionalField name="recordType" width="1"/>
                      <sx:when test="recordType='A'">
                        <sx:flatRecordType ref="A"/>
                      </sx:when>
                      <sx:when test="recordType='B'">
                        <sx:flatRecordType ref="B-AGS"/>
                      </sx:when>
                      <sx:when test="recordType='C'">
                        <sx:flatRecordType ref="C"/>
                      </sx:when>
                      <sx:when test="recordType='D'">
                        <sx:flatRecordType ref="D"/>
                      </sx:when>
                    </sx:flatRecordTypeChoice>
                  </sx:recordComposition>
                </sx:compositeFlatRecordType>
              </sx:otherwise>
            </sx:flatRecordTypeChoice>
          </sx:flatFileBody>
        </sx:flatFile>
      </sx:flatFileReader>

      <sx:flatRecordType id="A" name="A">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="code" width="3"/>
        <sx:positionalField name="book" width="10"/>
        <sx:positionalField name="isbn" width="6"/>
        <sx:positionalField name="identifier" width="3"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="C" name="C">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="author" width="10"/>
        <sx:positionalField name="publisher" width="10"/>
        <sx:positionalField name="flag" width="2"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="D" name="D">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="count" width="6"/>
        <sx:positionalField name="amount" width="10"/>
        <sx:positionalField name="tracking" width="6"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="E" name="E">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="total" width="5"/>
        <sx:positionalField name="amount" width="10"/>
        <sx:positionalField name="count" width="4"/>
        <sx:positionalField name="tracking" width="3"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="B-SAS" name="B-SAS">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="library" width="6"/>
        <sx:positionalField name="shelf" width="10"/>
        <sx:positionalField name="ebook" width="3"/>
        <sx:positionalField name="code" width="3"/>
      </sx:flatRecordType>
      <sx:flatRecordType id="B-AGS" name="B-AGS">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="store" width="8"/>
        <sx:positionalField name="cost" width="8"/>
        <sx:positionalField name="code" width="3"/>
        <sx:positionalField name="make" width="3"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="O" name="O">
        <sx:positionalField name="recordType" width="1"/>
        <sx:positionalField name="name" width="10"/>
        <sx:positionalField name="id" width="3"/>
        <sx:positionalField name="code" width="6"/>
        <sx:positionalField name="tracking" width="3"/>
      </sx:flatRecordType>

      <sx:recordMapping id="composite-record-mapping">
        <document>
          <header>
            <sx:onRecord recordType="O">
              <O>
                <sx:defaultFieldElementMap fields="*"/>
              </O>
            </sx:onRecord>
          </header>
          <body>
            <sx:onRecord recordType="composite">
              <sx:subrecordMapping repeatingGroup="group">
                <sx:onRecord recordType="A">
                  <A>
                    <sx:defaultFieldElementMap fields="*"/>
                  </A>
                </sx:onRecord>
                <sx:onRecord recordType="B-SAS">
                  <B-SAS>
                    <sx:defaultFieldElementMap fields="*"/>
                  </B-SAS>
                </sx:onRecord>
                <sx:onRecord recordType="B-AGS">
                  <B-AGS>
                    <sx:defaultFieldElementMap fields="*"/>
                  </B-AGS>
                </sx:onRecord>
                <sx:onRecord recordType="C">
                  <C>
                    <sx:defaultFieldElementMap fields="*"/>
                  </C>
                </sx:onRecord>
                <sx:onRecord recordType="D">
                  <D>
                    <sx:defaultFieldElementMap fields="*"/>
                  </D>
                </sx:onRecord>
                <sx:onRecord recordType="O">
                  <O>
                    <sx:defaultFieldElementMap fields="*"/>
                  </O>
                </sx:onRecord>
              </sx:subrecordMapping>
            </sx:onRecord>
          </body>
          <trailer>
            <sx:onRecord recordType="E">
              <E>
                <sx:defaultFieldElementMap fields="*"/>
              </E>
            </sx:onRecord>
          </trailer>
        </document>
      </sx:recordMapping>

    </sx:resources>

##### Remarks

- The field definitions that appear as children of sx:compositeFlatRecordType belong to the header of a group of related records.

- The sx:recordComposition element controls the aggregation of a group of physical records into a composite logical record.

You can run this example on the command line by entering

    servingxml -r resources-recordComposition.xml -i data/composite-records.txt
        -o output/composite-records.xml composite-records-to-xml

#### Computing an Aggregate Value over Multiple Records

This example shows a resources script that will concatenate the values of several records into a single XML element value. It illustrates the use of the sx:composeRecord element.

The input file is

Figure 108. Input flat file orders.txt

    HEADER
    ORDER/CUS/CON/USD/
    PO1/PAR1/MFR FADD1/5/
    REM AAAAA
    REM BBBBB
    REM CCCCC
    PO2/PAR2/MFR 30066/1/
    PO3/PAR3/MFR PAGT2/2/
    REM CCCCC
    REM DDDDD
    TRAILER

The desired output is

Figure 109. Output XML file order.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Order>
      <OrderHeader>
        <CUS>CUS</CUS>
        <CON>CON</CON>
        <CUR>USD</CUR>
      </OrderHeader>
      <OrderPosition>
        <ORN>PO1</ORN>
        <PAR>PAR1</PAR>
        <QTY>5</QTY>
        <MFR>FADD1</MFR>
        <REM>AAAAA BBBBB CCCCC</REM>
      </OrderPosition>
      <OrderPosition>
        <ORN>PO2</ORN>
        <PAR>PAR2</PAR>
        <QTY>1</QTY>
        <MFR>30066</MFR>
      </OrderPosition>
      <OrderPosition>
        <ORN>PO3</ORN>
        <PAR>PAR3</PAR>
        <QTY>2</QTY>
        <MFR>PAGT2</MFR>
        <REM>CCCCC DDDDD</REM>
      </OrderPosition>
    </Order>

Note that the REM values are concatenated in a space separated list.

The following resources script does the transformation.

Figure 110. Resources script resources-order.xml

    <?xml version="1.0"?>
    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="Order">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="Order"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="Order" name="Order">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="OrderFile"/>
          </sx:flatFileReader>
          <sx:composeRecord recordType="composite" repeatingGroup="detail"
                            startTest="sx:current/orderRemarks"
                            endTest="not(sx:current/orderRemarks)">
            <sx:newField name="remarks" select="detail/orderRemarks/REM"/>
          </sx:composeRecord>
        </sx:recordStream>
        <sx:recordMapping ref="OrderToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="OrderFile">
        <sx:commentStarter value="#"/>
        <sx:fieldDelimiter value="/"/>
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="tag" width="3"/>
            <sx:when test="tag='ORD'">
              <sx:flatRecordType name="orderHeader">
                <sx:delimitedField name="CMD" label="Command Code"/>
                <sx:delimitedField name="CUS" label="Customer"/>
                <sx:delimitedField name="CON" label="Contractor"/>
                <sx:delimitedField name="CUR" label="Currency Code"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='REM'">
              <sx:flatRecordType name="orderRemarks">
                <sx:positionalField name="REMTEI" label="Remark" width="3"/>
                <sx:delimitedField name="REM" label="Remark"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='TRA'">
              <sx:flatRecordType name="orderTrailer">
              </sx:flatRecordType>
            </sx:when>
            <sx:otherwise>
              <sx:flatRecordType name="orderPosition">
                <sx:delimitedField name="ORN" label="Order Number"/>
                <sx:delimitedField name="PAR" label="Part Number"/>
                <sx:delimitedField name="MFRTEI" maxWidth="3"/>
                <sx:delimitedField name="MFR" label="Manufacturer"/>
                <sx:delimitedField name="QTY" label="Order Quantity"/>
              </sx:flatRecordType>
            </sx:otherwise>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="OrderToXmlMapping">
        <Order>
          <sx:innerGroup startTest="sx:current/orderHeader">
            <OrderHeader>
              <sx:fieldElementMap field="CUS" element="CUS"/>
              <sx:fieldElementMap field="CON" element="CON"/>
              <sx:fieldElementMap field="CUR" element="CUR"/>
            </OrderHeader>

            <sx:innerGroup startTest="sx:current/orderPosition">
              <OrderPosition>
                <sx:fieldElementMap field="ORN" element="ORN" minOccurs="1"/>
                <sx:fieldElementMap field="PAR" element="PAR"/>
                <sx:fieldElementMap field="QTY" element="QTY"/>
                <sx:fieldElementMap field="MFR" element="MFR" minOccurs="0"/>
                <sx:onRecord recordType="composite">
                  <sx:choose>
                    <sx:when test="remarks != ''">
                      <sx:fieldElementMap select="remarks" element="REM"/>
                    </sx:when>
                  </sx:choose>
                </sx:onRecord>
              </OrderPosition>
            </sx:innerGroup>
          </sx:innerGroup>
        </Order>
      </sx:recordMapping>
    </sx:resources>

The effect of sx:composeRecord is to collect all the adjacent `orderRemarks` records, where they become sub-records belonging to a field called "detail", and create one record of type "composite" that has the repeating group field "detail", plus another field "remarks" that is computed from "detail".

The `orderRemarks` records flowing into sx:composeRecord look like, in their XML representation,

    <orderRemarks><REMTEI>REM</REMTEI><REM>AAAA</REM></orderRemarks>
    <orderRemarks><REMTEI>REM</REMTEI><REM>BBBB</REM></orderRemarks>
    <orderRemarks><REMTEI>REM</REMTEI><REM>CCCC</REM></orderRemarks>

The composite record flowing out looks like

    <composite>
      <detail>
        <orderRemarks><REMTEI>REM</REMTEI><REM>AAAA</REM></orderRemarks>
        <orderRemarks><REMTEI>REM</REMTEI><REM>BBBB</REM></orderRemarks>
        <orderRemarks><REMTEI>REM</REMTEI><REM>CCCC</REM></orderRemarks>
      </detail>
      <remarks>AAAA BBBB CCCC</remarks>
    </composite>

You can run this example on the command line by entering

    servingxml -i data/order.txt -r resources-order.xml -o output/order.xml Order

#### Mapping Two Records at the Top of a File to a Header Element in the XML Output

This example shows a resources script that maps two records at the top of a file to a header element in the XML output. To map fields from multiple records to the attributes of a single element, you first need to compose an aggregate of the two records with the sx:composeRecord instruction. You can then select against the composite record with an sx:fieldAttributeMap and populate the attributes of the parent element.

The input file is

Figure 111. Input flat file files.txt

    DP1 458293048kd02
    *****2LV 2008070132 EX9
    E0000015498273
    F0000014938294
    G000001E7839201
    D0000024839305

The rules for mapping the third through last records are as follows.

- If a "D" record is found, start a new `file` element, but "D" records are optional.

- If an "E" record is found, start a new `file` element, unless one has already been started because of a "D" record.

The desired output is

Figure 112. Output XML file files.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <output date="20080701" sequence="32"
            code="EX9" reference="458293048kd0">
      <file filenumber="000001"/>
      <file filenumber="000002"/>
    </output>

The following resources script does the transformation.

Figure 113. Resources script resources-files.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="files">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="filesContent"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="filesContent">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="fileFlatFile"/>
          </sx:flatFileReader>
          <sx:composeRecord recordType="header" repeatingGroup="headerRecords"
                            startTest="sx:current/header1" endTest="sx:previous/header2">
          </sx:composeRecord>
        </sx:recordStream>
        <sx:recordMapping ref="fileToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="fileFlatFile">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="tag" width="1"/>
            <sx:positionalField name="tag2" width="2"/>
            <sx:when test="tag='D' and tag2='P1'">
              <sx:flatRecordType name="header1">
                <sx:positionalField name="reference" start="5" width="13"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='*'">
              <sx:flatRecordType name="header2">
                <sx:positionalField name="date" start="10" width="8"/>
                <sx:positionalField name="sequence" width="2"/>
                <sx:positionalField name="code" start="21" width="3"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='E'">
              <sx:flatRecordType name="E">
                <sx:positionalField name="filenumber" start="2" width="6"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='D'">
              <sx:flatRecordType name="D">
                <sx:positionalField name="filenumber" start="2" width="6"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="fileToXmlMapping">
        <output>
          <sx:fieldAttributeMap select="headerRecords/header2/date" attribute="date"/>
          <sx:fieldAttributeMap select="headerRecords/header2/sequence" attribute="sequence"/>
          <sx:fieldAttributeMap select="headerRecords/header2/code" attribute="code"/>
          <sx:fieldAttributeMap select="headerRecords/header1/reference" attribute="reference"/>
          <sx:groupChoice>
            <sx:innerGroup startTest="sx:current/D" endTest="sx:current/D">
              <file>
                <sx:fieldAttributeMap field="filenumber" attribute="filenumber"/>
              </file>
            </sx:innerGroup>
            <sx:innerGroup startTest="sx:current/E" endTest="sx:current/D or sx:current/E">
              <file>
                <sx:fieldAttributeMap field="filenumber" attribute="filenumber"/>
              </file>
            </sx:innerGroup>
          </sx:groupChoice>
        </output>
      </sx:recordMapping>

    </sx:resources>

The effect of sx:composeRecord is to collect the top two header records, `header1` and `header2` , where they become sub-records belonging to a field called "headerRecords", and create one record of type "header" that has the repeating group field "headerRecords".

The `header` record has the following XML representation,

    <header>
      <headerRecords>
        <header1>
          <reference>458293048kd0</reference>
        </header1>
        <header2>
          <date>20080701</date>
          <sequence>32</sequence>
          <code>EX9</code>
        </header2>
      </headerRecords>
    </header>

You can run this example on the command line by entering

    servingxml -i data/files.txt -r resources-files.xml -o output/files.xml files

### Start/End Delimited Records.

#### Converting flat files with start/end record delimiters (opdef)

The example below shows how to convert a flat file with start/end record delimiters.

The input file is shown below.

Figure 114. Input file opdef.txt

    1 20050505 121205 xtyzzya
    {:HJSED
    :20:886452311980706
    :23:CREDDFED
    :32:030612USD5443,99
    :33:USD5443,99
    :50K:GIAN
    NNPLLS
    :52A:B0
    :53A:B33
    :54A:I3N
    :57A:BRE
    :59:/20041010050500001M02606
    KKL S.A
    GNRBBL
    :70:/FF/INV 559661
    :71:SAH
    -}
    3 20050505 121205 bahthy

This file has a number of features.

- The first and last lines are header and footers, delimited by new line symbols.

- The middle part must be taken as a whole chunk, starting from the opening `{` and ending at the closing `}` .

The desired output is shown below.

Figure 115. Output XML file opdef.xml

    <?xml version="1.0" encoding="utf-8"?>
    <opdef>
      <start>
        <opdate>20050505</opdate>
        <optime>121205</optime>
        <optarget>xtyzzya</optarget>
      </start>
    :HJSED
    :20:886452311980706
    :23:CREDDFED
    :32:030612USD5443,99
    :33:USD5443,99
    :50K:GIAN
    NNPLLS
    :52A:B0
    :53A:B33
    :54A:I3N
    :57A:BRE
    :59:/20041010050500001M02606
    KKL S.A
    GNRBBL
    :70:/FF/INV 559661
    :71:SAH
    -
      <end>
        <tpdate>20050505</tpdate>
        <tptime>121205</tptime>
        <tptarget>bahthy</tptarget>
      </end>
    </opdef>

The following resources script does the transformation.

Figure 116. Resources script resources-opdef.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="opdef">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="opdef"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="opdef">
        <sx:flatFileReader>
          <sx:flatFile ref="opdefFlatFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="opdefRecordMapping"/>
      </sx:recordContent>

      <sx:flatFile id="opdefFlatFile" name="opdef">
        <sx:recordDelimiter value="\r\n"/>
        <sx:recordDelimiter value="\n"/>
        <sx:recordDelimiter start="{" end="}"/>
        <sx:flatFileBody trim="true">
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="tag" width="1"/>
            <sx:when test="tag='1'">
              <sx:flatRecordType name="start">
                <sx:positionalField name="tag" width="2"/>
                <sx:positionalField name="opdate" width="9"/>
                <sx:positionalField name="optime" width="7"/>
                <sx:positionalField name="optarget" width="7"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='3'">
              <sx:flatRecordType name="end">
                <sx:positionalField name="tag" width="2"/>
                <sx:positionalField name="tpdate" width="9"/>
                <sx:positionalField name="tptime" width="7"/>
                <sx:positionalField name="tptarget" width="7"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:otherwise>
              <sx:flatRecordType name="data">
                <sx:delimitedField name="field1"/>
              </sx:flatRecordType>
            </sx:otherwise>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="opdefRecordMapping">
        <opdef>
          <sx:onRecord recordType="start">
            <start>
              <sx:defaultFieldElementMap fields="opdate optime optarget"/>
            </start>
          </sx:onRecord>
          <sx:onRecord recordType="end">
            <end>
              <sx:defaultFieldElementMap fields="tpdate tptime tptarget"/>
            </end>
          </sx:onRecord>
          <sx:onRecord recordType="data">
            <sx:cdata>
              <sx:toString value="{field1}"/>
            </sx:cdata>
          </sx:onRecord>
        </opdef>
      </sx:recordMapping>

    </sx:resources>

Note the following points about this script.

- There are three record delimiters, which are checked sequentially. The third record delimiter specifies both start and end values. When the start value is found, the matching end value is looked for, taking account of nesting, and the whole returned as one record. The chunk returned may contain occurances of the other two delimiters.

- The data is mapped to a single field by using a sx:delimitedField element, taking the default field delimiter, which is the end of the record. The data is enclosed in a `CDATA` section, using the sx:cdata element.

You can run this example on the command line by entering

    servingxml -i data/opdef.txt -r resources-opdef.xml
      -o output/opdef.xml opdef

#### Converting flat files with nested start/end record and segment delimiters and name/value pairs (transaction)

The example below shows how to convert a flat file with nested start/end record and segment delimiters and name/value pairs.

The input file is shown below.

Figure 117. Input file transaction.txt

    {TRAN:trantype=PRIN#origin=Toronto#
    {TRANREF:reftype=TREF#ref=1234567890123456B#}
    {TRANREF:reftype=FOTR#ref=1234567890123456#}
    {DRIVER:drivertype=OPER#drivercode=SELL#}
    {DRIVER:drivertype=STRM#drivercode=FOP#}
    {INSTR:instrtype=PINS#instrreftype=INT#instrref=PTEQTY1#qty=1000000#}
    {INSTR:instrtype=TCCY#instrreftype=ISO#instrref=USD#}
    {INSTR:instrtype=SINS#instrreftype=ISO#instrref=USD#}
    {PARTY:partytype=COMP#partyreftype=X3#partyref=CMP1#}
    {PARTY:partytype=SECP#partyreftype=X3#partyref=PTSECP1#}
    {DATE:datetype=TDAT#date=01-Jan-2007#}
    {DATE:datetype=VDAT#date=01-Jan-2007#}
    {CHARGE:chargetype=COMM#calctype=NONE#chargeamount=50.00#instrreftype=ISO#instrref=USD}
    {PRICE:ratetype=TPRC#price=4.9#multdiv=X1#pricetype=YP#}
    }

This file has a number of features.

- The entire record is contained inside "{" - "}" record delimiters.

- The first field in the record is delimited by a ":" and identifies the type of the record.

- The record contains a number of name/value pairs, with names and values separated by the character "=", and fields terminated by the character "#".

- The record contains a number of segments enclosed in "{" - "}" segment delimiters.

- The first field in each segment is delimited by a ":" and identifies the type of the segment.

- Each segment contains a number of name/value pairs, with names and values separated by the character "=", and fields terminated by the character "#".

The desired output is shown below.

Figure 118. Output XML file transaction.xml

    <?xml version="1.0" encoding="utf-8"?>
    <transaction>
      <TRAN>
        <record-type>TRAN</record-type>
        <trantype>PRIN</trantype>
        <origin>Toronto</origin>
        <segments>
          <TRANREF>
            <record-type>TRANREF</record-type>
            <reftype>TREF</reftype>
            <ref>1234567890123456B</ref>
          </TRANREF>
          <TRANREF>
            <record-type>TRANREF</record-type>
            <reftype>FOTR</reftype>
            <ref>1234567890123456</ref>
          </TRANREF>
          <DRIVER>
            <record-type>DRIVER</record-type>
            <drivertype>OPER</drivertype>
            <drivercode>SELL</drivercode>
          </DRIVER>
          <DRIVER>
            <record-type>DRIVER</record-type>
            <drivertype>STRM</drivertype>
            <drivercode>FOP</drivercode>
          </DRIVER>
          <INSTR>
            <record-type>INSTR</record-type>
            <instrtype>PINS</instrtype>
            <instrreftype>INT</instrreftype>
            <instrref>PTEQTY1</instrref>
            <qty>1000000</qty>
          </INSTR>
          <INSTR>
            <record-type>INSTR</record-type>
            <instrtype>TCCY</instrtype>
            <instrreftype>ISO</instrreftype>
            <instrref>USD</instrref>
          </INSTR>
          <INSTR>
            <record-type>INSTR</record-type>
            <instrtype>SINS</instrtype>
            <instrreftype>ISO</instrreftype>
            <instrref>USD</instrref>
          </INSTR>
          <PARTY>
            <record-type>PARTY</record-type>
            <partytype>COMP</partytype>
            <partyreftype>X3</partyreftype>
            <partyref>CMP1</partyref>
          </PARTY>
          <PARTY>
            <record-type>PARTY</record-type>
            <partytype>SECP</partytype>
            <partyreftype>X3</partyreftype>
            <partyref>PTSECP1</partyref>
          </PARTY>
          <DATE>
            <record-type>DATE</record-type>
            <datetype>TDAT</datetype>
            <date>01-Jan-2007</date>
          </DATE>
          <DATE>
            <record-type>DATE</record-type>
            <datetype>VDAT</datetype>
            <date>01-Jan-2007</date>
          </DATE>
          <CHARGE>
            <record-type>CHARGE</record-type>
            <chargetype>COMM</chargetype>
            <calctype>NONE</calctype>
            <chargeamount>50.00</chargeamount>
            <instrreftype>ISO</instrreftype>
            <instrref>USD</instrref>
          </CHARGE>
          <PRICE>
            <record-type>PRICE</record-type>
            <ratetype>TPRC</ratetype>
            <price>4.9</price>
            <multdiv>X1</multdiv>
            <pricetype>YP</pricetype>
          </PRICE>
        </segments>
      </TRAN>
    </transaction>

The following resources script does the transformation.

Figure 119. Resources script resources-transaction.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="transaction">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="transaction-content"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="transaction-content" name="transaction">
        <sx:flatFileReader>
          <sx:flatFile ref="transaction-flat-file"/>
        </sx:flatFileReader>
      </sx:recordContent>

      <sx:flatFile id="transaction-flat-file">
        <sx:recordDelimiter start="{" end="}"/>
        <sx:flatFileBody>
          <sx:repeatDelimiter start="{" end="}"/>
          <sx:nameDelimiter value="="/>
          <sx:flatRecordType name="TRAN">
            <sx:delimitedField name="record-type">
              <sx:fieldDelimiter value=":"/>
            </sx:delimitedField>
            <sx:repeatingField>
              <sx:fieldDelimiter value="#"/>
              <sx:segmentDelimiter value="{"/>
              <sx:delimitedNamedField/>
            </sx:repeatingField>
            <sx:repeatingGroup name="segments">
              <sx:flatRecordTypeChoice>
                <sx:delimitedField name="record-type">
                  <sx:fieldDelimiter value=":"/>
                </sx:delimitedField>
                <sx:when test="1=1">
                  <sx:flatRecordType name="{record-type}">
                    <sx:delimitedField name="record-type">
                      <sx:fieldDelimiter value=":"/>
                    </sx:delimitedField>
                    <sx:repeatingField>
                      <sx:delimitedNamedField/>
                      <sx:fieldDelimiter value="#"/>
                    </sx:repeatingField>
                  </sx:flatRecordType>
                </sx:when>
              </sx:flatRecordTypeChoice>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

Note the following points about this script.

- The value of the first field in the segment, terminated by ":", is assigned as the record type name of the segment.

- All the name/value pairs within a segment are read with a single sx:repeatingField element.

You can run this example on the command line by entering

    servingxml -i data/transaction.txt -r resources-transaction.xml
      -o output/transaction.xml transaction

### Custom Record Reader

#### Converting the output of a custom record reader to XML (custom record reader)

The example below shows how to convert the the output of a custom record reader to XML.

The input records are generated by the following Java class.

Figure 120. TradeRecordReader Java class

    package flat2xml;

    import com.servingxml.app.ServiceContext;
    import com.servingxml.components.recordio.AbstractRecordReader;
    import com.servingxml.app.Flow;
    import com.servingxml.util.Name;
    import com.servingxml.util.QualifiedName;
    import com.servingxml.util.ServingXmlException;
    import com.servingxml.util.record.Record;
    import com.servingxml.util.record.RecordBuilder;

    public class TradeRecordReader extends AbstractRecordReader {
      private static final Name TRADE_RECORD_TYPE = new QualifiedName("trade");
      private static final Name TRANSACTION_RECORD_TYPE = new QualifiedName("transaction");
      private static final Name RECORD_TYPE_NAME = new QualifiedName("record_type");
      private static final Name ID_NAME = new QualifiedName("id");
      private static final Name TRADE_DATE_NAME = new QualifiedName("trade_date");
      private static final Name TRADE_TIME_NAME = new QualifiedName("trade_time");
      private static final Name DESCRIPTION_NAME = new QualifiedName("description");
      private static final Name REFERENCE_NAME = new QualifiedName("reference");

      public void readRecords(ServiceContext context, Flow flow)
       {

        //  Start the record stream
        startRecordStream(context, flow);

        RecordBuilder trRecordBuilder = new RecordBuilder(TRADE_RECORD_TYPE);
        RecordBuilder tnRecordBuilder = new RecordBuilder(TRANSACTION_RECORD_TYPE);

        Record record;

        trRecordBuilder.setString(RECORD_TYPE_NAME,"TR");
        trRecordBuilder.setString(ID_NAME,"0001");
        trRecordBuilder.setString(TRADE_DATE_NAME,"03/25/2005");
        trRecordBuilder.setString(TRADE_TIME_NAME,"01:50:00");
        trRecordBuilder.setString(DESCRIPTION_NAME,"This is a trade record");
        record = trRecordBuilder.toRecord();
        trRecordBuilder.clear();

        //  Write the "trade" record
        writeRecord(context, flow, record);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0002");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1234");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"A child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the first "transaction" record
        writeRecord(context, flow, record);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0003");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1235");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"Another child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the second "transaction" record
        writeRecord(context, flow, record);

        //  End the record stream
        endRecordStream(context, flow);
      }
    }

The `TradeRecordReader` class will generate the following stream of records.

record_type | id | trade_date | trade_time | description
--- | --- | --- | --- | ---
TR | 0001 | 03/25/2005 | 1:50:00 | This is a trade record

record_type | id | reference | description
--- | --- | --- | ---
TN | 0002 | X1234 | A child transaction
TN | 0003 | X1235 | Another child transaction

The desired output is the same as for the trades example.

Figure 121. Output XML file trades.xml

    <?xml version="1.0" encoding="utf-8"?>
    <trades feed="LONDON">
      <trade id="0001">
        <trade-date>2005-03-25T01:50:00.000-05:00</trade-date>
        <description>This is a trade record</description>
      </trade>
      <transaction id="0002">
        <reference>X1234</reference>
        <description>A child transaction</description>
      </transaction>
      <transaction id="0003">
        <reference>X1235</reference>
        <description>Another child transaction</description>
      </transaction>
    </trades>

The following resources script does the transformation.

Figure 122. Resources script resources-custom_trade_reader.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="trades">
        <sx:parameter name="feed">
          <sx:defaultValue>NY</sx:defaultValue>
        </sx:parameter>
        <sx:serialize>
          <sx:transform>
            <sx:content ref="trades"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="trades">
        <sx:customRecordReader class="flat2xml.TradeRecordReader"/>
        <sx:recordMapping ref="tradesToXmlMapping"/>
      </sx:recordContent>

      <sx:recordMapping id="tradesToXmlMapping">
        <trades>
          <sx:fieldAttributeMap value="{$feed}" attribute="feed"/>
          <sx:onRecord recordType="trade">
            <trade>
              <sx:fieldAttributeMap field="id" attribute="id"/>
              <sx:elementMap element="trade-date">
                <sx:convertToDateTime format="MM/dd/yyyy H:mm:ss">
                  <sx:concat separator=" ">
                    <sx:toString value="{trade_date}"/>
                    <sx:toString value="{trade_time}"/>
                  </sx:concat>
                </sx:convertToDateTime>
              </sx:elementMap>
              <sx:fieldElementMap field="description" element="description"/>
            </trade>
          </sx:onRecord>
          <sx:onRecord recordType="transaction">
            <transaction>
              <sx:fieldAttributeMap field="id" attribute="id"/>
              <sx:fieldElementMap field="reference" element="reference"/>
              <sx:fieldElementMap field="description" element="description"/>
            </transaction>
          </sx:onRecord>
        </trades>
      </sx:recordMapping>

    </sx:resources>

Note the following points.

- The record content is defined by a custom record reader, implemented in the Java class `TradeRecordReader` .

You can run this example on the command line by

- Compiling the Java class `TradeRecordReader` and copying the resulting `.class` file into the `dir/classes` directory
- Entering the command

      servingxml -r resources-custom_trade_reader.xml
        -o output/trades.xml trades feed=LONDON

### Batching XML Output

#### Serializing XML in Batches

This example shows how to serialize an XML document in batches. It illustrates the use of the sx:batchedSerializer element.

Suppose you want to read the CSV file countries.csv and serialize it to XML, but you also want the following:

- Country subtrees to be split over multiple documents, no more than 50 to any one file.

                06/12/2007  07:37 PM    <DIR>          .
                06/12/2007  07:37 PM    <DIR>          ..
                06/12/2007  07:37 PM             3,773 countries-1.xml
                06/12/2007  07:37 PM             3,758 countries-2.xml
                06/12/2007  07:37 PM             3,747 countries-3.xml
                06/12/2007  07:37 PM             3,753 countries-4.xml
                06/12/2007  07:37 PM             3,416 countries-5.xml

- Each document to begin with all the XML nodes up to the occurance of the first country subtree.

The following resources script produces the desired output.

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="countries">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="countries"/>
          </sx:transform>
          <sx:batchedSerializer path="country" batchSize="50">
            <sx:xsltSerializer>
              <sx:fileSink file="output/countries-{$sx:batchSequenceNumber}.xml"/>
            </sx:xsltSerializer>
          </sx:batchedSerializer>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="countries" name="countries">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:urlSource url="data/countries.csv"/>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileReader>
          <msv:recordValidator>
            <sx:urlSource url="data/country-record.xsd"/>
          </msv:recordValidator>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
          </sx:discardHandler>
        </sx:recordStream>
        <sx:recordMapping ref="countriesToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileHeader lineCount="1"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="country">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="code"/>
            <sx:delimitedField name="name" trimLeading="true"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="countriesToXmlMapping">
        <countries>
          <sx:onRecord>
            <country>
              <sx:fieldElementMap field="name" element="countryName"/>
              <sx:fieldAttributeMap field="code" attribute="countryCode"/>
            </country>
          </sx:onRecord>
        </countries>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-batches.xml countries
      rootName=countries

#### Writing Records to Batch Files, and Converting the Batches to Separate XML Files

The example illustrates an alternative way of producing XML in batches, in two steps:

- First, split a large flat file into a number of smaller flat files.

- Second, convert each of the smaller flat files to XML.

Suppose you want to convert the countries.csv file containing 245 country records to XML, each country record mapped to one country subtree, and the output spilt over multiple XML files.

The first step is to produce the batched csv files and output them to an intermediate directory.

Figure 123. Batched intermediate CSV files.

    06/12/2007  07:24 PM    <DIR>          .
    06/12/2007  07:24 PM    <DIR>          ..
    06/12/2007  07:24 PM               814 countries-1.csv
    06/12/2007  07:24 PM               795 countries-2.csv
    06/12/2007  07:24 PM               786 countries-3.csv
    06/12/2007  07:24 PM               784 countries-4.csv
    06/12/2007  07:24 PM               793 countries-5.csv

The second step is to read all the csv files in the intermediate directory and serialize them to XML.

Figure 124. Batched XML files.

    06/12/2007  07:37 PM    <DIR>          .
    06/12/2007  07:37 PM    <DIR>          ..
    06/12/2007  07:37 PM             3,773 countries-1.xml
    06/12/2007  07:37 PM             3,758 countries-2.xml
    06/12/2007  07:37 PM             3,747 countries-3.xml
    06/12/2007  07:37 PM             3,753 countries-4.xml
    06/12/2007  07:37 PM             3,416 countries-5.xml

The following resources script performs both steps.

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="countries">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:fileSource file="data/{$rootName}.csv"/>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileReader>
          <sx:batchedRecordWriter batchSize="50">
            <sx:flatFileWriter>
              <sx:fileSink file="intermediate/{$rootName}-{$sx:batchSequenceNumber}.csv"/>
              <sx:flatFile ref="countriesFile"/>
            </sx:flatFileWriter>
          </sx:batchedRecordWriter>
        </sx:recordStream>

        <sx:recordStream>
          <sx:directoryReader directory="intermediate">
            <sx:fileFilter pattern="countries.*[.]csv"/>
          </sx:directoryReader>
          <sx:processRecord>
            <sx:parameter name="output-file">
              <sx:findAndReplace searchFor ="(countries.*)[.]csv" replaceWith ="$1.xml">
                <sx:toString value="{name}"/>
              </sx:findAndReplace>
            </sx:parameter>
            <sx:serialize>
              <sx:xsltSerializer>
                <sx:fileSink directory="output" file="{$output-file}"/>
              </sx:xsltSerializer>
              <sx:transform>
                <sx:content ref="countries"/>
              </sx:transform>
            </sx:serialize>
          </sx:processRecord>
        </sx:recordStream>
      </sx:service>

      <sx:recordContent id="countries">
        <sx:flatFileReader>
          <sx:fileSource directory="{parentDirectory}" file="{name}"/>
          <sx:flatFile ref="countriesFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="countriesToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="country">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="code"/>
            <sx:delimitedField name="name" trimLeading="true"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="countriesToXmlMapping">
        <countries>
          <sx:onRecord>
            <country>
              <sx:fieldElementMap field="name" element="countryName"/>
              <sx:fieldAttributeMap field="code" attribute="countryCode"/>
            </country>
          </sx:onRecord>
        </countries>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-batchRecords.xml countries
      rootName=countries

### Many XML Documents

#### Flat File to Summary and Detail XML Documents (eobclaims)

This example requires reading the flat file

      ADM2301EOBS
      HDR06/13/08200815501300101MEMBER01          BISHOP
      ADR1234 TEST DR                                                                                       ARVADA
      HDR06/13/08200815501300102MEMBER02          BISHOP
      ADR5678 TEST DR                                                                                       ARVADA
      TLRTOTAL EOBS GENERATED: 0000001000000000000010000018

and producing this output:

- One summary XML document capturing the header ("ADM") and trailer ("TLR") records

                  eobinputoutput.xml

- Separate XML documents for each pair of "HDR" and "ADR" records

                  eobinputoutput-061308200815501300101.xml
                  eobinputoutput-061308200815501300102.xml

The summary XML document is to look like

      <?xml version="1.0" encoding="UTF-8"?>
      <eobs>
         <ADMIN_RECTYPE record_type="ADM">
            <ADM_0001_TEXT>2301EOBS</ADM_0001_TEXT>
         </ADMIN_RECTYPE>
         <TLR-REC-TYPE record_type="TLR">
            <TLR_0001_TEXT>TOTAL EOBS GENERATED: 0</TLR_0001_TEXT>
            <TLR_0001_COUNT>0000010</TLR_0001_COUNT>
            <FILLER>00000000000010000018</FILLER>
         </TLR-REC-TYPE>
      </eobs>

and the first detail documents as

    <?xml version="1.0" encoding="UTF-8"?>
    <eob>
       <HEADER_RECTYPE record_type="HDR">
          <HDR_STMT_DATE>06/13/08</HDR_STMT_DATE>
          <HDR_CLAIM_NUMBER>200815501300101</HDR_CLAIM_NUMBER>
          <HDR_PATIENT_NAME_FIRST>MEMBER01</HDR_PATIENT_NAME_FIRST>
          <HDR_PATIENT_NAME_MIDDLE/>
          <HDR_PATIENT_NAME_LAST>BISHOP</HDR_PATIENT_NAME_LAST>
       </HEADER_RECTYPE>
       <ADR_REC_TYPE record_type="ADR">
          <ADR_8512_ATTN>1234 TEST DR</ADR_8512_ATTN>
          <ADR_8513_STREET1/>
          <ADR_8513_STREET2/>
          <ADR_8514_CITY>ARVADA</ADR_8514_CITY>
       </ADR_REC_TYPE>
    </eob>

Below is the required resources script. Since the structure of the record input will be shared with another example, we'll collect that in a separate script and include it in the main one.

Figure 125. Record structure resource-eobclaims-flatfile.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:flatFile id="eobclaims-file">
        <sx:flatFileBody>
          <sx:flatRecordType ref="eobclaims-record"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordTypeChoice id="eobclaims-record">
        <sx:positionalField name="record_type" width="3"/>
        <sx:when test="record_type='ADM'">
          <sx:flatRecordType ref="adm"/>
        </sx:when>
        <sx:when test="record_type='HDR'">
          <sx:flatRecordType ref="hdr"/>
        </sx:when>
        <sx:when test="record_type='ADR'">
          <sx:flatRecordType ref="adr"/>
        </sx:when>
        <sx:when test="record_type='TLR'">
          <sx:flatRecordType ref="tlr"/>
        </sx:when>
      </sx:flatRecordTypeChoice>

      <sx:flatRecordType id="adm" name="ADM">
        <sx:positionalField name="record_type" width="3"/>
        <sx:positionalField name="ADM_0001_TEXT" width="20"/>
        <sx:positionalField name="ADM_0001_RUN_DATE" width="249"/>
        <sx:positionalField name="FILLER" width="8"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="tlr" name="TLR">
        <sx:positionalField name="record_type" width="3"/>
        <sx:positionalField name="TLR_0001_TEXT" width="23"/>
        <sx:positionalField name="TLR_0001_COUNT" width="7"/>
        <sx:positionalField name="FILLER" width="268"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="hdr" name="HDR">
        <sx:positionalField name="record_type" width="3"/>
        <sx:positionalField name="HDR_STMT_DATE" width="8"/>
        <sx:positionalField name="HDR_CLAIM_NUMBER" width="15"/>
        <sx:positionalField name="HDR_PATIENT_NAME_FIRST" width="18"/>
        <sx:positionalField name="HDR_PATIENT_NAME_LAST" width="15"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="adr" name="ADR">
        <sx:positionalField name="record_type" width="3"/>
        <sx:positionalField name="ADR_8512_ATTN" width="99"/>
        <sx:positionalField name="ADR_8514_CITY" start="103" width="18"/>
      </sx:flatRecordType>

    </sx:resources>

Figure 126. Main resources script resource-eobclaims.xml

    <?xml version="1.0"?>

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:include href="resources-eobclaims-flatfile.xml"/>

      <sx:service id="eobclaims-to-xml">
        <!-- Do the header/trailer first -->
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:fileSink directory="output" file="eobinputoutput.xml"/>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:recordContent>
              <sx:flatFileReader>
                <sx:flatFile ref="eobclaims-file"/>
              </sx:flatFileReader>
              <sx:recordMapping ref="eobclaims-header-trailer-mapping"/>
            </sx:recordContent>
          </sx:transform>
        </sx:serialize>

        <!-- Now do the body -->
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="eobclaims-file"/>
          </sx:flatFileReader>
          <!-- Compose a composite record out of each HDR-ADR pair -->
          <sx:composeRecord recordType="composite" repeatingGroup="eob"
                      startTest="sx:current/HDR"
                      endTest="sx:previous/ADR">
            <sx:newField name="HDR_STMT_DATE" select="eob/HDR/HDR_STMT_DATE"/>
            <sx:newField name="HDR_CLAIM_NUMBER" select="eob/HDR/HDR_CLAIM_NUMBER"/>
          </sx:composeRecord>
          <!-- Discard the records that are not HDR-ADR pairs -->
          <sx:restrictRecordFilter>
            <sx:recordRestriction recordType="composite"/>
          </sx:restrictRecordFilter>
          <!-- Process each HDR-ADR composite record -->
          <sx:processRecord>
            <!-- Construct and store the output filename in the parameter "output-file" -->
            <sx:parameter name="output-file">
              <sx:findAndReplace searchFor="/" replaceWith ="">
                <sx:toString value="eobinputoutput-{HDR_STMT_DATE}{HDR_CLAIM_NUMBER}.xml"/>
              </sx:findAndReplace>
            </sx:parameter>

            <sx:serialize>
              <sx:xsltSerializer>
                <sx:fileSink directory="output" file="{$output-file}"/>
                <sx:outputProperty name="indent" value="yes"/>
              </sx:xsltSerializer>
              <sx:transform>
                <sx:recordContent>
                  <sx:recordMapping ref="eobclaims-body-mapping"/>
                </sx:recordContent>
              </sx:transform>
            </sx:serialize>
          </sx:processRecord>
        </sx:recordStream>
      </sx:service>

      <sx:recordMapping id="eobclaims-header-trailer-mapping">
        <eobs>
          <sx:onRecord recordType="ADM">
            <ADMIN_RECTYPE record_type="ADM">
              <sx:fieldElementMap field="ADM_0001_TEXT" element="ADM_0001_TEXT"/>
            </ADMIN_RECTYPE>
          </sx:onRecord>
          <sx:onRecord recordType="TLR">
            <TLR-REC-TYPE record_type="TLR">
              <sx:fieldElementMap field="TLR_0001_TEXT" element="TLR_0001_TEXT"/>
              <sx:fieldElementMap field="TLR_0001_COUNT" element="TLR_0001_COUNT"/>
              <sx:fieldElementMap field="FILLER" element="FILLER"/>
            </TLR-REC-TYPE>
          </sx:onRecord>
        </eobs>
      </sx:recordMapping>

      <sx:recordMapping id="eobclaims-body-mapping">
        <eob>
          <!-- Our composite record has a repeating group field, "eob",
               that has two sub-records, "HDR" and "ADR" -->
          <sx:onRecord recordType="composite">
            <sx:subrecordMapping repeatingGroup="eob">
              <sx:onRecord recordType="HDR">
                <HEADER_RECTYPE>
                  <sx:fieldAttributeMap field="record_type" attribute="record_type"/>
                  <sx:fieldElementMap field="HDR_STMT_DATE" element="HDR_STMT_DATE"/>
                  <sx:fieldElementMap field="HDR_CLAIM_NUMBER" element="HDR_CLAIM_NUMBER"/>
                  <sx:fieldElementMap field="HDR_PATIENT_NAME_FIRST" element="HDR_PATIENT_NAME_FIRST"/>
                  <sx:fieldElementMap field="HDR_PATIENT_NAME_MIDDLE" element="HDR_PATIENT_NAME_MIDDLE"/>
                  <sx:fieldElementMap field="HDR_PATIENT_NAME_LAST" element="HDR_PATIENT_NAME_LAST"/>
                </HEADER_RECTYPE>
              </sx:onRecord>
              <sx:onRecord recordType="ADR">
                <ADR_REC_TYPE>
                  <sx:fieldAttributeMap field="record_type" attribute="record_type"/>
                  <sx:fieldElementMap field="ADR_8512_ATTN" element="ADR_8512_ATTN"/>
                  <sx:fieldElementMap field="ADR_8513_STREET1" element="ADR_8513_STREET1"/>
                  <sx:fieldElementMap field="ADR_8513_STREET2" element="ADR_8513_STREET2"/>
                  <sx:fieldElementMap field="ADR_8514_CITY" element="ADR_8514_CITY"/>
                </ADR_REC_TYPE>
              </sx:onRecord>
            </sx:subrecordMapping>
          </sx:onRecord>
        </eob>
      </sx:recordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -i data/eobclaims.txt -r resources-eobclaims.xml eobclaims-to-xml

### Many Flat Input Files

#### Multiple XML Readers

This example shows a resources script that maps a primary input file to XML, with lookups from a secondary file.

The primary input file is the pipe delimited file, `primary-input.txt`.

    HDR|a|b
    DET|a|b
    DET|a|b
    MSG|1|a
    MSG|3|a
    END

The secondary input file is the pipe delimited file, `messages.txt`,

    1|test message 1
    2|test message 2
    3|test message 3

You need to read the primary input file and replace the `id` tag values under message with the message corresponding to the id. The expected output is

    <?xml version="1.0" encoding="UTF-8"?>
    <mytest>
      <field1>a</field1>
      <field2>b</field2>
      <detail>
        <field1>a</field1>
        <field2>b</field2>
      </detail>
      <detail>
        <field1>a</field1>
        <field2>b</field2>
      </detail>
      <message>
        <message>test message 1</message>
        <field2>a</field2>
      </message>
      <message>
        <message>test message 3</message>
        <field2>a</field2>
      </message>
    </mytest>

The required resources script is

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="multiple-reader-test">
        <sx:serialize>
          <sx:xsltSerializer>
            <sx:outputProperty name="indent" value="yes"/>
          </sx:xsltSerializer>
          <sx:transform>
            <sx:content ref="multiple-reader-content"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="multiple-reader-content">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="primary-file"/>
          </sx:flatFileReader>
        </sx:recordStream>
        <sx:recordMapping ref="primary-file-to-xml"/>
      </sx:recordContent>

      <sx:flatFile id="primary-file">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:fieldDelimiter value="|"/>
            <sx:delimitedField name="tag"/>
            <sx:when test="tag='HDR'">
              <sx:flatRecordType name="header">
                <sx:delimitedField name="tag"/>
                <sx:delimitedField name="field1"/>
                <sx:delimitedField name="field2"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='DET'">
              <sx:flatRecordType name="detail">
                <sx:delimitedField name="tag"/>
                <sx:delimitedField name="field1"/>
                <sx:delimitedField name="field2"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="tag='MSG'">
              <sx:flatRecordType name="message">
                <sx:delimitedField name="tag"/>
                <sx:delimitedField name="field1"/>
                <sx:delimitedField name="field2"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:recordMapping id="primary-file-to-xml">
        <mytest>
          <sx:onRecord recordType="header">
            <sx:fieldElementMap field="field1" element="field1"/>
            <sx:fieldElementMap field="field2" element="field2"/>
          </sx:onRecord>
          <sx:onRecord recordType="detail">
            <detail>
              <sx:fieldElementMap field="field1" element="field1"/>
              <sx:fieldElementMap field="field2" element="field2"/>
            </detail>
          </sx:onRecord>
          <sx:onRecord recordType="message">
            <message>
              <sx:parameter name="my-id" value="{field1}"/>
              <sx:fieldElementMap select="document('messageContent')/messages/message[id=$my-id]/description" element="message"/>
              <sx:fieldElementMap field="field2" element="field2"/>
            </message>
          </sx:onRecord>
        </mytest>
      </sx:recordMapping>

      <sx:recordContent id="messageContent" name="messages">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:urlSource url="messages.txt"/>
            <sx:flatFile ref="messagesFile"/>
          </sx:flatFileReader>
        </sx:recordStream>
      </sx:recordContent>

      <sx:flatFile id="messagesFile" name="messages">
        <sx:flatFileBody>
          <sx:fieldDelimiter value="|"/>
          <sx:flatRecordType name="message">
            <sx:delimitedField name="id"/>
            <sx:delimitedField name="description"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -i primary-input.txt -r resources.xml -o output.xml multiple-reader-test

## XML to Flat File

### Converting an XML document to a positional flat file with record count in trailer (books-positional)

The example below illustrates how to prepare a resources script that will convert an XML document into a positional flat file, and include the record count in a trailer section.

The input file is an XML file.

Figure 127. Input XML file books.xml

    <myns:books xmlns:myns="http://mycompany.com/mynames/">
      <myns:book categoryCode="F">
        <myns:title>Kafka on the Shore</myns:title>
        <myns:author>Haruki Murakami</myns:author>
        <myns:price>25.17</myns:price>
      </myns:book>
      <myns:book categoryCode="C">
        <myns:title>Concurrent Programming in Java</myns:title>
        <myns:author>Doug Lea</myns:author>
        <myns:price>77.13</myns:price>
      </myns:book>
    </myns:books>

The desired output is shown below.

Figure 128. Output positional flat file books.txt

    CAuthor                        Title                              Price

    FHaruki Murakami               Kafka on the Shore                 25.17
    CDoug Lea                      Concurrent Programming in Java     77.13

    Number of records: 2

The following resources script does the transformation.

Figure 129. Resources script resources-books.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="books2pos">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="booksToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:flatRecordType ref="trailer"/>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="trailer" name="trailer">
        <sx:positionalField name="recordCountLabel" width="19" label="Number of records: "/>
        <sx:positionalField name="recordCount"  width="10" label="{$sx:recordCount}"/>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="booksToFileMapping">
        <sx:onSubtree path="/myns:books/myns:book">
          <sx:flattenSubtree recordType="book">
            <sx:subtreeFieldMap select="myns:title" field="title"/>
            <sx:subtreeFieldMap select="@categoryCode" field="category"/>
            <sx:subtreeFieldMap select="myns:author" field="author"/>
            <sx:subtreeFieldMap select="myns:price" field="price"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>
    </sx:resources>

Note the following:

- In header and trailer sections, the `label` attributes of fields are treated as data values.

- In a sx:positionalField element, the value of the `label` attribute can contain references to parameters, in particular to `$sx:recordCount`, which is a parameter that the file record writer adds after finishing writing records.

You can run this example on the command line by entering

    servingxml -r resources-books2pos.xml -i data/books.xml
        -o output/books.txt books2pos

### Converting an XML document with multiple occurances of an element to a CSV file (books-csv)

This example shows one way to prepare a resources script that will convert an XML document having multiple occurances of an element into a CSV file.

The input file is an XML file with multiple occurances of the element `myns:review`.

Figure 130. Input XML file books.xml

    <myns:books xmlns:myns="http://mycompany.com/mynames/">
      <myns:book categoryCode="F">
        <myns:title>Factotum</myns:title>
        <myns:author>Charles Bukowski</myns:author>
        <myns:price/>
        <myns:reviews>
          <myns:review>*****</myns:review>
          <myns:review>*</myns:review>
        </myns:reviews>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>Kafka on the Shore</myns:title>
        <myns:author>Haruki Murakami</myns:author>
        <myns:price>25.17</myns:price>
        <myns:reviews>
          <myns:review>*****</myns:review>
          <myns:review>****</myns:review>
        </myns:reviews>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>Mr Mee</myns:title>
        <myns:author>Andrew Crumey</myns:author>
        <myns:price>22.00</myns:price>
        <myns:reviews>
          <myns:review>*****</myns:review>
          <myns:review>*</myns:review>
        </myns:reviews>
      </myns:book>
      <myns:book categoryCode="C">
        <myns:title>Concurrent Programming in Java</myns:title>
        <myns:author>Doug Lea</myns:author>
        <myns:price>77.13</myns:price>
        <myns:reviews>
          <myns:review>****</myns:review>
        </myns:reviews>
      </myns:book>
      <myns:book categoryCode="B">
        <myns:title>Sex, Lies, and Headlocks</myns:title>
        <myns:author>Shaun Assael</myns:author>
        <myns:price>7.24 </myns:price>
        <myns:reviews>
          <myns:review>****</myns:review>
        </myns:reviews>
      </myns:book>
    </myns:books>

The desired output is shown below.

Figure 131. Output CSV file books.csv

    Category,Author,Title,Price,Review 1,Review 2
    F,Charles Bukowski,Factotum,,*****,*
    F,Haruki Murakami,Kafka on the Shore,25.17,*****,****
    F,Andrew Crumey,Mr Mee,22.00,*****,*
    C,Doug Lea,Concurrent Programming in Java,77.13,****,
    B,Shaun Assael,"Sex, Lies, and Headlocks",7.24 ,****,

Note that, because it contains embedded commas, the title of the last book must be enclosed in quote marks.

The following resources script does the transformation.

Figure 132. Resources script resources-books2csv.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="books2csv">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="booksToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:fieldDelimiter value=","/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="title" label="Title"/>
        <sx:delimitedField name="price" label="Price"/>
        <sx:delimitedField name="review1" label="Review 1"/>
        <sx:delimitedField name="review2" label="Review 2"/>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="booksToFileMapping">
        <sx:onSubtree path="/myns:books/myns:book">
          <sx:flattenSubtree recordType="book">
            <sx:subtreeFieldMap select="myns:title" field="title"/>
            <sx:subtreeFieldMap select="@categoryCode" field="category"/>
            <sx:subtreeFieldMap select="myns:author" field="author"/>
            <sx:subtreeFieldMap select="myns:price" field="price"/>
            <sx:subtreeFieldMap select="myns:reviews/myns:review[1]" field="review1"/>
            <sx:subtreeFieldMap select="myns:reviews/myns:review[2]" field="review2"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -o output/persons.xml -r resources-persons.xml
        personsAddresses

### Converting an XML document with multiple occurances of an element to a flat file with subfield delimiters (books-pipe)

This example shows another way of writing a resources script that will convert an XML document having multiple occurances of an element into a delimited flat file.

The XML input file is the same as the previous example. The desired output, though, is slightly different.

Figure 133. Output pipe delimited flat file books-pipe.txt

    Category|Author|Title|Price|review
    Fiction|Charles Bukowski|Factotum||*****;*
    Fiction|Haruki Murakami|Kafka on the Shore|25.17|*****;****
    Fiction|Andrew Crumey|Mr Mee|22.00|*****;*
    Computers|Doug Lea|Concurrent Programming in Java|77.13|****
    Biography|Shaun Assael|Sex, Lies, and Headlocks|7.24 |****

Note the following features of this output.

- Fields are separated by the pipe delimiter.

- Ratings appear in a multivalued field separated by semi-colon subfield delimiters.

- "Fiction" has been substituted for "F", "Computers" for "C", etc.

This time the resources script looks like this.

Figure 134. Resources script resources-books2pipe.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="books2pipe">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="booksToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>

          <sx:choose>
            <sx:when test="category='F'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Fiction"/>
              </sx:modifyRecord>
            </sx:when>
            <sx:when test="category='C'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Computers"/>
              </sx:modifyRecord>
            </sx:when>
            <sx:when test="category='B'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Biography"/>
              </sx:modifyRecord>
            </sx:when>
          </sx:choose>

          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="title" label="Title"/>
        <sx:delimitedField name="price" label="Price"/>
        <sx:delimitedField name="review" label="review">
          <sx:subfieldDelimiter value=";"/>
        </sx:delimitedField>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="booksToFileMapping">
        <sx:onSubtree path="/myns:books/myns:book">
          <sx:flattenSubtree recordType="book">
            <sx:subtreeFieldMap select="myns:title" field="title"/>
            <sx:subtreeFieldMap select="@categoryCode" field="category"/>
            <sx:subtreeFieldMap select="myns:author" field="author"/>
            <sx:subtreeFieldMap select="myns:price" field="price"/>
            <sx:subtreeFieldMap match="myns:reviews/myns:review" select="text()" field="review"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-books2pipe.xml -i data/books.xml
        -o output/books-pipe.txt books2pipe

### Converting each book entry in books.xml to a list of three records (book-listing)

This example shows how to write a resources script that will convert each book entry in the `books.xml` document into a list of three records, the first comma separated, and the last two space separated.

The XML input file is the same as the previous example.

The desired output is follows.

Figure 135. Output flat file book_lisiting.txt

    "Factotum", Charles Bukowski
    Category Fiction
    Price $
    "Kafka on the Shore", Haruki Murakami
    Category Fiction
    Price $25.17
    "Mr Mee", Andrew Crumey
    Category Fiction
    Price $22.00
    "Concurrent Programming in Java", Doug Lea
    Category Computers
    Price $77.13
    "Sex, Lies, and Headlocks", Shaun Assael
    Category Biography
    Price $7.24

Note the following features of this output.

- The categoryCode 'F' is mapped to "Fiction", 'C' to "Computers", 'B' to "Biography".

- A dollar sign '$' is prepended to the price (even if the price is empty.)

- Each book title is enclosed in quotation marks.

- Each book subtree is output as three records; the first delimited by ", ", the second delimited by a space character.

This time the resources script looks like this.

Figure 136. Resources script resources-book_listing.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="book-listing">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="booksToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>

          <!-- Convert category code to label -->
          <sx:choose>
            <sx:when test="category='F'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Fiction"/>
              </sx:modifyRecord>
            </sx:when>
            <sx:when test="category='C'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Computers"/>
              </sx:modifyRecord>
            </sx:when>
            <sx:when test="category='B'">
              <sx:modifyRecord>
                <sx:newField name="category" value="Biography"/>
              </sx:modifyRecord>
            </sx:when>
          </sx:choose>

          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordTypeChoice id="bookType">
        <sx:delimitedField name="xxx"/>  <!-- unused here, used for input only -->
        <sx:when test="/book">  <!-- matches against record type "book" -->
          <sx:flatRecordType name="book">
            <sx:fieldDelimiter value=", "/>
            <sx:delimitedField name="title" quote="always"/>
            <sx:delimitedField name="author"/>
          </sx:flatRecordType>
        </sx:when>
        <sx:when test="/bookCategory">  <!-- matches against record type "bookCategory" -->
          <sx:flatRecordType name="book">
            <sx:fieldDelimiter value=" "/>
            <sx:delimitedField name="label"/>
            <sx:delimitedField name="category"/>
          </sx:flatRecordType>
        </sx:when>
        <sx:when test="/bookPrice">  <!-- matches against record type "bookPrice" -->
          <sx:flatRecordType name="book">
            <sx:fieldDelimiter value=" "/>
            <sx:delimitedField name="label"/>
            <sx:delimitedField name="price"/>
          </sx:flatRecordType>
        </sx:when>
      </sx:flatRecordTypeChoice>

      <sx:inverseRecordMapping id="booksToFileMapping">
        <sx:onSubtree path="/myns:books/myns:book">
          <sx:flattenSubtree recordType="book">
            <sx:subtreeFieldMap select="myns:title" field="title"/>
            <sx:subtreeFieldMap select="myns:author" field="author"/>
          </sx:flattenSubtree>
          <sx:flattenSubtree recordType="bookCategory">
            <sx:subtreeFieldMap select="'Category'" field="label"/>
            <sx:subtreeFieldMap select="@categoryCode" field="category"/>
          </sx:flattenSubtree>
          <sx:flattenSubtree recordType="bookPrice">
            <sx:subtreeFieldMap select="'Price'" field="label"/>
            <!-- Append dollar sign to price -->
            <sx:subtreeFieldMap select="concat('$',myns:price)" field="price"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-book_listing.xml -i data/books.xml
        -o output/book_listing.txt book-listing

### Converting an XML document into flat file records with multi-valued fields (swath)

This example shows another way of writing a resources script that will convert an XML document into flat file records with multi-valued fields.

The input file is an XML file with multiple occurances of the element `GeoPoint` inside a `Swath` element.

Figure 137. Input XML file swath.xml

    <Swath>
      <Shape>Polygon</Shape>
      <GeoPoint>
        <lat>1</lat>
        <lon>2</lon>
      </GeoPoint>
      <GeoPoint>
        <lat>3</lat>
        <lon>4</lon>
      </GeoPoint>
      <GeoPoint>
        <lat>5</lat>
        <lon>6</lon>
      </GeoPoint>
    </Swath>

The desired output is shown below.

Figure 138. Output pipe delimited flat file swath.txt

    Shape|GeoPoint
    Polygon|1 2;3 4;5 6

This time the resources script looks like this.

Figure 139. Resources script resources-swath2pipe.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="swath">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping >
              <sx:onSubtree  path="/Swath">
                <sx:flattenSubtree recordType="hdrType">
                  <sx:subtreeFieldMap select="Shape" field="Shape"/>
                  <sx:subtreeFieldMap match="GeoPoint" select="concat(lat,' ',lon)" field="GeoPoint"/>
                </sx:flattenSubtree>
              </sx:onSubtree>
            </sx:inverseRecordMapping>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="swathFlatFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="swathFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="hdrType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="hdrType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="hdrType" name="hdrType">
        <sx:delimitedField name="Shape">
          <sx:fieldDelimiter value="|"/>
        </sx:delimitedField>
        <sx:delimitedField name="GeoPoint">
          <sx:subfieldDelimiter value=";"/>
        </sx:delimitedField>
      </sx:flatRecordType>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-swath.xml -i data/swath.xml
        -o output/swath.txt swath

### Converting an XML document into a flat file with header and multiple detail records (XML-flat header-detail)

This example shows how to write a resources script that will convert an XML document into a flat file records with header and multiple detail records.

The input file is an XML file with multiple occurances of the element `myns:review` inside a `myns:book/myns:reviews` path.

Figure 140. Input XML file bookReviews.xml

    <?xml version="1.0" encoding="utf-8"?>
    <myns:books xmlns:myns="http://mycompany.com/mynames/">
      <myns:book id="002" categoryCode="F">
        <myns:title>Kafka on the Shore</myns:title>
        <myns:author>Haruki Murakami</myns:author>
        <myns:price>25.17</myns:price>
        <myns:reviews>
          <myns:review>
            <myns:reviewer>Curley</myns:reviewer>
            <myns:rating>*****</myns:rating>
          </myns:review>
          <myns:review>
            <myns:reviewer>Larry</myns:reviewer>
            <myns:rating>***</myns:rating>
          </myns:review>
          <myns:review>
            <myns:reviewer>Moe</myns:reviewer>
            <myns:rating>*</myns:rating>
          </myns:review>
        </myns:reviews>
      </myns:book>
    </myns:books>

The desired output is shown below.

Figure 141. Output positional flat file bookReviews.txt

    TID CAuthor                        Title                              Price

    H002FHaruki Murakami               Kafka on the Shore                 25.17
    L002Curley    *****
    L002Larry     ***
    L002Moe       *

    This is a trailer record

The following resources script does the job.

Figure 142. Resources script resources-bookReviews.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="bookReviews">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="booksToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
        </sx:recordStream>
        <sx:flatFileWriter>
          <sx:flatFile ref="booksFile"/>
        </sx:flatFileWriter>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="book"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="tag"  width="1"/>
            <sx:when test="tag='H'">
              <sx:flatRecordType ref="book"/>
            </sx:when>
            <sx:when test="tag='L'">
              <sx:flatRecordType ref="review"/>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="book" name="book">
        <sx:positionalField name="tag" label="Tag" width="1"/>
        <sx:positionalField name="id" label="ID" width="3"/>
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="review" name="review">
        <sx:positionalField name="tag" label="Tag" width="1"/>
        <sx:positionalField name="id" label="ID" width="3"/>
        <sx:positionalField name="reviewer" label="Reviewer" width="10"/>
        <sx:positionalField name="rating" label="Rating" width="5"/>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="booksToFileMapping">
        <sx:onSubtree path="/myns:books/myns:book">
          <sx:parameter name="id" select="@id"/>
          <sx:flattenSubtree recordType="book">
            <sx:subtreeFieldMap select="'H'" field="tag"/>
            <sx:subtreeFieldMap select="$id" field="id"/>
            <sx:subtreeFieldMap select="myns:title" field="title"/>
            <sx:subtreeFieldMap select="@categoryCode" field="category"/>
            <sx:subtreeFieldMap select="myns:author" field="author"/>
            <sx:subtreeFieldMap select="myns:price" field="price"/>
          </sx:flattenSubtree>
          <sx:flattenSubtree match="myns:reviews/myns:review" recordType="review">
            <sx:subtreeFieldMap select="'L'" field="tag"/>
            <sx:subtreeFieldMap select="$id" field="id"/>
            <sx:subtreeFieldMap select="myns:reviewer" field="reviewer"/>
            <sx:subtreeFieldMap select="myns:rating" field="rating"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>
    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-bookReviews.xml -i data/bookReviews.xml
        -o output/bookReviews.txt bookReviews

### Breaking up a sequence of elements into flat file records ("sequence")

The example below illustrates how to use an XSLT stylesheet in an XML-flat pipeline to transform the XML into a form that is easier to map.

The input file is an XML file containing a sequence of elements.

Figure 143. Input XML file sequence.xml

    <root>
      <first>1111</first>
      <second>AD</second>
      <third>1111</third>
      <fourth>1111</fourth>
      <first>2222</first>
      <second>MO</second>
      <third>2222</third>
      <fourth>2222</fourth>
    </root>

The desired output file is shown below.

Figure 144. Output CSV file sequence.csv.

    1111,AD,1111,1111
    2222,MO,2222,2222

The following resources script does the transformation.

Figure 145. Resources script resources-sequence.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="sequence-csv">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document>
                <sx:fileSource file="data/sequence.xml"/>
              </sx:document>
              <sx:xslt ref="sequence-hierarchy"/>
            </sx:transform>
            <sx:inverseRecordMapping ref="sequenceToFileMapping"/>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="sequenceFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:xslt id="sequence-hierarchy">
        <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
          <xsl:template match="/root">
            <xsl:copy>
              <xsl:apply-templates select="first"/>
            </xsl:copy>
          </xsl:template>
          <xsl:template match="first">
            <PivotNode>
              <first>
                <xsl:value-of select="."/>
              </first>
              <second>
                <xsl:value-of select="following-sibling::second[1]"/>
              </second>
              <third>
                <xsl:value-of select="following-sibling::third[1]"/>
              </third>
              <fourth>
                <xsl:value-of select="following-sibling::fourth[1]"/>
              </fourth>
            </PivotNode>
          </xsl:template>
        </xsl:stylesheet>
      </sx:xslt>

      <sx:flatFile id="sequenceFile">
        <sx:flatFileBody>
          <sx:flatRecordType name="sequence">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="first"/>
            <sx:delimitedField name="second"/>
            <sx:delimitedField name="third"/>
            <sx:delimitedField name="fourth"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:inverseRecordMapping id="sequenceToFileMapping">
        <sx:onSubtree path="/root/PivotNode">
          <sx:flattenSubtree  recordType="sequence">
            <sx:subtreeFieldMap select="first" field="first"/>
            <sx:subtreeFieldMap select="second" field="second"/>
            <sx:subtreeFieldMap select="third" field="third"/>
            <sx:subtreeFieldMap select="fourth" field="fourth"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -o output/sequence.csv -r resources-sequence.xml sequence-csv

### Converting an XML file with hexidecimal encoded values to a flat file with special characters

The example below illustrates how to prepare a resources script that will convert an XML document with hexadecimal encoded values to a flat file with special characters.

The input XML document is shown below.

Figure 146. Input XML file transaction.xml

    <?xml version="1.0" encoding="utf-8"?>
    <transaction>
      <R03>
        <firstField>03</firstField>
        <CLIENUM>1111111</CLIENUM>
      </R03>
      <R04>
        <firstField>04</firstField>
        <NAME>John Smith</NAME>
      </R04>
    </transaction>

The desired output file is a flat file with two adjacent records (no line delimiters):

- A record beginning with the hexadecimal value 03, followed by the text "1111111"

- A record beginning with the hexadecimal value 04, followed by the text "John Smith"

The following resources script does the transformation.

Figure 147. Resources script resources-specialChar.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="transaction">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="fieldsMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter >
            <sx:flatFile ref="transactionFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="transactionFile" lineDelimited="false">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:binaryField name="firstField" width="1"/>
            <sx:when test="firstField='03'">
              <sx:flatRecordType id="R03" name="R03">
                <sx:binaryField name="firstField" label="firstField" width="1" />
                <sx:positionalField name="CLIENUM" label="CLIENUM" width="007" />
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="firstField='04'">
              <sx:flatRecordType id="R04" name="R04">
                <sx:binaryField name="firstField" label="firstField" width="1" />
                <sx:positionalField name="NAME" label="NAME" width="020" />
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:inverseRecordMapping id="fieldsMapping">
        <sx:onSubtree path="/transaction/R03">
          <sx:flattenSubtree match="/R03" recordType="R03">
            <sx:subtreeFieldMap select="firstField" field="firstField"/>
            <sx:subtreeFieldMap select="CLIENUM" field="CLIENUM"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
        <sx:onSubtree path="/transaction/R04">
          <sx:flattenSubtree match="/R04" recordType="R04">
            <sx:subtreeFieldMap select="firstField" field="firstField"/>
            <sx:subtreeFieldMap select="NAME" field="NAME"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>
    </sx:resources>

Note that the first field of each record is declared as an sx:binaryField. The string rendering of this field is as an hexidecimal value.

You can run this example on the command line by entering

    servingxml  -r resources-specialChar.xml -i data/transaction.xml
      -o output/transaction.txt transaction

### Converting an XML document to a positional flat file and ungrouping an XML element (ungrouping)

The example below illustrates how to prepare a resources script that will convert an XML document into a positional flat file and ungroup an XML element.

The input file is an XML file.

Figure 148. Input XML file grouped.xml

    <RT_STLMT>
      <OPERATING_DATE>03/04/2003</OPERATING_DATE>
      <SETTLEMENT_CODE>S14</SETTLEMENT_CODE>
      <LINE_ITEMS>
        <CHG_TYP>
          <CHG_TYP_ID>RT_ADMIN1</CHG_TYP_ID>
          <CHG_TYP_NM>Market Administration Amount1</CHG_TYP_NM>
          <TOTAL>
            <AMT>288.81</AMT>
            <INT>
              <INT_NUM>1</INT_NUM>
              <VAL>54.4</VAL>
            </INT>
            <INT>
              <INT_NUM>2</INT_NUM>
              <VAL>31.16</VAL>
            </INT>
            <INT>
              <INT_NUM>3</INT_NUM>
              <VAL>81.48</VAL>
            </INT>
            <INT>
              <INT_NUM>4</INT_NUM>
              <VAL>121.77</VAL>
            </INT>
          </TOTAL>
        </CHG_TYP>
        <CHG_TYP>
          <CHG_TYP_ID>RT_ADMIN2</CHG_TYP_ID>
          <CHG_TYP_NM>Market Administration Amount2</CHG_TYP_NM>
          <TOTAL>
            <AMT>288.81</AMT>
            <INT>
              <INT_NUM>1</INT_NUM>
              <VAL>54.4</VAL>
            </INT>
            <INT>
              <INT_NUM>2</INT_NUM>
              <VAL>31.16</VAL>
            </INT>
            <INT>
              <INT_NUM>3</INT_NUM>
              <VAL>81.48</VAL>
            </INT>
            <INT>
              <INT_NUM>4</INT_NUM>
              <VAL>121.77</VAL>
            </INT>
          </TOTAL>
        </CHG_TYP>
      </LINE_ITEMS>
    </RT_STLMT>

The desired output is shown below.

Figure 149. Output positional flat file ungrouped.csv

    OPERATING_DATE,SETTLEMENT_CODE,DETERMINANT_ID,DETERMINANT_NAME,TOTAL_AMT,INT01,INT02,INT03,INT04
    03/04/2003,S14,RT_ADMIN1,Market Administration Amount1,288.81,54.4,31.16,81.48,121.77
    03/04/2003,S14,RT_ADMIN2,Market Administration Amount2,288.81,54.4,31.16,81.48,121.77

The following resources script does the transformation.

Figure 150. Resources script resources-ungrouping.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core" xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="settlements2csv">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:parameter name="foo">bar</sx:parameter>
            <sx:inverseRecordMapping ref="settlementsToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="settlementsFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="settlementsFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="settlementType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="settlementType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="settlementType" name="settlementType">
        <sx:fieldDelimiter value=","/>
        <sx:delimitedField name="OPERATING_DATE" label="OPERATING_DATE"/>
        <sx:delimitedField name="SETTLEMENT_CODE" label="SETTLEMENT_CODE"/>
        <sx:delimitedField name="DETERMINANT_ID" label="DETERMINANT_ID"/>
        <sx:delimitedField name="DETERMINANT_NAME" label="DETERMINANT_NAME"/>
        <sx:delimitedField name="TOTAL_AMT" label="TOTAL_AMT"/>
        <sx:delimitedField name="INT01" label="INT01"/>
        <sx:delimitedField name="INT02" label="INT02"/>
        <sx:delimitedField name="INT03" label="INT03"/>
        <sx:delimitedField name="INT04" label="INT04"/>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="settlementsToFileMapping">
        <sx:onSubtree path="/RT_STLMT">
          <sx:parameter name="OPERATING_DATE" select="OPERATING_DATE"/>
          <sx:parameter name="SETTLEMENT_CODE" select="SETTLEMENT_CODE"/>
          <sx:parameter name="id" select="@id"/>
          <sx:onSubtree path="LINE_ITEMS/CHG_TYP">
            <sx:flattenSubtree recordType="settlement">
              <sx:subtreeFieldMap select="$OPERATING_DATE" field="OPERATING_DATE"/>
              <sx:subtreeFieldMap select="$SETTLEMENT_CODE" field="SETTLEMENT_CODE"/>
              <sx:subtreeFieldMap select="'RT'" field="STLMT_TYP"/>
              <sx:subtreeFieldMap select="CHG_TYP_ID" field="DETERMINANT_ID"/>
              <sx:subtreeFieldMap select="CHG_TYP_NM" field="DETERMINANT_NAME"/>
              <sx:subtreeFieldMap select="TOTAL/AMT" field="TOTAL_AMT"/>
              <sx:subtreeFieldMap select="TOTAL/INT[1]/VAL" field="INT01"/>
              <sx:subtreeFieldMap select="TOTAL/INT[2]/VAL" field="INT02"/>
              <sx:subtreeFieldMap select="TOTAL/INT[3]/VAL" field="INT03"/>
              <sx:subtreeFieldMap select="TOTAL/INT[4]/VAL" field="INT04"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>
    </sx:resources>

Note the use of nested sx:onSubtree elements, and the sx:parameter elements appearing in the outermost sx:onSubtree. The `select` attribute in these parameter declarations can refer to the outer tags in the XML document under "/RT_STLMT", but not the inner tags handled by the inner sx:onSubtree elements.

An alternative inverse record mapping, which will produce the same output, is shown below.

      <sx:inverseRecordMapping id="settlementsToFileMapping">
        <sx:onSubtree path="/RT_STLMT">
          <sx:parameter name="OPERATING_DATE" select="OPERATING_DATE"/>
          <sx:parameter name="SETTLEMENT_CODE" select="SETTLEMENT_CODE"/>
          <sx:parameter name="id" select="@id"/>
          <sx:flattenSubtree match="LINE_ITEMS/CHG_TYP" recordType="settlement">
            <sx:subtreeFieldMap select="$OPERATING_DATE" field="OPERATING_DATE"/>
            <sx:subtreeFieldMap select="$SETTLEMENT_CODE" field="SETTLEMENT_CODE"/>
            <sx:subtreeFieldMap select="'RT'" field="STLMT_TYP"/>
            <sx:subtreeFieldMap select="CHG_TYP_ID" field="DETERMINANT_ID"/>
            <sx:subtreeFieldMap select="CHG_TYP_NM" field="DETERMINANT_NAME"/>
            <sx:subtreeFieldMap select="TOTAL/AMT" field="TOTAL_AMT"/>
            <sx:subtreeFieldMap select="TOTAL/INT[1]/VAL" field="INT01"/>
            <sx:subtreeFieldMap select="TOTAL/INT[2]/VAL" field="INT02"/>
            <sx:subtreeFieldMap select="TOTAL/INT[3]/VAL" field="INT03"/>
            <sx:subtreeFieldMap select="TOTAL/INT[4]/VAL" field="INT04"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

This mapping allows for more flexibility in the sx:parameter declarations, as the `select` attribute can now refer to any content in the XML document under "/RT_STLMT", but at the expense of keeping more of the document in memory.

You can run this example on the command line by entering

    servingxml -r resources-ungrouping.xml -i data/grouped.xml
        -o output/ungrouped.csv  settlements2csv

### XML to Flat Repeating Group (all)

The example below illustrates how to prepare a resources script that will convert an XML document into a positional flat file with repeating groups.

The input file is an XML file.

Figure 151. Input XML file all.xml

    <all>
        <field1>val1</field1>
        <field2>val2</field2>
        <field3>val3</field3>
        <compositeA attr1="av11" attr2="av21" attr3="av31"/>
        <compositeA attr1="av21" attr2="av22" attr3="av23"/>
        <compositeA attr1="av31" attr2="av32" attr3="av33"/>
        <compositeB attrx="avx" attry="avy"/>
    </all>

The desired output is shown below.

Figure 152. Output positional flat file all.txt

    val1      val2      val3      av11 av21 av31 av21 av22 av23 av31 av32 av33 avx  avy

The following resources script does the transformation.

Figure 153. Resources script resources-all.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="all2flat">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="allToFileMapping"/>
            <sx:transform>
              <sx:document/>
            </sx:transform>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="allFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="allFile">
        <sx:flatFileBody>
          <sx:flatRecordType ref="allType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="allType" name="allType">
        <sx:positionalField name="f1" label="F1" width="10"/>
        <sx:positionalField name="f2" label="F2" width="10"/>
        <sx:positionalField name="f3" label="F3" width="10"/>
        <sx:repeatingGroup name="compA">
          <sx:flatRecordType name="compARecord">
            <sx:positionalField name="ca1"  width="5"/>
            <sx:positionalField name="ca2"  width="5"/>
            <sx:positionalField name="ca3"  width="5"/>
          </sx:flatRecordType>
        </sx:repeatingGroup>
        <sx:repeatingGroup name="compB">
          <sx:flatRecordType name="compBRecord">
            <sx:positionalField name="cb1"  width="5"/>
            <sx:positionalField name="cb2"  width="5"/>
          </sx:flatRecordType>
        </sx:repeatingGroup>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="allToFileMapping">
        <sx:onSubtree path="/all">
          <sx:flattenSubtree recordType="allType">
            <sx:subtreeFieldMap select="field1" field="f1"/>
            <sx:subtreeFieldMap select="field2" field="f2"/>
            <sx:subtreeFieldMap select="field3" field="f3"/>
            <sx:subtreeFieldMap match="compositeA" field="compA">
              <sx:flattenSubtree  recordType="compARecord">
                <sx:subtreeFieldMap select="@attr1" field="ca1"/>
                <sx:subtreeFieldMap select="@attr2" field="ca2"/>
                <sx:subtreeFieldMap select="@attr3" field="ca3"/>
              </sx:flattenSubtree>
            </sx:subtreeFieldMap>
            <sx:subtreeFieldMap match="compositeB" field="compB">
              <sx:flattenSubtree  recordType="compBRecord">
                <sx:subtreeFieldMap select="@attrx" field="cb1"/>
                <sx:subtreeFieldMap select="@attry" field="cb2"/>
              </sx:flattenSubtree>
            </sx:subtreeFieldMap>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-all.xml -i data/all.xml
        -o output/all.txt  all2flat

### Formating Distinct Record Lines (blocks)

The example below illustrates how to prepare a resources script that will convert an XML document into a positional flat file with different record types corresponding to different XML subtrees.

The input file is an XML file.

Figure 154. Input XML file blocks.xml

    <BLOCKS>
      <BLOCK1>
        <FIELD1>XXXX</FIELD1>
        <FIELD2>YYYY</FIELD2>
        <FIELD3>ZZZZ</FIELD3>
        <FIELD4>WWWW</FIELD4>
      </BLOCK1>
      <BLOCK2>
        <TEXT1>AAAA</TEXT1>
        <TEXT2>BBBB</TEXT2>
        <TEXT3>CCCC</TEXT3>
        <TEXT4>DDDD</TEXT4>
      </BLOCK2>
      <BLOCK3>
        <INTEGER1>1111</INTEGER1>
        <INTEGER2>2222</INTEGER2>
        <INTEGER3>3333</INTEGER3>
      </BLOCK3>
    </BLOCKS>

The desired output is shown below.

Figure 155. Output positional flat file blocks.txt

    XXXXYYYYZZZZWWWW
    AAAABBBBCCCCDDDD
    111122223333

The following resources script does the transformation.

Figure 156. Resources script resources-blocks.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="blocks">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document/>
            </sx:transform>
            <sx:inverseRecordMapping ref="blocksToFileMapping"/>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile>
              <sx:flatFileBody>
                <sx:flatRecordType ref="blockType"/>
              </sx:flatFileBody>
            </sx:flatFile>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatRecordTypeChoice id="blockType" name="blockType">
        <sx:positionalField name="dummy" width="1"/>
        <sx:when test="/Block1Record">
          <sx:flatRecordType name="BLOCK1">
            <sx:positionalField name="Field1" width="4"/>
            <sx:positionalField name="Field2" width="4"/>
            <sx:positionalField name="Field3" width="4"/>
            <sx:positionalField name="Field4" width="4"/>
          </sx:flatRecordType>
        </sx:when>
        <sx:when test="/Block2Record">
          <sx:flatRecordType name="BLOCK2">
            <sx:positionalField name="Text1" width="4"/>
            <sx:positionalField name="Text2" width="4"/>
            <sx:positionalField name="Text3" width="4"/>
            <sx:positionalField name="Text4" width="4"/>
          </sx:flatRecordType>
        </sx:when>
        <sx:when test="/Block3Record">
          <sx:flatRecordType name="BLOCK3">
            <sx:positionalField name="Integer1" width="4"/>
            <sx:positionalField name="Integer2" width="4"/>
            <sx:positionalField name="Integer3" width="4"/>
          </sx:flatRecordType>
        </sx:when>
      </sx:flatRecordTypeChoice>

      <sx:inverseRecordMapping id="blocksToFileMapping">
        <sx:onSubtree path="/BLOCKS">
          <sx:onSubtree path="BLOCK1">
            <sx:flattenSubtree recordType="Block1Record">
              <sx:subtreeFieldMap select="FIELD1" field="Field1"/>
              <sx:subtreeFieldMap select="FIELD2" field="Field2"/>
              <sx:subtreeFieldMap select="FIELD3" field="Field3"/>
              <sx:subtreeFieldMap select="FIELD4" field="Field4"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
          <sx:onSubtree path="BLOCK2">
            <sx:flattenSubtree recordType="Block2Record">
              <sx:subtreeFieldMap select="TEXT1" field="Text1"/>
              <sx:subtreeFieldMap select="TEXT2" field="Text2"/>
              <sx:subtreeFieldMap select="TEXT3" field="Text3"/>
              <sx:subtreeFieldMap select="TEXT4" field="Text4"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
          <sx:onSubtree path="BLOCK3">
            <sx:flattenSubtree recordType="Block3Record">
              <sx:subtreeFieldMap select="INTEGER1" field="Integer1"/>
              <sx:subtreeFieldMap select="INTEGER2" field="Integer2"/>
              <sx:subtreeFieldMap select="INTEGER3" field="Integer3"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-blocks.xml -i data/blocks.xml
        -o output/blocks.txt  blocks

### Block Subsequences in XML to One Line

This example illustrates a resources script that will map subsequences of blocks in the XML to a single line in the flat file.

The input file is an XML file.

Figure 157. Input XML file block_subsequences.xml

    <BLOCKS>
      <BLOCK1>
        <FIELD1>XXXX1</FIELD1>
        <FIELD2>YYYY1</FIELD2>
        <FIELD3>ZZZZ1</FIELD3>
        <FIELD4>WWWW1</FIELD4>
      </BLOCK1>
      <BLOCK2>
        <TEXT1>AAAA1</TEXT1>
        <TEXT2>BBBB1</TEXT2>
        <TEXT3>CCCC1</TEXT3>
        <TEXT4>DDDD1</TEXT4>
      </BLOCK2>
      <BLOCK1>
        <FIELD1>XXXX2</FIELD1>
        <FIELD2>YYYY2</FIELD2>
        <FIELD3>ZZZZ2</FIELD3>
        <FIELD4>WWWW2</FIELD4>
      </BLOCK1>
      <BLOCK2>
        <TEXT1>AAAA2</TEXT1>
        <TEXT2>BBBB2</TEXT2>
        <TEXT3>CCCC2</TEXT3>
        <TEXT4>DDDD2</TEXT4>
      </BLOCK2>
    </BLOCKS>

Note that BLOCK1 and BLOCK2 repeat.

The desired output is shown below.

Figure 158. Output positional flat file block_subsequences.txt

    XXXX1YYYY1ZZZZ1WWWW1AAAA1BBBB1CCCC1DDDD1
    XXXX2YYYY2ZZZZ2WWWW2AAAA2BBBB2CCCC2DDDD2

Note that BLOCK1 and BLOCK2 form a single line in the Flat output file, and then BLOCK1 and BLOCK2 repeat.

The following resources script does the transformation.

Figure 159. Resources script resources-subsequences.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="blocks">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document/>
            </sx:transform>
            <sx:inverseRecordMapping ref="blocksToFileMapping"/>
          </sx:subtreeRecordReader>
          <sx:composeRecord recordType="BLOCKS" repeatingGroup="blocks"
                            beginTest="sx:current/BLOCK1"
                            endTest="sx:current/BLOCK1">
          </sx:composeRecord>
          <sx:flatFileWriter>
            <sx:flatFile>
              <sx:flatFileBody>
                <sx:flatRecordType ref="blockType"/>
              </sx:flatFileBody>
            </sx:flatFile>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatRecordType id="blockType" name="blockType">
        <sx:repeatingGroup name="blocks">
          <sx:flatRecordTypeChoice>
            <sx:when test="/BLOCK1">
              <sx:flatRecordType name="BLOCK1">
                <sx:positionalField name="Field1" width="5"/>
                <sx:positionalField name="Field2" width="5"/>
                <sx:positionalField name="Field3" width="5"/>
                <sx:positionalField name="Field4" width="5"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="/BLOCK2">
              <sx:flatRecordType name="BLOCK2">
                <sx:positionalField name="Text1" width="5"/>
                <sx:positionalField name="Text2" width="5"/>
                <sx:positionalField name="Text3" width="5"/>
                <sx:positionalField name="Text4" width="5"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:repeatingGroup>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="blocksToFileMapping">
        <sx:onSubtree path="/BLOCKS">
          <sx:onSubtree path="BLOCK1">
            <sx:flattenSubtree recordType="BLOCK1">
              <sx:subtreeFieldMap select="FIELD1" field="Field1"/>
              <sx:subtreeFieldMap select="FIELD2" field="Field2"/>
              <sx:subtreeFieldMap select="FIELD3" field="Field3"/>
              <sx:subtreeFieldMap select="FIELD4" field="Field4"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
          <sx:onSubtree path="BLOCK2">
            <sx:flattenSubtree recordType="BLOCK2">
              <sx:subtreeFieldMap select="TEXT1" field="Text1"/>
              <sx:subtreeFieldMap select="TEXT2" field="Text2"/>
              <sx:subtreeFieldMap select="TEXT3" field="Text3"/>
              <sx:subtreeFieldMap select="TEXT4" field="Text4"/>
            </sx:flattenSubtree>
          </sx:onSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

The strategy is to map the BLOCK1 and BLOCK2 subtrees to BLOCK1 and BLOCK2 records, compose the BLOCK1 and BLOCK2 records into a composite record, and to map the composite record to a single line in the flat file.

You can run this example on the command line by entering

    servingxml -r resources-subsequences.xml -i data/block_subsequences.xml
        -o output/block_subsequences.txt blocks

### Ungrouping Elements with Multiple Levels (book hierarchy)

The example below illustrates how to prepare a resources script that will convert an XML document into a delimited flat file with different record types corresponding to different XML subtrees.

The input file is an XML file.

Figure 160. Input XML file book_hierarchy.xml

    <myns:rootNode xmlns:myns="http://mycompany.com/mynames/">
      <myns:name>root</myns:name>
      <attribute>
        <myns:name>NONE</myns:name>
        <myns:value>NONE</myns:value>
      </attribute>
      <myns:childNode>
        <myns:name>Entity.LONDON</myns:name>
        <attribute>
          <myns:name>Entity</myns:name>
          <myns:value>LONDON</myns:value>
        </attribute>
        <myns:childNode>
          <myns:name>BookName.EQUITIES_NY</myns:name>
          <attribute>
            <myns:name>BookName</myns:name>
            <myns:value>EQUITIES_NY</myns:value>
          </attribute>
          <myns:childNode>
            <myns:name>BookName.EQUITIES_NY_TR_JAYME</myns:name>
            <attribute>
              <myns:name>BookName</myns:name>
              <myns:value>EQUITIES_NY_TR_JAYME</myns:value>
            </attribute>
            <myns:childNode>
              <myns:name>BookName.PR_EQUITY_NY</myns:name>
              <attribute>
                <myns:name>BookName</myns:name>
                <myns:value>PR_EQUITY_NY</myns:value>
              </attribute>
            </myns:childNode>
            <myns:childNode>
              <myns:name>BookName.PR_EQUITY_NY1</myns:name>
              <attribute>
                <myns:name>BookName</myns:name>
                <myns:value>PR_EQUITY_NY1</myns:value>
              </attribute>
            </myns:childNode>
          </myns:childNode>
        </myns:childNode>
        <myns:childNode>
          <myns:name>BookName.EQUITIES_LONDON</myns:name>
          <attribute>
            <myns:name>BookName</myns:name>
            <myns:value>EQUITIES_LONDON</myns:value>
          </attribute>
          <myns:childNode>
            <myns:name>BookName.EQUITIES_NY_TR_JAYME</myns:name>
            <attribute>
              <myns:name>BookName</myns:name>
              <myns:value>EQUITIES_NY_TR_JAYME</myns:value>
            </attribute>
            <myns:childNode>
              <myns:name>BookName.PR_EQUITY_NY</myns:name>
              <attribute>
                <myns:name>BookName</myns:name>
                <myns:value>PR_EQUITY_NY</myns:value>
              </attribute>
            </myns:childNode>
          </myns:childNode>
        </myns:childNode>
      </myns:childNode>
    </myns:rootNode>

The desired output is shown below.

Figure 161. Output delimited flat file book_hierarchy.txt

    root,NONE,,Entity.LONDON,Entity,LONDON,,BookName,EQUITIES_NY,BookName,EQUITIES_NY_TR_JAYME,BookName,PR_EQUITY_NY,BookName,PR_EQUITY_NY1
    root,NONE,,Entity.LONDON,Entity,LONDON,,BookName,EQUITIES_LONDON,BookName,EQUITIES_NY_TR_JAYME,BookName,PR_EQUITY_NY

The following resources script does the transformation.

Figure 162. Resources script resources-bookHierarchy.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
      xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="bookHierarchyToFile">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document/>
            </sx:transform>
            <sx:inverseRecordMapping ref="bookHierarchyToFileMapping"/>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile>
              <sx:flatFileBody>
                <sx:flatRecordType ref="settlementType"/>
              </sx:flatFileBody>
            </sx:flatFile>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatRecordType id="settlementType" name="settlementType">
        <sx:fieldDelimiter value=","/>
        <sx:repeatDelimiter value=","/>
        <sx:segmentDelimiter value="|"/>
        <sx:delimitedField name="root" label="root"/>
        <sx:delimitedField name="bookCode" label="bookCode"/>
        <sx:delimitedField name="bookValue" label="bookValue"/>
        <sx:delimitedField name="level1" label="level1"/>
        <sx:delimitedField name="level1Code" label="level1Code"/>
        <sx:delimitedField name="level1Value" label="level1Value"/>
        <sx:delimitedField name="level2" label="level2"/>
        <sx:delimitedField name="level2Code" label="level2Code"/>
        <sx:delimitedField name="level2Value" label="level2Value"/>
        <sx:repeatingGroup name="level3" label="level3">
          <sx:flatRecordType name="level3Record">
            <sx:delimitedField name="code" label="code"/>
            <sx:delimitedField name="value" label="value"/>
            <sx:repeatingGroup name="level4" label="level4">
              <sx:flatRecordType name="level4Record">
                <sx:delimitedField name="code" label="code"/>
                <sx:delimitedField name="value" label="value"/>
              </sx:flatRecordType>
            </sx:repeatingGroup>
          </sx:flatRecordType>
        </sx:repeatingGroup>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="bookHierarchyToFileMapping">
        <sx:onSubtree path="/myns:rootNode">
          <sx:parameter name="root" select="myns:name"/>
          <sx:parameter name="bookCode" select="attribute/myns:name"/>
          <sx:parameter name="bookValue" select="attribute/value"/>

          <sx:onSubtree path="/myns:rootNode/myns:childNode">
            <sx:parameter name="level1" select="myns:name"/>
            <sx:parameter name="level1Code" select="attribute/myns:name"/>
            <sx:parameter name="level1Value" select="attribute/myns:value"/>
            <sx:onSubtree path="/myns:childNode/myns:childNode">
              <sx:flattenSubtree recordType="settlement">
                <sx:subtreeFieldMap select="$root" field="root"/>
                <sx:subtreeFieldMap select="$bookCode" field="bookCode"/>
                <sx:subtreeFieldMap select="$bookValue" field="bookValue"/>
                <sx:subtreeFieldMap select="$level1" field="level1"/>
                <sx:subtreeFieldMap select="$level1Code" field="level1Code"/>
                <sx:subtreeFieldMap select="$level1Value" field="level1Value"/>
                <sx:subtreeFieldMap select="name" field="level2"/>
                <sx:subtreeFieldMap select="attribute/myns:name" field="level2Code"/>
                <sx:subtreeFieldMap select="attribute/myns:value" field="level2Value"/>
                <sx:subtreeFieldMap match="myns:childNode" field="level3">
                  <sx:flattenSubtree  recordType="level3Record">
                    <sx:subtreeFieldMap select="attribute/myns:name" field="code"/>
                    <sx:subtreeFieldMap select="attribute/myns:value" field="value"/>
                    <sx:subtreeFieldMap match="myns:childNode" field="level4">
                      <sx:flattenSubtree  recordType="level4Record">
                        <sx:subtreeFieldMap select="attribute/myns:name" field="code"/>
                        <sx:subtreeFieldMap select="attribute/myns:value" field="value"/>
                      </sx:flattenSubtree>
                    </sx:subtreeFieldMap>
                  </sx:flattenSubtree>
                </sx:subtreeFieldMap>
              </sx:flattenSubtree>
            </sx:onSubtree>
          </sx:onSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-bookHierarchy.xml -i data/book_hierarchy.xml
        -o output/book_hierarchy.txt bookHierarchyToFile

### X12 XML to Flat File using XSLT

The example below illustrates how to prepare a resources script that will convert an XML representation of an X12 file into a positional flat file. Different record types correspond to different subtrees in the XML document.

The input file is an XML file.

Figure 163. Input file Edi517M2xml.xml

    <?xml version="1.0" encoding="utf-8"?>
    <X12>
      <!-- Reapting group with in the x12 document -->
      <ST>
        <ST01>517</ST01>
        <ST02>0006</ST02>
      </ST>
      <BR>
        <BR01>00</BR01>
        <BR02>AN</BR02>
        <BR03>20071019</BR03>
        <BR04/>
        <BR05/>
        <BR06/>
        <BR07>X7</BR07>
        <BR08>0101</BR08>
        <BR09>150401</BR09>
        <BR10>XM</BR10>
        <BR11>003</BR11>
        <BR12>0101003</BR12>
      </BR>
      <G62>
        <G6201>CA</G6201>
        <G6202>20070720</G6202>
      </G62>
      <G62>
        <G6201>64</G6201>
        <G6202>20070903</G6202>
      </G62>
      <LM>
        <LM01>DF</LM01>
      </LM>
      <LQ>
        <LQ01>0</LQ01>
        <LQ02>AN9</LQ02>
      </LQ>
      <N1>
        <N101>Z4</N101>
        <N102/>
        <N103>M4</N103>
        <N104>GSA</N104>
        <N105/>
        <N106>FR</N106>
      </N1>
      <N1>
        <N101>ZR</N101>
        <N102/>
        <N103>10</N103>
        <N104>N23194</N104>
        <N105/>
        <N106>TO</N106>
      </N1>
      <!-- Repating group with in ST, nmber of times based on value in BR11( current value =3) -->
      <QTY>
        <QTY01>63</QTY01>
        <QTY02>4</QTY02>
        <QTY03>BX</QTY03>
        <QTY04>BX4</QTY04>
      </QTY>
      <N9>
        <N901>TN</N901>
        <N902>N2319471430196</N902>
      </N9>
      <N9>
        <N901>NS</N901>
        <N902>7920006199162</N902>
      </N9>
      <G62>
        <G6201>17</G6201>
        <G6202>20070718</G6202>
      </G62>
      <LM>
        <LM01>DF</LM01>
      </LM>
      <LQ>
        <LQ01>0</LQ01>
        <LQ02>AN1</LQ02>
      </LQ>
      <LQ>
        <LQ01>81</LQ01>
        <LQ02>BV</LQ02>
      </LQ>
      <LQ>
        <LQ01>COG</LQ01>
        <LQ02>9Q</LQ02>
      </LQ>
      <LQ>
        <LQ01>DE</LQ01>
        <LQ02>C</LQ02>
      </LQ>
      <LQ>
        <LQ01>DF</LQ01>
        <LQ02>S</LQ02>
      </LQ>
      <LQ>
        <LQ01>A9</LQ01>
        <LQ02>N23194</LQ02>
      </LQ>
      <LQ>
        <LQ01>78</LQ01>
        <LQ02>ZQ0</LQ02>
      </LQ>
      <LQ>
        <LQ01>79</LQ01>
        <LQ02>03</LQ02>
      </LQ>
      <FA1>
        <FA101>DN</FA101>
        <FA102>D340</FA102>
      </FA1>
      <FA2>
        <FA201>B5</FA201>
        <FA202>RC</FA202>
      </FA2>
      <QTY>
        <QTY01>63</QTY01>
        <QTY02>1</QTY02>
        <QTY03>EA</QTY03>
        <QTY04>EA1</QTY04>
      </QTY>
      <N9>
        <N901>TN</N901>
        <N902>N2319471430158</N902>
      </N9>
      <N9>
        <N901>NS</N901>
        <N902>5140003694927</N902>
      </N9>
      <G62>
        <G6201>17</G6201>
        <G6202>20070725</G6202>
      </G62>
      <LM>
        <LM01>DF</LM01>
      </LM>
      <LQ>
        <LQ01>0</LQ01>
        <LQ02>AN1</LQ02>
      </LQ>
      <LQ>
        <LQ01>81</LQ01>
        <LQ02>BB</LQ02>
      </LQ>
      <LQ>
        <LQ01>COG</LQ01>
        <LQ02>9Q</LQ02>
      </LQ>
      <LQ>
        <LQ01>DE</LQ01>
        <LQ02>C</LQ02>
      </LQ>
      <LQ>
        <LQ01>DF</LQ01>
        <LQ02>S</LQ02>
      </LQ>
      <LQ>
        <LQ01>A9</LQ01>
        <LQ02>N23194</LQ02>
      </LQ>
      <LQ>
        <LQ01>78</LQ01>
        <LQ02>ZQ0</LQ02>
      </LQ>
      <LQ>
        <LQ01>79</LQ01>
        <LQ02>03</LQ02>
      </LQ>
      <FA1>
        <FA101>DN</FA101>
        <FA102>D340</FA102>
      </FA1>
      <FA2>
        <FA201>B5</FA201>
        <FA202>RC</FA202>
      </FA2>
      <QTY>
        <QTY01>63</QTY01>
        <QTY02>6</QTY02>
        <QTY03>BG</QTY03>
        <QTY04>BG6</QTY04>
      </QTY>
      <N9>
        <N901>TN</N901>
        <N902>N2319471430272</N902>
      </N9>
      <N9>
        <N901>NS</N901>
        <N902>5610009851800</N902>
      </N9>
      <G62>
        <G6201>17</G6201>
        <G6202>20070703</G6202>
      </G62>
      <LM>
        <LM01>DF</LM01>
      </LM>
      <LQ>
        <LQ01>0</LQ01>
        <LQ02>AN1</LQ02>
      </LQ>
      <LQ>
        <LQ01>81</LQ01>
        <LQ02>BV</LQ02>
      </LQ>
      <LQ>
        <LQ01>COG</LQ01>
        <LQ02>9Q</LQ02>
      </LQ>
      <LQ>
        <LQ01>DE</LQ01>
        <LQ02>C</LQ02>
      </LQ>
      <LQ>
        <LQ01>DF</LQ01>
        <LQ02>S</LQ02>
      </LQ>
      <LQ>
        <LQ01>A9</LQ01>
        <LQ02>N23194</LQ02>
      </LQ>
      <LQ>
        <LQ01>78</LQ01>
        <LQ02>ZQ0</LQ02>
      </LQ>
      <LQ>
        <LQ01>79</LQ01>
        <LQ02>03</LQ02>
      </LQ>
      <FA1>
        <FA101>DN</FA101>
        <FA102>D340</FA102>
      </FA1>
      <FA2>
        <FA201>B5</FA201>
        <FA202>RC</FA202>
      </FA2>
      <SE>
        <SE01>54</SE01>
        <SE02>0006</SE02>
      </SE>
    </X12>

The desired output is shown below.

Figure 164. Output delimited flat file Edi517M.txt

    AN9GSA0101003               N23194
    AN1  7920006199162 CBX4N2319471430196N23194SRC9QZQ003  BV
    AN1  5140003694927 CEA1N2319471430158N23194SRC9QZQ003  BB
    AN1  5610009851800 CBG6N2319471430272N23194SRC9QZQ003  BV

The following resources script does the transformation.

Figure 165. Resources script resources-xml-517M.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="xmltodlss">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document/>
              <sx:xslt ref="dlss-layout"/>
            </sx:transform>
            <sx:inverseRecordMapping ref="dlssToFileMapping"/>
          </sx:subtreeRecordReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="dlssFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:xslt id="dlss-layout">
        <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
          <xsl:strip-space elements="*"/>
          <xsl:template match="/X12">
            <xsl:copy>
              <xsl:apply-templates/>
            </xsl:copy>
          </xsl:template>

          <xsl:template match="ST">
            <PivotNode1>
              <ST>
                <xsl:value-of select="."/>
              </ST>
              <BR>
                <xsl:value-of select="following-sibling::BR/BR08"/>
              </BR>
              <BR01>
                <xsl:value-of select="following-sibling::BR/BR11"/>
              </BR01>
              <G62Sub1>
                <xsl:value-of select="following-sibling::G62[1]/G6203"/>
              </G62Sub1>
              <G62Sub2>
                <xsl:value-of select="following-sibling::G62[1]/G6204"/>
              </G62Sub2>
              <G62Sub3>
                <xsl:value-of select="following-sibling::G62[2]/G6203"/>
              </G62Sub3>
              <G62Sub4>
                <xsl:value-of select="following-sibling::G62[2]/G6204"/>
              </G62Sub4>
              <LMSub1>
                <xsl:value-of select="following-sibling::LM[1]/LM01"/>
              </LMSub1>
              <LQSub1>
                <xsl:value-of select="following-sibling::LQ[1]/LQ02"/>
              </LQSub1>
              <N1Sub1>
                <xsl:value-of select="following-sibling::N1[1]/N104"/>
              </N1Sub1>
              <N1Sub2>
                <xsl:value-of select="following-sibling::N1[2]/N104"/>
              </N1Sub2>
              <SE>
                <xsl:value-of select="following-sibling::SE/SE01"/>
              </SE>
              <xsl:apply-templates />
            </PivotNode1>
          </xsl:template>

          <xsl:template match="QTY">
            <PivotNode>
              <QTY>
                <xsl:value-of select="."/>
              </QTY>
              <QTY01>
                <xsl:value-of select="QTY04"/>
              </QTY01>
              <N104Sub>
                <xsl:value-of select="QTY05"/>
              </N104Sub>
              <N9Sub1>
                <xsl:value-of select="following-sibling::N9[1]/N902"/>
              </N9Sub1>
              <N9Sub2>
                <xsl:value-of select="following-sibling::N9[2]/N902"/>
              </N9Sub2>
              <G62Sub5>
                <xsl:value-of select="following-sibling::G62/G6203"/>
              </G62Sub5>
              <G62Sub6>
                <xsl:value-of select="following-sibling::G62/G6204"/>
              </G62Sub6>
              <G62Sub7>
                <xsl:value-of select="following-sibling::G62/G6205"/>
              </G62Sub7>
              <G62Sub8>
                <xsl:value-of select="following-sibling::G62/G6206"/>
              </G62Sub8>
              <LMSub2>
                <xsl:value-of select="following-sibling::LM[1]/LM01"/>
              </LMSub2>
              <LQSub2>
                <xsl:value-of select="following-sibling::LQ[1]/LQ02"/>
              </LQSub2>
              <LQSub3>
                <xsl:value-of select="following-sibling::LQ[2]/LQ02"/>
              </LQSub3>
              <LQSub4>
                <xsl:value-of select="following-sibling::LQ[3]/LQ02"/>
              </LQSub4>
              <LQSub5>
                <xsl:value-of select="following-sibling::LQ[4]/LQ02"/>
              </LQSub5>
              <LQSub6>
                <xsl:value-of select="following-sibling::LQ[5]/LQ02"/>
              </LQSub6>
              <LQSub7>
                <xsl:value-of select="following-sibling::LQ[6]/LQ02"/>
              </LQSub7>
              <LQSub8>
                <xsl:value-of select="following-sibling::LQ[7]/LQ02"/>
              </LQSub8>
              <LQSub9>
                <xsl:value-of select="following-sibling::LQ[8]/LQ02"/>
              </LQSub9>
              <FA1>
                <xsl:value-of select="following-sibling::FA1/FA102"/>
              </FA1>
              <FA2>
                <xsl:value-of select="following-sibling::FA2/FA202"/>
              </FA2>
            </PivotNode>
          </xsl:template>

        </xsl:stylesheet>
      </sx:xslt>

      <sx:flatFile id="dlssFile">
        <sx:flatFileBody>
          <sx:flatRecordTypeChoice>
            <sx:positionalField name="placeholder" width="1"/>
            <sx:when test="/dlss1">
              <sx:flatRecordType name="dlss1">
                <sx:positionalField name="LQSub1" width="3"/>
                <sx:positionalField name="N1Sub1" width="3"/>
                <sx:positionalField name="BR" width="4"/>
                <sx:positionalField name="BR01" width="3"/>
                <sx:positionalField name="spaces1" width="15"/>
                <sx:positionalField name="N1Sub2" width="6"/>
                <sx:positionalField name="G62Sub1" width="2"/>
                <sx:positionalField name="G62Sub2" width="3"/>
                <sx:positionalField name="G62Sub3" width="4"/>
                <sx:positionalField name="G62Sub4" width="2"/>
              </sx:flatRecordType>
            </sx:when>
            <sx:when test="/dlss2">
              <sx:flatRecordType name="dlss2">
                <sx:positionalField name="LQSub2" width="3"/>
                <sx:positionalField name="N104Sub" width="2"/>
                <sx:positionalField name="N9Sub2" width="14"/>
                <sx:positionalField name="LQSub5" width="1"/>
                <sx:positionalField name="QTY01" width="3"/>
                <sx:positionalField name="N9Sub1" width="14"/>
                <sx:positionalField name="LQSub7" width="6"/>
                <sx:positionalField name="LQSub6" width="1"/>
                <sx:positionalField name="FA2" width="2"/>
                <sx:positionalField name="LQSub4" width="2"/>
                <sx:positionalField name="LQSub8" width="3"/>
                <sx:positionalField name="LQSub9" width="2"/>
                <sx:positionalField name="G62Sub6" width="2"/>
                <sx:positionalField name="LQSub3" width="3"/>
                <sx:positionalField name="G62Sub7" width="3"/>
                <sx:positionalField name="G62Sub8" width="3"/>
              </sx:flatRecordType>
            </sx:when>
          </sx:flatRecordTypeChoice>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:inverseRecordMapping id="dlssToFileMapping">
        <sx:onSubtree path="/X12/PivotNode1">
          <sx:flattenSubtree recordType="dlss1">
            <sx:subtreeFieldMap select="LQSub1" field="LQSub1"/>
            <sx:subtreeFieldMap select="N1Sub1" field="N1Sub1"/>
            <sx:subtreeFieldMap select="BR" field="BR"/>
            <sx:subtreeFieldMap select="BR01" field="BR01"/>
            <sx:subtreeFieldMap select="N1Sub2" field="N1Sub2"/>
            <sx:subtreeFieldMap select="G62Sub1" field="G62Sub1"/>
            <sx:subtreeFieldMap select="G62Sub2" field="G62Sub2"/>
            <sx:subtreeFieldMap select="G62Sub3" field="G62Sub3"/>
            <sx:subtreeFieldMap select="G62Sub4" field="G62Sub4"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
        <sx:onSubtree path="/X12/PivotNode">
          <sx:flattenSubtree recordType="dlss2">
            <sx:subtreeFieldMap select="LQSub2" field="LQSub2"/>
            <sx:subtreeFieldMap select="N104Sub" field="N104Sub"/>
            <sx:subtreeFieldMap select="N9Sub2" field="N9Sub2"/>
            <sx:subtreeFieldMap select="LQSub5" field="LQSub5"/>
            <sx:subtreeFieldMap select="QTY01" field="QTY01"/>
            <sx:subtreeFieldMap select="N9Sub1" field="N9Sub1"/>
            <sx:subtreeFieldMap select="LQSub7" field="LQSub7"/>
            <sx:subtreeFieldMap select="LQSub6" field="LQSub6"/>
            <sx:subtreeFieldMap select="FA2" field="FA2"/>
            <sx:subtreeFieldMap select="LQSub4" field="LQSub4"/>
            <sx:subtreeFieldMap select="LQSub8" field="LQSub8"/>
            <sx:subtreeFieldMap select="LQSub9" field="LQSub9"/>
            <sx:subtreeFieldMap select="G62Sub6" field="G62Sub6"/>
            <sx:subtreeFieldMap select="LQSub3" field="LQSub3"/>
            <sx:subtreeFieldMap select="G62Sub7" field="G62Sub7"/>
            <sx:subtreeFieldMap select="G62Sub8" field="G62Sub8"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -i Edi517M2xml.xml -r resources-xml-517M.xml -o Edi517M.txt xmltodlss

### A Directory of XML Documents to a Flat File

This example shows a resources script that will read a number of XML documents in a directory and transform them into a single CSV file. It illustrates the use of the sx:documentSequence element.

The input is a directory of XML documents.

      countries-1.xml
      countries-2.xml
      countries-3.xml
      countries-4.xml
      countries-5.xml

The desired output is one CSV file containing all of the country entries.

    country_code,country_name
    ABW,ARUBA
    ADH,UNITED ARAB EMIRATES
    ...
    ZWE,ZIMBABWE

The resources script `resources-countrySequence.xml` does the transformation.

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="countries-to-csv">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="countriesToFileMapping"/>
            <sx:transform>
              <sx:documentSequence wrapWith="result">
                <sx:recordStream>
                  <sx:directoryReader directory="data">
                    <sx:fileFilter pattern="countries.*[.]xml"/>
                  </sx:directoryReader>
                </sx:recordStream>
                <sx:transform>
                  <sx:document>
                    <sx:fileSource directory="{parentDirectory}" file="{name}"/>
                  </sx:document>
                  <msv:schemaValidator>
                    <sx:urlSource url="data/countries.xsd"/>
                  </msv:schemaValidator>
                </sx:transform>
              </sx:documentSequence>
            </sx:transform>
          </sx:subtreeRecordReader>

          <sx:flatFileWriter>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileHeader>
          <sx:flatRecordType ref="countryType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="countryType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType name="country" id="countryType">
        <sx:fieldDelimiter value=","/>
        <sx:delimitedField name="code" label="country_code"/>
        <sx:delimitedField name="name" label="country_name"/>
      </sx:flatRecordType>

      <sx:inverseRecordMapping id="countriesToFileMapping">
        <sx:onSubtree path="/result/countries/country">
          <sx:flattenSubtree  recordType="country">
            <sx:subtreeFieldMap select="countryName" field="name"/>
            <sx:subtreeFieldMap select="@countryCode" field="code"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

The effect of the sx:documentSequence instruction is to read all the documents in the `data` directory whose file names match the regular expression "countries.*[.]xml". The content of these documents is wrapped in the root element `result`.

You can run this example on the command line by entering

    servingxml -r resources-countrySequence.xml -o output/countries.csv countries-to-csv

### Summary and Detail XML Documents to a Flat File (eobclaims)

This example shows a resources script that will read a number of XML documents in a directory and transform them into a single CSV file. It illustrates the use of the sx:documentSequence and saxon:xquery, elements.

The input is a directory of XML documents.

    eobclaims1.xml
    eobclaims2.xml
    eobclaimshdrtlr.xml

The desired output is one CSV file containing all of the country entries.

    ADM2301EOBS GENERATED ON 06/1 3/08
    HDR06/13/08200815501300101MEMBER01          BISHOP
    ADR1234 TEST DR                                                                                       ARVADA
    HDR06/13/08200815501300102MEMBER02          BISHOP
    ADR5678 TEST DR                                                                                       ARVADA
    TLRTOTAL EOBS GENERATED: 0000001000000000000010000018

The resources script `resources-eobclaims.xml` does the transformation. Note that this script includes the shared eobclaims record structure declarations.

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:saxon="http://www.servingxml.com/extensions/saxon">

      <sx:include href="resources-eobclaims-flatfile.xml"/>

      <sx:service id="eobclaims-to-flat">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:inverseRecordMapping ref="eobclaims-to-flat-mapping"/>
            <sx:transform>
              <sx:documentSequence wrapWith="result">
                <sx:content ref="eobclaims-document-header"/>
                <sx:content ref="eobclaims-document-body"/>
                <sx:content ref="eobclaims-document-trailer"/>
              </sx:documentSequence>
            </sx:transform>
          </sx:subtreeRecordReader>

          <sx:flatFileWriter>
            <sx:flatFile ref="eobclaims-file"/>
            <sx:fileSink file="output/eobclaims.txt"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <saxon:xquery id="eobclaims-document-header">
        <sx:preserveMarkup>
          <header>
              {doc('data/eobclaimshdrtlr.xml')/headertrailer/ADMIN_RECTYPE}
          </header>
        </sx:preserveMarkup>
      </saxon:xquery>

      <saxon:xquery id="eobclaims-document-trailer">
        <sx:preserveMarkup>
            <trailer>
                {doc('data/eobclaimshdrtlr.xml')/headertrailer/TLR-REC-TYPE}
            </trailer>
        </sx:preserveMarkup>
      </saxon:xquery>

      <sx:documentSequence id="eobclaims-document-body">
        <sx:directoryReader directory="data">
          <sx:fileFilter pattern=".*[1-9].xml"/>
        </sx:directoryReader>
        <sx:document>
          <sx:fileSource directory="{parentDirectory}" file="{name}"/>
        </sx:document>
      </sx:documentSequence>

      <sx:inverseRecordMapping id="eobclaims-to-flat-mapping">
        <sx:onSubtree path="header/ADMIN_RECTYPE">
          <sx:flattenSubtree recordType="ADM">
            <sx:subtreeFieldMap select="@record_type" field="record_type"/>
            <sx:subtreeFieldMap select="ADM_0001_TEXT" field="ADM_0001_TEXT"/>
            <sx:subtreeFieldMap select="ADM_0001_RUN_DATE" field="ADM_0001_RUN_DATE"/>
            <sx:subtreeFieldMap select="FILLER" field="FILLER"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
        <sx:onSubtree path="eob/HEADER_RECTYPE">
          <sx:flattenSubtree recordType="HDR">
            <sx:subtreeFieldMap select="@record_type" field="record_type"/>
            <sx:subtreeFieldMap select="HDR_STMT_DATE" field="HDR_STMT_DATE"/>
            <sx:subtreeFieldMap select="HDR_CLAIM_NUMBER" field="HDR_CLAIM_NUMBER"/>
            <sx:subtreeFieldMap select="HDR_PATIENT_NAME_FIRST" field="HDR_PATIENT_NAME_FIRST"/>
            <sx:subtreeFieldMap select="HDR_PATIENT_NAME_LAST" field="HDR_PATIENT_NAME_LAST"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
        <sx:onSubtree path="eob/ADR_REC_TYPE">
          <sx:flattenSubtree recordType="HDR">
            <sx:subtreeFieldMap select="@record_type" field="record_type"/>
            <sx:subtreeFieldMap select="ADR_8512_ATTN" field="ADR_8512_ATTN"/>
            <sx:subtreeFieldMap select="HDR_CLAIM_NUMBER" field="HDR_CLAIM_NUMBER"/>
            <sx:subtreeFieldMap select="ADR_8514_CITY" field="ADR_8514_CITY"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
        <sx:onSubtree path="trailer/TLR-REC-TYPE">
          <sx:flattenSubtree recordType="TLR">
            <sx:subtreeFieldMap select="@record_type" field="record_type"/>
            <sx:subtreeFieldMap select="TLR_0001_TEXT" field="TLR_0001_TEXT"/>
            <sx:subtreeFieldMap select="TLR_0001_COUNT" field="TLR_0001_COUNT"/>
            <sx:subtreeFieldMap select="FILLER" field="FILLER"/>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-eobclaims.xml -o output/eobclaims.txt eobclaims-to-flat

## Flat File to Flat File

### Converting a positional file to a pipe delimited file with header (positional to delimited)

The example below illustrates how to prepare a resources script that will convert the positional flat file `books.txt` into the delimited flat file `new-books.txt`.

The input file is a positional file.

Figure 166. Input positional flat file books.txt

    CAuthor                        Title                              Price

    FCharles Bukowski              Factotum                           22.95
    FSergei Lukyanenko             The Night Watch                    17.99
    FAndrew Crumey                 Mr Mee                             22.00
    CSteven John Metsker           Building Parsers with Java         39.95

    This is a trailer record

The desired output file is a pipe delimited flat file.

Figure 167. Output pipe delimited flat file new-books.txt

    Author|Category|Title|Price
    Charles Bukowski|F|Factotum|22.95
    Sergei Lukyanenko|F|The Night Watch|17.99
    Andrew Crumey|F|Mr Mee|22.00
    Steven John Metsker|C|Building Parsers with Java|39.95

The following resources script does the transformation.

Figure 168. Resources script resources-conversion.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="new-books">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="oldBooksFlatFile"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="newBooksFlatFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="newBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="newBookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="newBookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="newBookType" name="newBookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="title" label= "Title"/>
        <sx:delimitedField name="price" label="Price"/>
      </sx:flatRecordType>

      <sx:flatFile id="oldBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="oldBookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="oldBookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="oldBookType" name="oldBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-conversion.xml  -i data/books.txt
        -o output/new-books.txt new-books

### Converting a positional file to a new positional file with default field values (positional defaults)

The example below illustrates how to prepare a resources script that will convert the positional flat file `books.txt` into the new positional flat file `book-defaults.txt`.

The input file is a positional file.

Figure 169. Input positional flat file books.txt

    CAuthor                        Title                              Price

    FCharles Bukowski              Factotum                           22.95
    FSergei Lukyanenko             The Night Watch                    17.99
    FAndrew Crumey                 Mr Mee                             22.00
    CSteven John Metsker           Building Parsers with Java         39.95

    This is a trailer record

The desired output file is a new positional flat file with a different field ordering, a comma instead of a period in "Price", a new "Pub Date" field initialized to spaces, and a new Publisher field initialized to "Acme".

Figure 170. Output positional flat file books-defaults.txt

    Author                        Title                         CPub Date  Publisher           Price
    Charles Bukowski              Factotum                      F          Acme                22,95
    Sergei Lukyanenko             The Night Watch               F          Acme                17,99
    Andrew Crumey                 Mr Mee                        F          Acme                22,00
    Steven John Metsker           Building Parsers with Java    C          Acme                39,95

This can be accomplished with the following resources script..

Figure 171. Resources script resources-bookDefaults.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="book-defaults">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="oldBooksFlatFile"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="newBooksFlatFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="newBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="newBookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="newBookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="newBookType" name="newBookType">
        <sx:positionalField name="author" width="30" label="Author"/>
        <sx:positionalField name="title" width="30" label="Title"/>
        <sx:positionalField name="category" width="1" label="C"/>
        <sx:positionalField name="pubDate" width="10" label="Pub Date"/>
        <sx:positionalField name="publisher" width="15" label="Publisher">
          <sx:defaultValue>Acme</sx:defaultValue>
        </sx:positionalField>
        <sx:positionalField name="price2" width="10" justify="right" label="Price">
          <sx:defaultValue>
            <sx:findAndReplace searchFor ="." replaceWith="," useRegex="false">
              <sx:toString value="{price}"/>
            </sx:findAndReplace>
          </sx:defaultValue>
        </sx:positionalField>
      </sx:flatRecordType>

      <sx:flatFile id="oldBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="oldBookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="oldBookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="oldBookType" name="oldBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-bookDefaults.xml  -i data/books.txt
        -o output/book-defaults.txt book-defaults

### Converting an ASCII Positional File to an EBCDIC Delimited File, and Vice Versa (ASCII-EBCDIC)

The example below illustrates how to prepare a resources script that will convert the ASCII books.txt file of the last example to an EBCDIC file, and vice versa.

The following resources script does the transformation.

Figure 172. Resources script resources-ascii-ebcdic.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="ascii2ebcdic">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="asciiBooksFile"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="ebcdicBooksFile"/>
            <sx:defaultStreamSink encoding="Cp1047"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:service id="ebcdic2ascii">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="ebcdicBooksFile"/>
            <sx:defaultStreamSource encoding="Cp1047"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="asciiBooksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="ebcdicBooksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="ebcdicBookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="ebcdicBookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="ebcdicBookType" name="ebcdicBookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="title" label= "Title"/>
        <sx:delimitedField name="price" label="Price"/>
      </sx:flatRecordType>

      <sx:flatFile id="asciiBooksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="asciiBookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="asciiBookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="asciiBookType" name="asciiBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

You can run convert the ASCII file books.txt to the EBCDIC file books-ebcdic.dat with the command

    servingxml -r resources-ascii-ebcdic.xml -i data/books.txt
        -o output/books-ebcdic.dat ascii2ebcdic

You can run convert the EBCDIC file books-ebcdic.dat to the ASCII file books-ascii.txt with the command

    servingxml -r resources-ascii-ebcdic.xml -i data/books-ebcdic.dat
        -o output/books-ascii.txt ebcdic2ascii

### Converting an ASCII Positional File to an EBCDIC Positional File with a Cobol Packed Decimal Field, and Vice Versa (ASCII-Packed)

The example below illustrates how to prepare a resources script that will convert the ASCII books.txt file to an EBCDIC positional file with a Cobol packed decimal field, and vice versa.

The following resources script does the transformation.

Figure 173. Resources script resources-ascii-ebcdic.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="books2ebcdic-packed">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile2"/>
            <sx:defaultStreamSink encoding="Cp1047"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:service id="ebcdic-packed2books">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="booksFile2"/>
            <sx:defaultStreamSource encoding="Cp1047"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="booksFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="booksFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="bookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatFile id="booksFile2" lineDelimited="false">
        <sx:flatFileBody>
          <sx:flatRecordType ref="bookType2"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="bookType" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:positionalField name="price" label="Price" width="10" justify="right"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="bookType2" name="bookType">
        <sx:positionalField name="category" label="Category" width="1"/>
        <sx:positionalField name="author" label="Author" width="30"/>
        <sx:positionalField name="title" label="Title" width="30"/>
        <sx:packedDecimalField name="price" label="Price" digitCount="10" decimalPlaces="2"/>
      </sx:flatRecordType>

    </sx:resources>

You can run convert the ASCII file, books.txt, to the EBCDIC file with the packed decimal price field, books-ebcdic.dat, with the command

    servingxml -r resources-books_ebcdic_packed.xml -i data/books.txt
        -o output/books_ebcdic_packed.dat   books2ebcdic-packed

You can run convert the EBCDIC file with the packed decimal price field, books_ebcdic_packed.dat, to the ASCII file, books-ascii_unpacked.txt, with the command

    servingxml -r resources-books_ebcdic_packed.xml -i data/books_ebcdic_packed.dat
        -o output/books_ascii_unpacked.txt   ebcdic-packed2books

### Converting a positional file to a pipe delimited file with record filtering (book orders)

The example below illustrates how to prepare a resources script that will convert the positional flat file `bookorders-pos.txt` into the pipe-delimited flat file `bookorders-delim.txt`.

The input file is a positional file.

Figure 174. Input positional flat file bookorders-pos.txt

    CAuthor                        Title                              Price InvoiceNo InvoiceDate

    FCharles Bukowski              Factotum                           22.95 001  12/Mar/2005
    FSergei Lukyanenko             The Night Watch                    17.99 002  13/Mar/2005
    FAndrew Crumey                 Mr Mee                             22.00 003 14/June/2005
    CSteven John Metsker           Building Parsers with Java         39.95 004 15/June/2005

    This is a trailer record

The desired output file is a pipe delimited flat file.

Figure 175. Output pipe delimited flat file bookorders-delim.txt

    Author|Category|Title|Price|InvoiceNo|InvoiceDate
    Charles Bukowski|F|Factotum|22.95|INV001|12/03/2005
    Sergei Lukyanenko|F|The Night Watch|17.99|INV002|13/03/2005
    Andrew Crumey|F|Mr Mee|22.00|INV003|14/06/2005
    Steven John Metsker|C|Building Parsers with Java|39.95|INV004|15/06/2005

Note the following:

- The invoice number has the text "INV" prepended, e.g. "001" becomes "INV001"

- The month labels Mar, June etc. have been converted to numbers.

The following resources script does the transformation.

Figure 176. Resources script resources-bookorders.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="orders-pos-delim">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="posBooksFlatFile"/>
          </sx:flatFileReader>
          <sx:replaceRecord ref="changeBookOrder"/>
          <sx:flatFileWriter>
            <sx:flatFile ref="delimBooksFlatFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:replaceRecord id="changeBookOrder">
        <sx:newRecord fields="*">
          <sx:newField name="invoiceno">
            <sx:findAndReplace searchFor ="([0-9]{3})" replaceWith ="INV$1"><sx:toString value="{invoiceno}"/></sx:findAndReplace>
          </sx:newField>
          <sx:newField name="invoicedate">
            <sx:convertDate fromFormat="dd/MMM/yyyy"
                            toFormat="dd/MM/yyyy">
              <sx:toString value="{invoicedate}"/>
            </sx:convertDate>
          </sx:newField>
        </sx:newRecord>
      </sx:replaceRecord>

       <sx:flatFile id="delimBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="delimBookType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="delimBookType"/>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatRecordType id="delimBookType" name="delimBookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="title" label= "Title"/>
        <sx:delimitedField name="price" label="Price"/>
        <sx:delimitedField name="invoiceno" label="InvoiceNo"/>
        <sx:delimitedField name="invoicedate" label="InvoiceDate"/>
      </sx:flatRecordType>

      <sx:flatFile id="posBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="posBookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="posBookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="posBookType" name="posBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
        <sx:positionalField name="invoiceno" width="4" justify="right"/>
        <sx:positionalField name="invoicedate" width="13" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -i data/bookorders-pos.txt -r resources-bookorders.xml
        -o output/bookorders-delim.txt orders-pos-delim

### Checking the Integrity of a Flat File ("checked books")

The example below illustrates how to prepare a resources script for checking and signing flat files with CRC values and byte counts. Here, the input file is the positional flat file `checkedBooks.txt`, to be converted into the pipe-delimited flat file `signedBooks.txt`.

The input file is a positional file.

Figure 177. Input positional flat file checkedBooks.txt

    356      2360012630

    FCharles Bukowski              Factotum                           22.95 001  12/Mar/2005
    FSergei Lukyanenko             The Night Watch                    17.99 002  13/Mar/2005
    FAndrew Crumey                 Mr Mee                             22.00 003  14/Jun/2005
    CSteven John Metsker           Building Parsers with Java         39.95 004  15/Jun/2005

    This is a trailer record

Note the following.

- The file has a two line header section, followed by a data section, followed by a two line trailer section.

- The first line of the header contains two positional fields, the size of the data section in bytes, followed by the CRC for the data section.

The desired output file is a pipe delimited flat file.

Figure 178. Output pipe delimited flat file signedBooks.txt

    Author|Category|Title|Price|InvoiceNo|InvoiceDate
    237|1661800843
    Charles Bukowski|F|Factotum|22.95|001|12/Mar/2005
    Sergei Lukyanenko|F|The Night Watch|17.99|002|13/Mar/2005
    Andrew Crumey|F|Mr Mee|22.00|003|14/Jun/2005
    Steven John Metsker|C|Building Parsers with Java|39.95|004|15/Jun/2005

The output file has a two line header section. The two delimited fields in the second line store the size and CRC of the data section below.

The following resources script does the transformation.

Figure 179. Resources script resources-checkedBooks.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="books-pos-delim">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:flatFile ref="posBooksFlatFile"/>
          </sx:flatFileReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="delimBooksFlatFile"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="delimBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="delimBookType"/>
          <sx:flatRecordType ref="delimBookHeaderType"/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="delimBookType"/>
        </sx:flatFileBody>
        <sx:flatFileSignature recordType="bookHeaderType" field="filecrc" method="crc"/>
        <sx:flatFileSignature recordType="bookHeaderType" field="filesize" method="size"/>
      </sx:flatFile>

      <sx:flatRecordType id="delimBookType" name="delimBookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="title" label= "Title"/>
        <sx:delimitedField name="price" label="Price"/>
        <sx:delimitedField name="invoiceno" label="InvoiceNo"/>
        <sx:delimitedField name="invoicedate" label="InvoiceDate"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="delimBookHeaderType" name="bookHeaderType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="filesize"/>
        <sx:delimitedField name="filecrc"/>
      </sx:flatRecordType>

      <sx:flatFile id="posBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="posBookHeaderType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>

        <sx:flatFileBody>
          <sx:flatRecordType ref="posBookType"/>
        </sx:flatFileBody>

        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>

        <sx:flatFileSignature recordType="bookHeaderType" field="filesize" method="size"/>
        <sx:flatFileSignature recordType="bookHeaderType" field="filecrc" method="crc"/>
      </sx:flatFile>

      <sx:flatRecordType id="posBookHeaderType" name="bookHeaderType">
        <sx:positionalField name="filesize" width="9"/>
        <sx:positionalField name="filecrc" width="10"/>
      </sx:flatRecordType>

      <sx:flatRecordType id="posBookType" name="posBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
        <sx:positionalField name="invoiceno" width="4" justify="right"/>
        <sx:positionalField name="invoicedate" width="13" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

Note the following.

- The sx:recordContent element contains two sx:flatFileSignature elements, which define the rules for checking the CRC and size when reading the file, and signing the header or trailer with a CRC and size when writing the file.

- The sx:flatFileSignature element has attributes `recordType` and `field`, which identify where in the header or trailer the signature value belongs.

You can run this example on the command line by entering

    servingxml  -i data/checkedBooks.txt -r resources-checkedBooks.xml
        -o output/checkedBooks-delim.txt books-pos-delim

If the CRC value in the header does not match the computed CRC value for the data, processing will be stopped and a message will be written to the log:

    CRC integrity check failed.  Expected 2360012630, found 252430032.

### Converting a directory of book positional files to pipe delimited files (new format)

The example below illustrates how to prepare a resources script that will process all the files in the `data` directory matching the regular expression "books.*[.]txt", convert them from positional flat file formats to pipe delimited formats, and write them out to the `output` directory.

This time the input is a directory of files.

Figure 180. Directory containing books fixed record length input files

    [.]                     countries.csv           multivalued-field.csv
    [..]                    countries.xsd           plans.txt
    3545_JH4DA3_4_H_.xml    country-record.xsd      tasks.csv
    bad-countries.csv       exotics.txt             timesheets.csv
    books.20040613.txt      hot-record.xsd          trades.txt
    books.20040802.txt      hot-record.xsx
    books.txt               hot.txt
    books.xml               messages.properties

We want to take the positional files whose names match the regular expression `"(books.*)[.]txt"` and convert them all into pipe delimited files.

Figure 181. Books pipe delimited output files.

    books-new.txt            books.20040802-new.txt
    books.20040613-new.txt

The following resources script does the transformation.

Figure 182. Resources script resources-books2csv.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="all-books">
        <sx:recordStream>
          <sx:directoryReader directory="data">
            <sx:fileFilter pattern="books.*[.]txt"/>
          </sx:directoryReader>
          <sx:processRecord>
            <sx:parameter name="output-file">
              <sx:findAndReplace searchFor ="(books.*)[.]txt" replaceWith ="$1-new.txt"><sx:toString value="{name}"/></sx:findAndReplace>
            </sx:parameter>
            <sx:recordStream>
              <sx:flatFileReader>
                <sx:fileSource directory="{parentDirectory}" file="{name}"/>
                <sx:flatFile ref="oldBooksFlatFile"/>
              </sx:flatFileReader>
              <sx:flatFileWriter>
                <sx:fileSink directory="output" file="{$output-file}"/>
                <sx:flatFile ref="newBooksFlatFile"/>
              </sx:flatFileWriter>
            </sx:recordStream>
          </sx:processRecord>
        </sx:recordStream>
      </sx:service>

      <sx:flatRecordType name="newBookType">
        <sx:fieldDelimiter value="|"/>
        <sx:delimitedField name="author" label="Author"/>
        <sx:delimitedField name="category" label="Category"/>
        <sx:delimitedField name="title" label= "Title"/>
        <sx:delimitedField name="price" label="Price"/>
      </sx:flatRecordType>

      <sx:flatFile id="oldBooksFlatFile">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="oldBookType"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="oldBookType"/>
        </sx:flatFileBody>
        <sx:flatFileTrailer>
          <sx:annotationRecord></sx:annotationRecord>
          <sx:annotationRecord>This is a trailer record</sx:annotationRecord>
        </sx:flatFileTrailer>
      </sx:flatFile>

      <sx:flatRecordType id="oldBookType" name="oldBookType">
        <sx:positionalField name="category" width="1"/>
        <sx:positionalField name="author" width="30"/>
        <sx:positionalField name="title" width="30"/>
        <sx:positionalField name="price" width="10" justify="right"/>
      </sx:flatRecordType>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-conversion.xml all-books

### Generated records to multiple files (multiple_output_files)

The example below illustrates how to prepare a resources script that will send the the output of a custom record reader to multiple output files, depending on parameter and record field values supplied by the custom record reader.

The input records are generated by the following Java class.

Figure 183. TradeRecordReader Java class

    package flat2flat;

    import com.servingxml.app.ServiceContext;
    import com.servingxml.components.recordio.AbstractRecordReader;
    import com.servingxml.app.Flow;
    import com.servingxml.util.Name;
    import com.servingxml.util.QualifiedName;
    import com.servingxml.util.ServingXmlException;
    import com.servingxml.util.record.Record;
    import com.servingxml.util.record.RecordBuilder;
    import com.servingxml.util.record.ParameterBuilder;

    public class TradeRecordReader extends AbstractRecordReader {
      private static final Name FEED_NAME = new QualifiedName("feed");
      private static final Name TRADE_RECORD_TYPE = new QualifiedName("trade");
      private static final Name TRANSACTION_RECORD_TYPE = new QualifiedName("transaction");
      private static final Name RECORD_TYPE_NAME = new QualifiedName("record_type");
      private static final Name ID_NAME = new QualifiedName("id");
      private static final Name TRADE_DATE_NAME = new QualifiedName("trade_date");
      private static final Name TRADE_TIME_NAME = new QualifiedName("trade_time");
      private static final Name DESCRIPTION_NAME = new QualifiedName("description");
      private static final Name REFERENCE_NAME = new QualifiedName("reference");

      public void readRecords(ServiceContext context, Flow flow)
       {

        //  Start the record stream
        startRecordStream(context, flow);

        ParameterBuilder parameterBuilder = new ParameterBuilder(flow.getParameters());

        RecordBuilder trRecordBuilder = new RecordBuilder(TRADE_RECORD_TYPE);
        RecordBuilder tnRecordBuilder = new RecordBuilder(TRANSACTION_RECORD_TYPE);

        Record newParameters;
        Record record;

        //  Load London trades
        //  Set the parameter feed=LONDON
        parameterBuilder.setString(FEED_NAME,"LONDON");
        newParameters = parameterBuilder.toRecord();

        trRecordBuilder.setString(RECORD_TYPE_NAME,"TR");
        trRecordBuilder.setString(ID_NAME,"0001");
        trRecordBuilder.setString(TRADE_DATE_NAME,"03/25/2005");
        trRecordBuilder.setString(TRADE_TIME_NAME,"01:50:00");
        trRecordBuilder.setString(DESCRIPTION_NAME,"This is a trade record");
        record = trRecordBuilder.toRecord();
        trRecordBuilder.clear();

        //  Write the London "trade" record
        Flow newFlow = flow.replaceParameters(context, newParameters).replaceRecord(context, record);
        writeRecord(context, newFlow);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0002");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1234");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"A child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the first London "transaction" record
        newFlow = flow.replaceRecord(context, record);
        writeRecord(context, newFlow);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0003");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1235");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"Another child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the second London "transaction" record
        newFlow = flow.replaceRecord(context, record);
        writeRecord(context, newFlow);

        //  Load Toronto trades
        //  Set the parameter feed=TORONTO
        parameterBuilder.setString(FEED_NAME,"TORONTO");
        newParameters = parameterBuilder.toRecord();

        trRecordBuilder.setString(RECORD_TYPE_NAME,"TR");
        trRecordBuilder.setString(ID_NAME,"0004");
        trRecordBuilder.setString(TRADE_DATE_NAME,"03/25/2005");
        trRecordBuilder.setString(TRADE_TIME_NAME,"04:50:00");
        trRecordBuilder.setString(DESCRIPTION_NAME,"This is a trade record");
        record = trRecordBuilder.toRecord();
        trRecordBuilder.clear();

        //  Write the Toronto "trade" record
        newFlow = flow.replaceParameters(context, newParameters).replaceRecord(context, record);
        writeRecord(context, newFlow);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0005");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1236");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"A child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the first Tronto "transaction" record
        newFlow = flow.replaceRecord(context, record);
        writeRecord(context, newFlow);

        tnRecordBuilder.setString(RECORD_TYPE_NAME,"TN");
        tnRecordBuilder.setString(ID_NAME,"0006");
        tnRecordBuilder.setString(REFERENCE_NAME,"X1237");
        tnRecordBuilder.setString(DESCRIPTION_NAME,"Another child transaction");
        record = tnRecordBuilder.toRecord();
        tnRecordBuilder.clear();

        //  Write the second Toronto "transaction" record
        newFlow = flow.replaceRecord(context, record);
        writeRecord(context, newFlow);

        //  End the record stream
        endRecordStream(context, flow);
      }
    }

With parameter `feed`=`LONDON`, the `TradeRecordReader` class will generate the following stream of records.

record_type | id |  |  |
--- | --- | --- | --- | ---
TR | 0001 | 03/25/2005 | 1:50:00 | This is a trade record
TN | 0002 | X1234 | A child transaction |
TN | 0003 | X1235 | Another child transaction |

With parameter `feed`=`TORONTO`, the `TradeRecordReader` class will generate the following stream of records.

record_type | id |  |  |
--- | --- | --- | --- | ---
TR | 0004 | 03/25/2005 | 4:50:00 | This is a trade record
TN | 0005 | X1236 | A child transaction |
TN | 0006 | X1237 | Another child transaction |

The LONDON master trades need to be written to the file `trades-London.txt`.

Figure 184. Output flat file trades-London.txt

    000103/25/200501:50:00This is a trade record

The LONDON transaction details need to be written to the file `transaction-London.txt`.

Figure 185. Output flat file transactions-London.txt

    0002X1234A child transaction
    0003X1235Another child transaction

The TORONTO master trades need to be written to the file `trades-Toronto.txt`.

Figure 186. Output flat file trades-Toronto.txt

    000403/25/200504:50:00This is a trade record

The TORONTO transaction details need to be written to the file `transaction-Toronto.txt`.

Figure 187. Output flat file transactions-Toronto.txt

    0005X1236A child transaction
    0006X1237Another child transaction

The following resources script does the transformation.

Figure 188. Resources script resources-multiple_output_files.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="trades">
        <sx:recordStream>
          <sx:customRecordReader class="flat2flat.TradeRecordReader"/>
          <!-- Which output file is determined by the parameter "feed"
                and the record field "record_type", which are both
                supplied in the custom record reader.
          -->
          <sx:choose>
            <sx:when test="$feed='LONDON' and record_type = 'TR'">
              <sx:flatFileWriter>
                <sx:fileSink directory="output" file="trades-London.txt"/>
                <sx:flatFile ref="trades"/>
              </sx:flatFileWriter>
            </sx:when>
            <sx:when test="$feed = 'LONDON' and record_type = 'TN'">
              <sx:flatFileWriter>
                <sx:fileSink directory="output" file="transactions-London.txt"/>
                <sx:flatFile ref="transactions"/>
              </sx:flatFileWriter>
            </sx:when>
            <sx:when test="$feed='TORONTO' and record_type = 'TR'">
              <sx:flatFileWriter>
                <sx:fileSink directory="output" file="trades-Toronto.txt"/>
                <sx:flatFile ref="trades"/>
              </sx:flatFileWriter>
            </sx:when>
            <sx:when test="$feed = 'TORONTO' and record_type = 'TN'">
              <sx:flatFileWriter>
                <sx:fileSink directory="output" file="transactions-Toronto.txt"/>
                <sx:flatFile ref="transactions"/>
              </sx:flatFileWriter>
            </sx:when>
          </sx:choose>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="trades">
        <sx:flatFileBody>
              <sx:flatRecordType name="trade">
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="trade_date" width="10"/>
                <sx:positionalField name="trade_time" width="8"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

      <sx:flatFile id="transactions">
        <sx:flatFileBody>
              <sx:flatRecordType name="transaction">
                <sx:positionalField name="id" width="4"/>
                <sx:positionalField name="reference" width="5"/>
                <sx:positionalField name="description" width="30"/>
              </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

Note the following points.

- The record content is defined by a custom record reader, implemented in the Java class `TradeRecordReader`.

You can run this example on the command line by

- Compiling the Java class `TradeRecordReader` and copying the resulting `.class` file into the `dir/classes` directory
- Entering the command

      servingxml -r resources-multiple_output_files.xml trades

### Reading remote directories

This example shows how to read a remote directory and write a listing of its contents to a flat file.

The desired output is a positional flat file showing a remote directory listing.

Figure 189. Output flat file with directory listing

    Parent                        Name                          Size      Date Modified

    /                             incoming                      28672     Mon Dec 20 03:27:00 PST 2004
    /                             lost+found                    16384     Wed Feb 07 00:00:00 PST 2001
    /                             pub                           4096      Sun Oct 17 23:45:00 PDT 2004
    /pub                          sf-overflow                   17        Sun Oct 17 23:45:00 PDT 2004
    /pub                          sourceforge                   581632    Mon Nov 15 18:34:00 PST 2004
    /pub/sourceforge              a                             28672     Fri Mar 19 00:00:00 PST 2004
    /pub/sourceforge/a            a-                            4096      Wed Oct 06 18:36:00 PDT 2004
    /pub/sourceforge/a/a-         a-4                           4096      Tue Dec 16 00:00:00 PST 2003
    /pub/sourceforge/a/a-/a-4     a4-0.01777.tar.gz             173990    Thu Feb 13 00:00:00 PST 2003
    /pub/sourceforge/a/a-/a-4     a4-0.03109.tar.gz             175327    Tue May 06 00:00:00 PDT 2003

The following resources script does the transformation.

Figure 190. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:edt="http://www.servingxml.com/extensions/edtftp">

      <edt:ftpClient id="sourceforge" host="upload.sourceforge.net"
                     user="anonymous" password="xxx"/>

      <sx:service id="remote-dir-listing">
        <sx:recordStream>
          <edt:ftpDirectoryReader remoteDirectory="." recurse="true" maxItems="10">
            <edt:ftpClient ref="sourceforge"/>
          </edt:ftpDirectoryReader>
          <sx:flatFileWriter>
            <sx:flatFile ref="directory-flat-file"/>
          </sx:flatFileWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatRecordType id="directory-record-type" name="directory-record-type">
        <sx:positionalField name="parentDirectory" label="Parent" width="30"/>
        <sx:positionalField name="name" label="Name" width="30"/>
        <sx:positionalField name="size" label="Size" width="10"/>
        <sx:positionalField name="lastModified" label="Date Modified" width="30"/>
      </sx:flatRecordType>

      <sx:flatFile id="directory-flat-file">
        <sx:flatFileHeader>
          <sx:flatRecordType ref="directory-record-type"/>
          <sx:annotationRecord/>
        </sx:flatFileHeader>
        <sx:flatFileBody>
          <sx:flatRecordType ref="directory-record-type"/>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources.xml -o output/remote-dir-listing.txt remote-dir-listing

### Converting a sequential stream of records into multiple batch files

The example below illustrates how to prepare a resources script that wil break up a stream of records into multiple batches.

We want to read the CSV file countries.csv containing 245 country records and produce five files containing 50, 50, 50, 50 and 45 country records.

Figure 191. Batched countries files.

    06/12/2007  07:24 PM    <DIR>          .
    06/12/2007  07:24 PM    <DIR>          ..
    06/12/2007  07:24 PM               814 countries-1.csv
    06/12/2007  07:24 PM               795 countries-2.csv
    06/12/2007  07:24 PM               786 countries-3.csv
    06/12/2007  07:24 PM               784 countries-4.csv
    06/12/2007  07:24 PM               793 countries-5.csv

The following resources script does the transformation.

Figure 192. Resources script resources-batchRecords.xml

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="countries">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:fileSource file="data/{$rootName}.csv"/>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileReader>
          <sx:batchedRecordWriter batchSize="50">
            <sx:flatFileWriter>
              <sx:fileSink file="output/{$rootName}-{$sx:batchSequenceNumber}.csv"/>
              <sx:flatFile ref="countriesFile"/>
            </sx:flatFileWriter>
          </sx:batchedRecordWriter>
        </sx:recordStream>
      </sx:service>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="country">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="code"/>
            <sx:delimitedField name="name" trimLeading="true"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-batchRecords.xml countries
      rootName=countries

### Batched Flat Files

#### Batched sequences of records

The example below illustrates how to prepare a resources script that wil break up a flat file into smaller batches.

Suppose you want to read the CSV file countries.csv containing 245 country records and write it out as five files containing 50, 50, 50, 50 and 45 records.

Figure 193. Batched CSV files.

    06/12/2007  07:24 PM    <DIR>          .
    06/12/2007  07:24 PM    <DIR>          ..
    06/12/2007  07:24 PM               814 countries-1.csv
    06/12/2007  07:24 PM               795 countries-2.csv
    06/12/2007  07:24 PM               786 countries-3.csv
    06/12/2007  07:24 PM               784 countries-4.csv
    06/12/2007  07:24 PM               793 countries-5.csv

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="countries">
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:fileSource file="data/{$rootName}.csv"/>
            <sx:flatFile ref="countriesFile"/>
          </sx:flatFileReader>
          <sx:batchedRecordWriter batchSize="50">
            <sx:flatFileWriter>
              <sx:fileSink file="intermediate/{$rootName}-{$sx:batchSequenceNumber}.csv"/>
              <sx:flatFile ref="countriesFile"/>
            </sx:flatFileWriter>
          </sx:batchedRecordWriter>
        </sx:recordStream>
      </sx:service>

      <sx:recordContent id="countries">
        <sx:flatFileReader>
          <sx:fileSource directory="{parentDirectory}" file="{name}"/>
          <sx:flatFile ref="countriesFile"/>
        </sx:flatFileReader>
        <sx:recordMapping ref="countriesToXmlMapping"/>
      </sx:recordContent>

      <sx:flatFile id="countriesFile">
        <sx:commentStarter value="#"/>
        <sx:flatFileBody>
          <sx:flatRecordType name="country">
            <sx:fieldDelimiter value=","/>
            <sx:delimitedField name="code"/>
            <sx:delimitedField name="name" trimLeading="true"/>
          </sx:flatRecordType>
        </sx:flatFileBody>
      </sx:flatFile>

    </sx:resources>

You can run this example on the command line by entering

    servingxml  -r resources-batchRecords.xml countries
      rootName=countries

## XML to XML

### Extracting subtrees and wrapping them in containing tags

This example shows how to extract a set of subtrees from an XML document according to some criteria and wrap them in containing tags.

The input file is an XML document.

Figure 194. Input XML document books.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <myns:books xmlns:myns="http://mycompany.com/mynames/"
                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xsi:schemaLocation="url2">
      <myns:book categoryCode="F">
        <myns:title>Factotum</myns:title>
        <myns:author>Charles Bukowski</myns:author>
        <myns:price>22.95</myns:price>
        <myns:isbn>0876852630</myns:isbn>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>The Night Watch</myns:title>
        <myns:author>Sergei Lukyanenko</myns:author>
        <myns:price>17.99</myns:price>
        <myns:isbn>0434014125</myns:isbn>
      </myns:book>
      <myns:book categoryCode="F">
        <myns:title>Mr Mee</myns:title>
        <myns:author>Andrew Crumey</myns:author>
        <myns:price>22.00</myns:price>
        <myns:isbn>0312282354</myns:isbn>
      </myns:book>
      <myns:book categoryCode="C">
        <myns:title>Building Parsers with Java</myns:title>
        <myns:author>Steven John Metsker</myns:author>
        <myns:price>39.95</myns:price>
        <myns:isbn>0201719622</myns:isbn>
      </myns:book>
    </myns:books>

The desired output is.

    <?xml version="1.0" encoding="UTF-8"?>
    <envelope>
      <header mode="xml">
        <creationDate>2007-12-03T21:39:48.265-05:00</creationDate>
      </header>
      <body>
        <myns:books xmlns:myns="http://mycompany.com/mynames/"
                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                    xsi:schemaLocation="url2">
          <myns:book categoryCode="F">
            <myns:title>Factotum</myns:title>
            <myns:author>Charles Bukowski</myns:author>
            <myns:price>22.95</myns:price>
            <myns:isbn>0876852630</myns:isbn>
          </myns:book>
          <myns:book categoryCode="F">
            <myns:title>The Night Watch</myns:title>
            <myns:author>Sergei Lukyanenko</myns:author>
            <myns:price>17.99</myns:price>
            <myns:isbn>0434014125</myns:isbn>
          </myns:book>
          <myns:book categoryCode="F">
            <myns:title>Mr Mee</myns:title>
            <myns:author>Andrew Crumey</myns:author>
            <myns:price>22.00</myns:price>
            <myns:isbn>0312282354</myns:isbn>
          </myns:book>
        </myns:books>
      </body>
      <trailer>This is a trailer element</trailer>
    </envelope>

The resources script below does the transformation.

Figure 195. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:myns="http://mycompany.com/mynames/">

      <sx:service id="wrap">
        <sx:wrap>
          <sx:xsltSerializer/>
          <envelope>
            <header>
              <sx:attribute name="mode" value="xml"/>
              <creationDate>
                <sx:currentDateTime/>
              </creationDate>
            </header>
            <body>
              <myns:books xmlns:myns="http://mycompany.com/mynames/"
                          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                          xsi:schemaLocation="url2">
                <sx:transform>
                  <sx:document/>
                  <sx:processSubtree path="/myns:books/myns:book">
                    <sx:choose>
                      <sx:when test="@categoryCode='F'">
                        <sx:transform/>
                      </sx:when>
                    </sx:choose>
                  </sx:processSubtree>
                </sx:transform>
              </myns:books>
            </body>
            <trailer>This is a trailer element</trailer>
          </envelope>
        </sx:wrap>
      </sx:service>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-wrap.xml -i documents/books.xml -o output/wrap.xml wrap

### Unwrapping subtrees in an XML document

Continuing from the previous example, the input file is the same, but you want to produce multiple files like the one shown below, with filenames differentiated by the isbn number.

Figure 196. Book XML output file book-0876852630.xml.

    <?xml version="1.0" encoding="UTF-8"?>
    <myns:book xmlns:myns="http://mycompany.com/mynames/"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               categoryCode="F">
      <myns:title>Factotum</myns:title>
      <myns:author>Charles Bukowski</myns:author>
      <myns:price>22.95</myns:price>
      <myns:isbn>0876852630</myns:isbn>
    </myns:book>

The resources script below does the transformation.

Figure 197. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                         xmlns:myns="http://mycompany.com/mynames/" >

      <sx:service id="unwrap">
        <sx:transform>
          <sx:document/>
          <sx:processSubtree path="/myns:books/myns:book">
            <sx:choose>
              <sx:when test="@categoryCode='F'">
                <sx:parameter name="isbn" select="myns:isbn"/>
                <sx:transform>
                  <sx:xsltSerializer>
                    <sx:fileSink file="output/book-{$isbn}.xml"/>
                  </sx:xsltSerializer>
                </sx:transform>
              </sx:when>
            </sx:choose>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-unwrap.xml -i documents/books.xml unwrap

### Streaming a file through a pipeline

This example shows an implementation of the SAX Pipeline example in Michael Kay's XSLT 2nd Edition Programmer's Reference, Appendix F, Example 5.

Figure 198. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="pipeline">
        <sx:serialize>
          <sx:transform>
            <sx:saxFilter class="PreFilter"/>
            <sx:xslt><sx:urlSource url="styles/filter.xsl"/></sx:xslt>
            <sx:saxFilter class="PostFilter"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

    </sx:resources>

`PreFilter` and `PostFilter` are the names of Java classes that implement the `org.xml.sax.XMLFilter` interface. The listings below show the Java code for the `PreFilter` and `PostFilter` classes (from Kay's book.)

Figure 199. PreFilter class

    public class PreFilter extends XMLFilterImpl {

      public void startElement (String uri, String localName, String qName,
      Attributes atts) throws SAXException {

        String newLocalName = localName.toLowerCase();
        String newQName = qName.toUpperCase();
        AttributesImpl newAtts =
          (atts.getLength()>0 ?
           new AttributesImpl(atts) :
           new AttributesImpl());
        newAtts.addAttribute("", "old-local-name", "old-local-name", "CDATA", localName);
        newAtts.addAttribute("", "old-qname", "old-qname", "CDATA", qName);
        super.startElement(uri, newLocalName, newQName, newAtts);
      }

      public void endElement (String uri, String localName, String qName)
      throws SAXException {
        String newLocalName = localName.toLowerCase();
        String newQName = qName.toUpperCase();
        super.endElement(uri, newLocalName, newQName);
      }
    }

Figure 200. PostFilter class

    public class PostFilter extends XMLFilterImpl {

      public Stack stack;

      public void startDocument() throws SAXException {
        stack = new Stack();
        super.startDocument();
      }

      public void startElement (String uri, String localName, String qName,
                                Attributes atts)
      throws SAXException {

        String originalLocalName = localName;
        String originalQName = qName;
        AttributesImpl newAtts = new AttributesImpl();
        for (int i=0; i<atts.getLength(); i++) {
          String name = atts.getQName(i);
          String val = atts.getValue(i);
          if (name.equals("old-local-name")) {
            originalLocalName = val;
          } else if (name.equals("old-qname")) {
            originalQName = val;
          } else {
            newAtts.addAttribute(atts.getURI(i),
              atts.getLocalName(i),
              name,
              atts.getType(i),
              val);
          }
        }
        super.startElement(uri, originalLocalName, originalQName, newAtts);
        stack.push(originalLocalName);
        stack.push(originalQName);
      }

      public void endElement (String uri, String localName, String qName)
      throws SAXException {
        String originalQName = (String)stack.pop();
        String originalLocalName = (String)stack.pop();
        super.endElement(uri, originalLocalName, originalQName);
      }
    }

You can run this example on the command line by

- Running the sample build script at the command line, to compile the pre and post Java `XMLFilter` classes and copy them into the `dir/classes` directory.
- Entering the following command to produce the XML output file.

      servingxml -r resources-file.xml -i documents/mixed-up.xml
          -o output/original-file.html pipeline

### Streaming dynamically generated SAX events through a pipeline

This example produces the same output as the previous, but this time, instead of reading an XML document from a file, we generate the SAX events dynamically in a custom SAX reader.

Figure 201. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="pipeline">
        <sx:serialize>
          <sx:transform>
            <sx:document>
              <sx:saxReader class="MySaxReader"/>
            </sx:document>
            <sx:saxFilter class="PreFilter"/>
            <sx:xslt><sx:urlSource url="styles/filter.xsl"/></sx:xslt>
            <sx:saxFilter class="PostFilter"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

    </sx:resources>

The Java class `MySaxReader` looks like this.

    public class MySaxReader implements XMLReader ...

      private static final AttributesImpl NO_ATTRIBUTES = new AttributesImpl();

      public void parse(String systemId)
      throws IOException, SAXException {
        if (contentHandler != null) {
          contentHandler.startDocument();
          contentHandler.startElement("","HTML","HTML",NO_ATTRIBUTES);
          contentHandler.startElement("","Head","Head",NO_ATTRIBUTES);
          contentHandler.startElement("","TITLE","TITLE",NO_ATTRIBUTES);
          char[] title = "This is a mixed-case HTML-like XML document".toCharArray();
          contentHandler.characters(title,0,title.length);
          contentHandler.endElement("","TITLE","TITLE");
          contentHandler.endElement("","Head","Head");
          contentHandler.startElement("","Body","Body");

          ...

          contentHandler.endElement("","Body","Body");
          contentHandler.endElement("","HTML","HTML");
          contentHandler.endDocument();
        }
      }

You can run this example on the command line by

- Run the sample build script at the command line, to compile the pre and post Java `XMLFilter` classes and copy them into the `dir/classes` directory.
- Entering the following command to produce the XML output file.

      servingxml -r resources-sax.xml -o output/original-sax.html pipeline

### Processing document subtrees and serializing them to separate files (invoices.)

This example shows how to serialize individual document subtrees to separate html and pdf files.

The input file is an XML document containing a number of invoices.

Figure 202. Input XML file invoices.xml

    <?xml version="1.0" encoding="iso-8859-1"?>
    <inv:invoices xmlns:inv="http://www.telio.be/ns/2002/invoice">
      <inv:invoice currency="EUR" id="200302-01" paid="false">
      	<inv:client id="test">
          <inv:email>john@abc.com</inv:email>
        </inv:client>
      	<inv:date>2003-02-03</inv:date>
      	<inv:description>Test description</inv:description>
      	<inv:item>
      	<inv:quantity>67.5</inv:quantity>
      	<inv:unit>h</inv:unit>
      	<inv:description>January 2004</inv:description>
      	<inv:unitprice>400</inv:unitprice>
      	<inv:tax>21.00</inv:tax>
      	</inv:item>
      </inv:invoice>
      <inv:invoice currency="EUR" id="200302-02" paid="true">
        <inv:client id="test">
          <inv:email>jane@efg.com</inv:email>
        </inv:client>
        <inv:date>2003-02-03</inv:date>
        <inv:description>Test description</inv:description>
        <inv:item>
        <inv:quantity>67.5</inv:quantity>
        <inv:unit>h</inv:unit>
        <inv:description>January 2004</inv:description>
        <inv:unitprice>400</inv:unitprice>
        <inv:tax>21.00</inv:tax>
        </inv:item>
      </inv:invoice>
    </inv:invoices>

The desired output is a set of HTML and PDF invoice documents, one pair for each inv:invoice element in the master XML document.

Figure 203. Directory listing of output files

    invoice200302-01.html
    invoice200302-01.pdf
    invoice200302-02.html
    invoice200302-02.pdf

The following resources script does the transformation.

Figure 204. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fop="http://www.servingxml.com/extensions/fop"
                  xmlns:inv="http://www.telio.be/ns/2002/invoice">

      <sx:service id="invoices">
        <sx:transform>
          <sx:document/>
          <!-- Here we extract a document subtree from the SAX stream -->
          <sx:processSubtree path="/inv:invoices/inv:invoice">
            <!-- Main pipeline - invoice document subtree to pdf-->
            <sx:transform>
              <!-- We initialize a parameter with an XPATH expression
                   applied to the document subtree -->
              <sx:parameter name="invoice-name" select="@id"/>
              <fop:foSerializer>
                <sx:fileSink file="output/invoice{$invoice-name}.pdf"/>
              </fop:foSerializer>
              <sx:transform>
                <sx:transform ref="steps1-4"/>
                <!-- Tee - invoice document subtree to html-->
                <sx:tagTee>
                  <sx:xslt documentBase="documents/">
                    <sx:urlSource url="styles/invoice2html.xsl"/>
                  </sx:xslt>
                  <sx:xsltSerializer>
                    <sx:fileSink file="output/invoice{$invoice-name}.html"/>
                  </sx:xsltSerializer>
                </sx:tagTee>
                <sx:xslt documentBase="documents/">
                  <sx:urlSource url="styles/invoice2fo.xsl"/>
                </sx:xslt>
              </sx:transform>
            </sx:transform>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

      <sx:transform id="steps1-4">
        <sx:xslt><sx:urlSource url="styles/step1.xsl"/></sx:xslt>
        <sx:xslt><sx:urlSource url="styles/step2.xsl"/></sx:xslt>
        <sx:xslt><sx:urlSource url="styles/step3.xsl"/></sx:xslt>
        <sx:xslt documentBase="documents/">
          <sx:urlSource url="styles/step4.xsl"/>
        </sx:xslt>
      </sx:transform>

    </sx:resources>

Not the sx:tagTee element, which forks the SAX event stream, and serializes this stream as HTML.

You can run this example on the command line by entering

    servingxml -r resources-invoices.xml -i documents/invoices.xml invoices

### Processing document subtrees and serializing them as mail attachments ("mail invoices")

Continuing with the previous example, suppose you want to send the invoices as pdf mail attachments. Then you need the following resources script.

Figure 205. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fop="http://www.servingxml.com/extensions/fop"
                  xmlns:inv="http://www.telio.be/ns/2002/invoice"
                  xmlns:jm="http://www.servingxml.com/extensions/javamail">

      <sx:service id="invoices">
        <sx:transform>
          <sx:document/>
          <!-- Here we extract a document subtree from the SAX stream -->
          <sx:processSubtree path="/inv:invoices/inv:invoice">
            <sx:parameter name="email" select="inv:client/inv:email"/>

            <!--  We want to send an email with the PDF invoice as an attachment   -->
            <jm:sendMail subject="Invoice" to="{$email}">
              <sx:parameter name="invoice-name" select="@id"/>

              <jm:mailAccount ref="myMailAccount"/>
              <jm:message>
                <jm:attachment filename="{$invoice-name}.pdf">
                  <sx:serialize>
                    <!-- We initialize a parameter with an XPATH expression
                         applied to the document subtree -->
                    <fop:foSerializer/>
                    <sx:transform>
                      <sx:transform ref="steps1-4"/>
                      <sx:xslt documentBase="documents/">
                        <sx:urlSource url="styles/invoice2fo.xsl"/>
                      </sx:xslt>
                    </sx:transform>
                  </sx:serialize>
                </jm:attachment>
              </jm:message>
              <sx:onError>
                <!-- If send mail fails (e.g. no smtp host), just proceed with the next action -->
                <!-- (the error will still be logged) -->
              </sx:onError>
            </jm:sendMail>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

      <sx:transform id="steps1-4">
        <sx:xslt>
          <sx:urlSource url="styles/step1.xsl"/>
        </sx:xslt>
        <sx:xslt>
          <sx:urlSource url="styles/step2.xsl"/>
        </sx:xslt>
        <sx:xslt>
          <sx:urlSource url="styles/step3.xsl"/>
        </sx:xslt>
        <sx:xslt documentBase="documents/">
          <sx:urlSource url="styles/step4.xsl"/>
        </sx:xslt>
      </sx:transform>

      <!-- Set up this mail account, substituting your smtp host, display name, and email address -->
      <!-- You will also need to enter the 'to' and 'cc' recipient addresses in the  jm:sendMail element -->
      <jm:mailAccount id="myMailAccount"
                      smtpHost="smtp1.abc.ca">
        <jm:sender displayName="Daniel"
                     emailAddress="danielaparker@abc.ca"/>
      </jm:mailAccount>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources-mailInvoices.xml -i documents/invoices.xml invoices

### Reading dynamic content

This example shows an implementation of dynamic content handlers.

Running this example requires the following steps:

- Run the sample build script at the command line, to compile the DynamicContentHandler classes and copy them into the distribution classes directory.

- Enter the following command to produce a page of books in the fiction (F) category,

      servingxml -r resources.xml -o output/books.html books category=F

Figure 206. Resources script

    <sx:resources>

      <sx:service id="swath">
        <sx:serialize>
          <sx:transform>
            <sx:content ref="swath"/>
            <sx:xslt ref="swath"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:xslt id="swath">
        <sx:urlSource url="styles/books.xsl"/>
        <sx:parameter name="category"/>
      </sx:xslt>

      <!-- BookCatalog is the name of a Java class used to produce dynamic content for the primary document -->
      <sx:dynamicContent name="swath" class="dynamiccontent.BookCatalog"/>

      <!-- Categories is the name of a Java class used to produce dynamic content in a
            document function in the stylesheet -->
      <sx:dynamicContent name="categories" class="dynamiccontent.Categories"/>

    </sx:resources>

`BookCatalog` and `Categories` are the names of Java classes that implement the `com.servingxml.components.content.dynamic.DynamicContentHandler` interface. The listing below shows the Java code for the `Categories` class.

Figure 207. Categories

    package dynamiccontent;

    public class Categories implements DynamicContentHandler {

      static class Category {
        private final String code;
        private final String description;

        public Category(String code, String description) {
          this.code = code;
          this.description = description;
        }

        public String getCode() {
          return code;
        }

        public String getDescription() {
          return description;
        }
      }

      private final HashMap categoryMap;
      private final String namespace = "";

      public Categories() {
        categoryMap = new HashMap();
        categoryMap.put("S", new Category("S","Science"));
        categoryMap.put("C", new Category("C","Computing"));
        categoryMap.put("CHILD", new Category("CHILD","Children"));
        categoryMap.put("F", new Category("F","Fiction"));
        categoryMap.put("P", new Category("P","Philospohy"));
        categoryMap.put("R", new Category("R","Religion"));
      }

      //  The onRequest method must be reentrant
      public void onRequest(ServiceContext context, Book parameters,
      ContentWriter contentWriter)  {

        contentWriter.startElement(namespace,"categories");

        String id = parameters.getCategory();

        if (id != null && id.length() != 0) {
          if (id.equals("all")) {
            Collection values = categoryMap.values();
            Iterator iter = values.iterator();
            while (iter.hasNext()) {
              Category category = (Category)iter.next();
              writeCategory(category,contentWriter);
            }
          } else {
            Category category = (Category)categoryMap.get(id);
            writeCategory(category,contentWriter);
          }
        }

        contentWriter.endElement(namespace,"categories");
      }

      private void writeCategory(Category category, ContentWriter contentWriter)
       {
        AttributeSet attributes = contentWriter.newAttributeSet();
        attributes.addAttribute(namespace,"code",category.getCode());
        attributes.addAttribute(namespace,"desc",category.getDescription());
        contentWriter.startEndElement(namespace,"category",attributes);
      }
    }

You can run this example on the command line by

- Running the sample build script at the command line, to compile the DynamicContentHandler classes and copy them into the distribution classes directory.
- Entering the following command to produce a page of swath in the fiction (F) category,

      servingxml -r resources.xml -o output/books.html swath category=F

### XML to Mail

The example below illustrates how to prepare a resources script that will transform an XML document to XSL-FO, format it to PDF, and send the result as an attachment in a mail message.

This example requires the mail module.

Once you have everything configured, rebuild the ServingXML jar file in the servingxml directory.

Figure 208. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fop="http://www.servingxml.com/extensions/fop"
                  xmlns:jm="http://www.servingxml.com/extensions/javamail">

      <sx:service id="pulp">
          <!--  Our first action is to send an email with the pulp novel as an attachment   -->
          <jm:sendMail subject="test" to="john@yyy.com" cc="linda@zzz.com">
            <jm:mailAccount ref="myMailAccount"/>
            <jm:message>

              <!-- Put some HTML in the message body -->
              <jm:part type="alternative">
                <sx:transform>
                  <sx:document><xxx/></sx:document>
                  <sx:xslt ref="welcome/html"/>
                </sx:transform>
              </jm:part>

              <jm:attachment filename="pulp.pdf">
                <fop:foSerializer/>
                <sx:transform>
                   <sx:content ref="pulp"/>
                   <sx:xslt ref="novel/fo"/>
                </sx:transform>
              </jm:attachment>
            </jm:message>
            <sx:onError>
                <!-- If send mail fails (e.g. no smtp host), just proceed with the next action -->
                <!-- (the error will still be logged) -->
            </sx:onError>
          </jm:sendMail>

          <!--  Our second action is to write the pulp novel to a file   -->
          <sx:serialize>
            <fop:foSerializer/>
            <sx:transform>
               <sx:content ref="pulp"/>
               <sx:xslt ref="novel/fo"/>
            </sx:transform>
          </sx:serialize>

      </sx:service>

      <!-- Set up this mail account, substituting your smtp host, display name, and email address -->
      <!-- You will also need to enter the 'to' and 'cc' recipient addresses in the  jm:sendMail element -->
      <jm:mailAccount id="myMailAccount"
                               smtpHost="smtp1.abc.ca">
        <jm:sender displayName="DP"
                         emailAddress="danielaparker@abc.ca"/>
      </jm:mailAccount>

      <sx:document id="pulp">
        <sx:urlSource url="data/pulp.xml"/>
      </sx:document>

      <sx:xslt id="novel/fo">
        <sx:urlSource url="data/novel.xsl"/>
      </sx:xslt>

      <sx:xslt id="welcome/html">
        <sx:urlSource url="data/welcome.xsl"/>
      </sx:xslt>

    </sx:resources>

This example produces two outputs:

- A mail message with an HTML body part displaying "Welcome to Presenting Books" and a PDF attachment.

- A PDF file pulp.pdf written to the output directory.

You can run this example on the command line by entering

    servingxml -r resources.xml -o output/pulp.pdf pulp

### Writing an XML file to an FTP sink

This example shows how to send stream output to an FTP host.

Figure 209. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:edt="http://www.servingxml.com/extensions/edtftp">

      <edt:ftpClient id="myFtpSite" host="myFtpHost"
                     user="anonymous" password="xxx"/>

      <sx:service id="books">
        <sx:serialize>
          <sx:xsltSerializer>
            <edt:ftpSink remoteDirectory="incoming" remoteFile="books{$version}.xml">
              <edt:ftpClient ref="myFtpSite"/>
            </edt:ftpSink>
          </sx:xsltSerializer/>

          <sx:transform>
            <sx:content ref="books"/>
          </sx:transform>
        </sx:serialize>
      </sx:service>

      <sx:document id="books">
        <sx:urlSource url="documents/books.xml"/>
      </sx:document>

    </sx:resources>

You can run this example on the command line by

- Updating the resources.xml file with a valid FTP host

- Entering the following command to send an XML file to the FTP host,

      servingxml -r resources.xml books version=1.0

### Comparing Two XML Documents with fn:deep-equal

The example below shows how to compare two XML documents for equality with the XPATH fn:deep-equal function.

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                  xmlns:fn="http://www.w3.org/2005/xpath-functions">

      <sx:service id="compare-documents">
        <sx:task ref="test-documents-are-equal"/>
      </sx:service>

      <sx:document id="odd-element-document">
        <document>
          <title>Title</title>
          <para>First paragraph.</para>
        </document>
      </sx:document>

      <sx:document id="another-odd-element-document">
        <document>
          <title>Title</title>
          <para>First paragraph.</para>
        </document>
      </sx:document>

      <sx:serialize id="test-documents-are-equal">
        <sx:transform>
          <sx:choose>
            <sx:when test="fn:deep-equal(fn:document('odd-element-document'),
                         fn:document('another-odd-element-document'))">
              <sx:document>
                <result>equal</result>
              </sx:document>
            </sx:when>
            <sx:otherwise>
              <sx:document>
                <result>unequal</result>
              </sx:document>
            </sx:otherwise>
          </sx:choose>
        </sx:transform>
      </sx:serialize>

    </sx:resources>

The documents are equal, so the script produces the output <result>equal</result>

You can run this example on the command line by entering

    servingxml -r resources-compare.xml -o output/compare-documents.xml
      compare-documents

## SQL Examples

The java run time needs to be able to find the database driver. The examples in this section are set up for the Oracle driver, oracle.jdbc.driver.OracleDriver. To configure ServingXML to recognize this driver, copy the Oracle classes12.jar file (not distributed) to the `lib/local` directory, and then rebuild. Of course, you can specify a different driver, making the appropriate changes in the sample resources script and copying the right jar file to the `lib/local` directory.

### SQL Query to Flat File

#### Writing the results of a SQL query to a CSV file (employees-csv)

This example shows how to prepare a resources script that will write the results of a SQL query to a CSV file, using a default flat file writer.

The desired CSV file is shown below.

Figure 210. Output XML file employees.csv

    EMPNO,NAME,JOB
    7902,FORD,ANALYST
    7788,SCOTT,ANALYST
    7876,ADAMS,CLERK
    7900,JAMES,CLERK
    7934,MILLER,CLERK
    7369,SMITH,CLERK

The following resources script does the transformation.

Figure 211. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees-to-csv">
        <sx:recordStream>
          <sx:recordReader ref="employees"/>
          <sx:flatFileWriter/>
        </sx:recordStream>
      </sx:service>

      <sx:sqlReader id="employees">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="{$user}"
                            password="{$password}"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `employees` service with two `job` parameters, like this

        servingxml  -r resources-employees_csv.xml -o output/employees.csv employees-to-csv
             user=scott password=spring job=ANALYST job=CLERK

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME,JOB
      FROM EMP
      WHERE 1=1 AND JOB IN ('ANALYST','CLERK')
      ORDER BY JOB,ENAME

The attribute names EMPNO, NAME and JOB become the headings of the CSV file, and the values in each row are written out below.

### SQL Query to XML

#### Mapping the results of a parameterized SQL query into XML (analysts)

The example below illustrates how to prepare a resources script that will convert the results of a SQL query into an XML document.

The desired output is shown below.

Figure 212. Output XML file employees.xml

    <?xml version="1.0" encoding="utf-8"?>
    <employees>
      <employee employee-no="7788">
        <name>SCOTT</name>
        <job>ANALYST</job>
      </employee>
      <employee employee-no="7902">
        <name>FORD</name>
        <job>ANALYST</job>
      </employee>
    </employees>

The following resources script does the transformation.

Figure 213. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees">
        <sx:serialize>
          <sx:content ref="employees"/>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="employees">
        <sx:sqlReader>
          <sx:sqlConnectionPool ref="jdbcPool"/>

          <sx:parameter name="jobList">
            <sx:toString value="{$job}" separator=",">
              <sx:quoteSymbol character="'" escapeCharacter="'"/>
            </sx:toString>
          </sx:parameter>

          <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE JOB = '{$job}'
          </sx:sqlQuery>

        </sx:sqlReader>

        <sx:recordMapping ref="employeesToXml"/>

      </sx:recordContent>

      <sx:recordMapping id="employeesToXml">
        <employees>
          <sx:groupBy fields="JOB">
            <sx:elementMap element="{JOB}">
              <sx:onRecord>
                <employee>
                  <sx:fieldAttributeMap field="EMPNO" attribute="employee-no"/>
                  <sx:fieldElementMap field="NAME" element="name"/>
                </employee>
              </sx:onRecord>
            </sx:elementMap>
          </sx:groupBy>
        </employees>
      </sx:recordMapping>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="{$user}"
                            password="{$password}"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

      </sx:resources>

You can run this example on the command line by entering

      servingxml -r resources.xml employees -o output/employees.txt employees
           user=scott password=spring job=ANALYST

#### Mapping the results of an ad hoc SQL query into XML (employees)

The example below illustrates how to prepare a resources script that will generate and execute a different SQL statement depending on passed parameters.

The desired output is shown below.

Figure 214. Output XML file employees.xml

    <?xml version="1.0" encoding="utf-8"?>
    <employees>
       <ANALYST>
          <employee employee-no="7902">
             <name>FORD</name>
          </employee>
          <employee employee-no="7788">
             <name>SCOTT</name>
          </employee>
       </ANALYST>
       <CLERK>
          <employee employee-no="7876">
             <name>ADAMS</name>
          </employee>
          <employee employee-no="7900">
             <name>JAMES</name>
          </employee>
          <employee employee-no="7934">
             <name>MILLER</name>
          </employee>
          <employee employee-no="7369">
             <name>SMITH</name>
          </employee>
       </CLERK>
    </employees>

The following resources script does the transformation.

Figure 215. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees">
        <sx:serialize>
          <sx:content ref="employeesDoc"/>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="employeesDoc">
        <sx:sqlReader>
          <sx:sqlConnectionPool ref="jdbcPool"/>

          <sx:parameter name="jobList">
            <sx:toString value="{$job}" separator=",">
              <sx:quoteSymbol character="'" escapeCharacter="'"/>
            </sx:toString>
          </sx:parameter>

          <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
            <sx:choose>
              <sx:when test="$jobList">
                AND JOB IN ({$jobList})
              </sx:when>
            </sx:choose>
            ORDER BY JOB,ENAME
          </sx:sqlQuery>

        </sx:sqlReader>

        <sx:recordMapping ref="employeesToXml"/>

      </sx:recordContent>

      <sx:recordMapping id="employeesToXml">
        <employees>
          <sx:groupBy fields="JOB">
            <sx:elementMap element="{JOB}">
              <sx:onRecord>
                <employee>
                  <sx:fieldAttributeMap field="EMPNO" attribute="employee-no"/>
                  <sx:fieldElementMap field="NAME" element="name"/>
                </employee>
              </sx:onRecord>
            </sx:elementMap>
          </sx:groupBy>
        </employees>
      </sx:recordMapping>

      <sx:sqlConnectionPool id="jdbcPool"
                             driver="oracle.jdbc.driver.OracleDriver"
                             databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                             user="scott"
                             password="spring"
                             minConnections="2"
                             testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `employees` service with two `job` parameters, like this

        servingxml -r resources.xml employees -o output/employees.xml employees
            job=ANALYST job=CLERK

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME,JOB
      FROM EMP
      WHERE 1=1 AND JOB IN ('ANALYST','CLERK')
      ORDER BY JOB,ENAME

But executing the `employees` service with no parameters results in a SQL select statement that returns all employees for all job categories.

#### Using multiple readers to join data across different data sources (chained readers)

The example below illustrates how to prepare a resources script that will execute two queries to two different databases, where the second is joined with the first.

The desired output is shown below.

Figure 216. Output XML file deptEmployees.xml

    <?xml version="1.0" encoding="utf-8"?>
    <employees>
      <department dept-no="10" dept-name="ACCOUNTING">
        <employee employee-no="7782">
          <name>CLARK</name>
        </employee>
        <employee employee-no="7839">
          <name>KING</name>
        </employee>
      </department>
      <department dept-no="30" dept-name="SALES">
        <employee employee-no="7499">
          <name>ALLEN</name>
        </employee>
        <employee employee-no="7698">
          <name>BLAKE</name>
        </employee>
      </department>
    </employees>

The following resources script does the transformation.

Figure 217. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees">
        <sx:serialize>
          <sx:content ref="employeesDoc"/>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="employeesDoc">
        <sx:recordStream>
          <sx:sqlReader>
            <sx:sqlConnectionPool ref="devPool"/>
            <sx:sqlQuery recordType = "employee">
            SELECT DEPTNO, DNAME FROM DEPT ORDER BY DNAME
            </sx:sqlQuery>
          </sx:sqlReader>

          <!-- for each row from the previous reader, run this reader -->
          <sx:sqlReader>
            <sx:sqlConnectionPool ref="dev2Pool"/>
            <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB, {DEPTNO} AS DEPTNO, '{DNAME}' AS DEPT FROM EMP  WHERE DEPTNO = {DEPTNO}
            ORDER BY ENAME
            </sx:sqlQuery>
          </sx:sqlReader>
        </sx:recordStream>

        <sx:recordMapping ref="employeesToXml"/>

      </sx:recordContent>

      <sx:recordMapping id="employeesToXml">
        <employees>
          <sx:groupBy fields="DEPT">
            <department>
              <sx:fieldAttributeMap field="DEPTNO" attribute="dept-no"/>
              <sx:fieldAttributeMap field="DEPT" attribute="dept-name"/>
              <sx:onRecord>
                <employee>
                  <sx:fieldAttributeMap field="EMPNO" attribute="employee-no"/>
                  <sx:fieldElementMap field="NAME" element="name"/>
                </employee>
              </sx:onRecord>
            </department>
          </sx:groupBy>
        </employees>
      </sx:recordMapping>

      <sx:sqlConnectionPool id="devPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

      <sx:sqlConnectionPool id="dev2Pool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev2"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `employees` service, like this

        servingxml -r resources-multipleReaders.xml employees -o output/deptEmployees.xml employees

results in the following two SQL statements being sent to the Oracle "dev2" database by the second reader:

    SELECT EMPNO, ENAME AS NAME, 10 AS DEPTNO, 'ACCOUNTING' AS DEPT
      FROM EMP WHERE DEPTNO = 10
      ORDER ENAME

    SELECT EMPNO, ENAME AS NAME, 30 AS DEPTNO, 'SALES' AS DEPT
      FROM EMP WHERE DEPTNO = 30
      ORDER ENAME

### XML to Database

#### Load Master Detail

This example shows how to prepare a resources script that will insert master/detail records into separate master and detail tables as a single transaction.

The input file is an XML file.

Figure 218. Input XML file masterDetail.xml

    <?xml version="1.0" encoding="utf-8"?>
    <masterDetail>
      <master id="1">
        <name>aName</name>
        <detail id="11">
          <date>2008-09-03</date>
          <type>aType</type>
        </detail>
        <detail id="12">
          <date>2008-09-06</date>
          <type>aType</type>
        </detail>
        <detail id="13">
          <date>2008-09-10</date>
          <type>aType</type>
        </detail>
      </master>
      <master id="2">
        <name>anotherName</name>
        <detail id="21">
          <date>2008-10-03</date>
          <type>aType</type>
        </detail>
        <detail id="22">
          <date>2008-10-06</date>
          <type>aType</type>
        </detail>
        <detail id="23">
          <date>2008-10-07</date>
          <type>anotherType</type>
        </detail>
      </master>
    </masterDetail>

The following resources script performs the update.

Figure 219. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="loadMasterDetail">
        <sx:recordStream>
          <sx:subtreeRecordReader>
            <sx:transform>
              <sx:document/>
            </sx:transform>
            <sx:inverseRecordMapping ref="masterDetailMapping"/>
          </sx:subtreeRecordReader>
          <!-- You should validate the record with an sx:recordValidator here -->
          <sx:sqlWriter ref="masterDetailWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can set up a record pipeline here to write bad records to a file or database table -->
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:inverseRecordMapping id="masterDetailMapping">
        <sx:onSubtree path="/masterDetail/master">
          <sx:flattenSubtree recordType="master">
            <sx:subtreeFieldMap select="@id" field="masterId"/>
            <sx:subtreeFieldMap select="name" field="name"/>
            <sx:subtreeFieldMap match="detail" field="details">
              <sx:flattenSubtree recordType="detail">
                <sx:subtreeFieldMap select="@id" field="detailId"/>
                <sx:subtreeFieldMap select="date" field="date"/>
                <sx:subtreeFieldMap select="type" field="type"/>
              </sx:flattenSubtree>
            </sx:subtreeFieldMap>
          </sx:flattenSubtree>
        </sx:onSubtree>
      </sx:inverseRecordMapping>

      <sx:sqlWriter id="masterDetailWriter">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
            INSERT INTO master(master_id,name) VALUES({masterId},'{name}')
          <sx:sqlUpdateDetail field="details">
            <sx:sqlUpdate>
              INSERT INTO detail(detail_id,detail_date,detail_type)
              VALUES({detailId},to_date('{date}','yyyy-mm-dd'),'{type}')
            </sx:sqlUpdate>
          </sx:sqlUpdateDetail>
        </sx:sqlUpdate>
      </sx:sqlWriter>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

The inverse record mapping instructions in the script will produce a stream of composite records of type "master" that contain a repeating group field named "details" having subrecords of type "detail".

Executing the `loadMasterDetail` service, like this,

    servingxml -i data/masterDetail.xml -r resources-masterDetail.xml loadMasterDetail

results in the following update statements being sent to the RDMS and committed to the database:

      INSERT INTO master(master_id,name) VALUES(1,'aName')
      INSERT INTO detail(detail_id,detail_date,detail_type)
      VALUES(11,to_date('2008-09-03','yyyy-mm-dd'),'aType')
      INSERT INTO detail(detail_id,detail_date,detail_type)
      VALUES(12,to_date('2008-09-06','yyyy-mm-dd'),'aType')
      INSERT INTO detail(detail_id,detail_date,detail_type)
      VALUES(13,to_date('2008-09-10','yyyy-mm-dd'),'aType')

The second set of master/detail records fails, however, because the last detail has a value for `detail_type`, "anotherType", that exceeds the length of the column, 10. The following error message is written to the log:

    SEVERE: ORA-12899: value too large for column "SCOTT"."DETAIL"."DETAIL_TYPE"
    (actual: 11, maximum: 10)

### SQL Query to Database

#### SQL inserts

This example shows how to prepare a resources script that will read rows from one database table and write them to another.

The following resources script does the transformation.

Figure 220. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                   xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="loadEmployees">
        <sx:recordStream>
          <sx:sqlReader ref="employeesReader"/>
          <msv:recordValidator>
            <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
              <!-- This element's name matches the value of the name attribute in the sx:sqlQuery element. -->
              <xsd:element name="employee" type="EmployeeType"/>
              <xsd:complexType name="EmployeeType">
                <xsd:sequence>
                  <xsd:element name="EMPNO" type="xsd:integer"/>
                  <xsd:element name="NAME" type="xsd:string"/>
                  <xsd:element name="JOB" type="xsd:string"/>
                </xsd:sequence>
              </xsd:complexType>
            </xsd:schema>
          </msv:recordValidator>

          <sx:sqlWriter ref="employeesWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can include a record pipeline here to write bad records to a file or database table -->
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:sqlWriter id="employeesWriter">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
            INSERT INTO EMP_HISTORY(EMPNO, ENAME) VALUES({EMPNO},'{NAME}')
        </sx:sqlUpdate>
      </sx:sqlWriter>

      <sx:sqlReader id="employeesReader">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `loadEmployees` service with two `job` parameters, like this

        servingxml -r resources-insert_employees.xml loadEmployees
            job=ANALYST job=CLERK

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME,JOB
      FROM EMP
      WHERE 1=1 AND JOB IN ('ANALYST','CLERK')
      ORDER BY JOB,ENAME

These results are then written to the database table `EMP_HISTORY`.

#### Batched SQL inserts

This example shows how to prepare a resources script that will read rows from one database table and write them in batches to another.

The following resources script does the transformation.

Figure 221. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                   xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="loadEmployees">
        <sx:recordStream>
          <sx:sqlReader ref="employeesReader"/>
          <msv:recordValidator>
            <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
              <!-- This element's name matches the value of the name attribute in the sx:sqlQuery element. -->
              <xsd:element name="employee" type="EmployeeType"/>
              <xsd:complexType name="EmployeeType">
                <xsd:sequence>
                  <xsd:element name="EMPNO" type="xsd:integer"/>
                  <xsd:element name="NAME" type="xsd:string"/>
                  <xsd:element name="JOB" type="xsd:string"/>
                </xsd:sequence>
              </xsd:complexType>
            </xsd:schema>
          </msv:recordValidator>

          <sx:sqlWriter ref="employeesWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can include a record pipeline here to write bad records to a file or database table -->
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:sqlBatchWriter id="employeesWriter" batchSize="4">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
            INSERT INTO EMP_HISTORY(EMPNO, ENAME) VALUES( {EMPNO}, '{NAME}')
        </sx:sqlUpdate>
      </sx:sqlBatchWriter>

      <sx:sqlReader id="employeesReader">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `loadEmployees` service with two `job` parameters, like this

        servingxml -r resources-batch_insert_employees.xml loadEmployees
            job=ANALYST job=CLERK

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME,JOB
      FROM EMP
      WHERE 1=1 AND JOB IN ('ANALYST','CLERK')
      ORDER BY JOB,ENAME

These results are then written in batches of 4 to the database table `EMP_HISTORY`.

#### Prepared SQL inserts

This example shows how to prepare a resources script that will read rows from one database table and write them using a prepared SQL statement to another.

The following resources script does the transformation.

Figure 222. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                   xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="loadEmployees">
        <sx:recordStream>
          <sx:sqlReader ref="employeesReader"/>

          <msv:recordValidator>
            <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
              <!-- This element's name matches the value of the name attribute in the sx:sqlQuery element. -->
              <xs:element name="employee" type="EmployeeType"/>
              <xs:complexType name="EmployeeType">
                <xs:sequence>
                  <xs:element name="EMPNO" type="xs:integer"/>
                  <xs:element name="NAME" type="xs:string"/>
                  <xs:element name="JOB" type="xs:string"/>
                  <xs:element name="MGR" type="xs:integer"/>
                  <xs:element name="HIREDATE" type="xs:date"/>
                  <xs:element name="SAL" type="xs:decimal"/>
                  <xs:element name="COMM" type="xs:decimal" minOccurs="0"/>
                  <xs:element name="DEPTNO" type="xs:integer"/>
                </xs:sequence>
              </xs:complexType>
            </xs:schema>
          </msv:recordValidator>

          <sx:sqlWriter ref="employeesWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can include a record pipeline here to write bad records to a database table -->
            <sx:sqlWriter ref="discardWriter"/>
            <sx:discardHandler>
              <sx:log message="{$sx:message}"/>
            </sx:discardHandler>
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:sqlWriter id="employeesWriter" xmlns:xs="http://www.w3.org/2001/XMLSchema">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
          <!-- Because the SQL insert executes a prepared statement, values must be
                associated with appropriate types.  In this example, because these values are
                coming from a SQL query source, they will already be associated with SQL types,
                and no further action is really necessary.  In the case of other data sources, however,
                values by default will be associated with string types, and may need
                to be recast in parameter assignments, as shown below.
          -->
          <sx:parameter name="employee-no" value="{EMPNO}" type="xs:long"/>
          <sx:parameter name="mgr" value="{MGR}" type="xs:long"/>
          <sx:parameter name="hiredate" value="{HIREDATE}" type="xs:date"/>
          <sx:parameter name="salary" value="{SAL}" type="xs:decimal"/>
          <sx:parameter name="commission" value="{COMM}" type="xs:decimal"/>
          <sx:sqlPrepare>
            INSERT INTO EMP_HISTORY(EMPNO, ENAME, JOB, MGR, HIREDATE, SAL, COMM)
            VALUES({$employee-no},{NAME},{JOB}, {$mgr}, {$hiredate},  {$salary}, {$commission})
          </sx:sqlPrepare>
        </sx:sqlUpdate>
      </sx:sqlWriter>

      <sx:sqlWriter id="discardWriter" xmlns:xs="http://www.w3.org/2001/XMLSchema">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
          <sx:parameter name="employee-no" value="{EMPNO}" type="xs:long"/>
          <sx:parameter name="mgr" value="{MGR}" type="xs:long"/>
          <sx:parameter name="hiredate" value="{HIREDATE}" type="xs:date"/>
          <sx:parameter name="salary" value="{SAL}" type="xs:decimal"/>
          <sx:parameter name="commission" value="{COMM}" type="xs:decimal"/>
          <sx:sqlPrepare>
            INSERT INTO EMP_DISCARD(MESSAGE, EMPNO, ENAME, JOB, MGR, HIREDATE, SAL,
                                    COMM)
            VALUES({$message}, {$employee-no}, {NAME}, {JOB}, {$mgr}, {$hiredate},
                   {$salary}, {$commission})
          </sx:sqlPrepare>
        </sx:sqlUpdate>
      </sx:sqlWriter>

      <sx:sqlReader id="employeesReader">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME, JOB, MGR, HIREDATE, SAL, COMM, DEPTNO FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `loadEmployees` service with two `job` parameters, like this

        servingxml -r resources-prepare_insert_employees.xml loadEmployees
            job=SALESMAN job=MANAGER

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME, JOB, MGR, HIREDATE, SAL, COMM, DEPTNO
      FROM EMP
      WHERE 1=1 AND JOB IN ('SALESMAN','MANAGER')
      ORDER BY JOB,ENAME

These results are then written in batches of 4 to the database table `EMP_HISTORY`.

#### Batched prepared SQL inserts

This example shows how to prepare a resources script that will read rows from one database table and write them in batches, using a prepared SQL statement, to another.

The following resources script does the transformation.

Figure 223. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                   xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="loadEmployees">
        <sx:recordStream>
          <sx:sqlReader ref="employeesReader"/>
          <msv:recordValidator>
            <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
              <!-- This element's name matches the value of the name attribute in the sx:sqlQuery element. -->
              <xs:element name="employee" type="EmployeeType"/>
              <xs:complexType name="EmployeeType">
                <xs:sequence>
                  <xs:element name="EMPNO" type="xs:integer"/>
                  <xs:element name="NAME" type="xs:string"/>
                  <xs:element name="JOB" type="xs:string"/>
                  <xs:element name="MGR" type="xs:integer"/>
                  <xs:element name="HIREDATE" type="xs:date"/>
                  <xs:element name="SAL" type="xs:decimal"/>
                  <xs:element name="COMM" type="xs:decimal" minOccurs="0"/>
                  <xs:element name="DEPTNO" type="xs:integer"/>
                </xs:sequence>
              </xs:complexType>
            </xs:schema>
          </msv:recordValidator>

          <sx:sqlWriter ref="employeesWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can include a record pipeline here to write bad records to a file or database table -->
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:sqlBatchWriter id="employeesWriter" xmlns:xs="http://www.w3.org/2001/XMLSchema" batchSize="4">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdate>
          <!-- Because the SQL insert executes a prepared statement, values must be
               associated with appropriate types.  In this example, because these values are
               coming from a SQL query source, they will already be associated with SQL types,
               and no further action is really necessary.  In the case of other data sources, however,
               values by default will be associated with string types, and may need
               to be recast in parameter assignments, as shown below.
          -->
          <sx:parameter name="employee-no" value="{EMPNO}" type="xs:long"/>
          <sx:parameter name="mgr" value="{MGR}" type="xs:long"/>
          <sx:parameter name="hiredate" value="{HIREDATE}" type="xs:date"/>
          <sx:parameter name="salary" value="{SAL}" type="xs:decimal"/>
          <sx:parameter name="commission" value="{COMM}" type="xs:decimal"/>
          <sx:sqlPrepare>
            INSERT INTO EMP_HISTORY(EMPNO, ENAME, JOB, MGR, HIREDATE, SAL, COMM)
            VALUES({$employee-no},{NAME},{JOB}, {$mgr}, {$hiredate},  {$salary}, {$commission})
          </sx:sqlPrepare>
        </sx:sqlUpdate>
      </sx:sqlBatchWriter>

      <sx:sqlReader id="employeesReader">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME, JOB, MGR, HIREDATE, SAL, COMM, DEPTNO FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `loadEmployees` service with two `job` parameters, like this

        servingxml -r resources-batch_prepare_insert_employees.xml loadEmployees
            job=SALESMAN job=MANAGER

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME, JOB, MGR, HIREDATE, SAL, COMM, DEPTNO
      FROM EMP
      WHERE 1=1 AND JOB IN ('SALESMAN','MANAGER')
      ORDER BY JOB,ENAME

These results are then written in batches of 4 to the database table `EMP_HISTORY`, using prepared SQL statements.

#### Updating an employee record using command-line parameters (employee update)

The example below illustrates how to prepare a resources script that will change an employee job categorization using command line parameters.

Figure 224. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="updateEmployee">
        <sx:recordStream>
          <sx:parameterReader recordType="employee"/>
          <sx:sqlWriter>
            <sx:sqlConnectionPool ref="jdbcPool"/>
            <sx:sqlUpdate>
              UPDATE EMP SET JOB = '{$job}' WHERE EMPNO='{$empNo}'
            </sx:sqlUpdate>
          </sx:sqlWriter>
        </sx:recordStream>
      </sx:service>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

You can run this example on the command line by entering

    servingxml -r resources.xml -o updateEmployee
        empNo=7499 job=ANALYST

#### Insert if record does not exist, update otherwise (insert/update).

This example shows how to prepare a resources script that will read rows from one database table and write them to another, inserting if the record does not already exist, updating otherwise.

The following resources script does the transformation.

Figure 225. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core"
                   xmlns:msv="http://www.servingxml.com/extensions/msv">

      <sx:service id="loadEmployees">
        <sx:recordStream>
          <sx:sqlReader ref="employeesReader"/>
          <msv:recordValidator>
            <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
              <!-- This element's name matches the value of the name attribute in the sx:sqlQuery element. -->
              <xs:element name="employee" type="EmployeeType"/>
              <xs:complexType name="EmployeeType">
                <xs:sequence>
                  <xs:element name="EMPNO" type="xs:integer"/>
                  <xs:element name="NAME" type="xs:string"/>
                  <xs:element name="JOB" type="xs:string"/>
                </xs:sequence>
              </xs:complexType>
            </xs:schema>
          </msv:recordValidator>

          <sx:sqlWriter ref="employeesWriter"/>
          <sx:discardHandler>
            <sx:log message="{$sx:message}"/>
            <!-- You can include a record pipeline here to write bad records to a file or database table -->
          </sx:discardHandler>
        </sx:recordStream>
      </sx:service>

      <sx:sqlWriter id="employeesWriter">
        <sx:sqlConnectionPool ref="jdbcPool"/>
        <sx:sqlUpdateChoice>
          <sx:sqlQuery>
            SELECT EMPNO FROM EMP_HISTORY WHERE EMPNO = {EMPNO}
          </sx:sqlQuery>
          <sx:recordNotFound>
            <sx:sqlUpdate>
              INSERT INTO EMP_HISTORY(EMPNO, ENAME) VALUES({EMPNO},'{NAME}')
            </sx:sqlUpdate>
          </sx:recordNotFound>
          <sx:recordFound>
            <sx:sqlUpdate>
              UPDATE EMP_HISTORY SET ENAME = '{NAME}' WHERE EMPNO = {EMPNO}
            </sx:sqlUpdate>
          </sx:recordFound>
        </sx:sqlUpdateChoice>
      </sx:sqlWriter>

      <sx:sqlReader id="employeesReader">
        <sx:sqlConnectionPool ref="jdbcPool"/>

        <sx:parameter name="jobList">
          <sx:toString value="{$job}" separator=",">
            <sx:quoteSymbol character="'" escapeCharacter="'"/>
          </sx:toString>
        </sx:parameter>

        <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
          <sx:choose>
            <sx:when test="$jobList">
                AND JOB IN ({$jobList})
            </sx:when>
          </sx:choose>
            ORDER BY JOB,ENAME
        </sx:sqlQuery>

      </sx:sqlReader>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Execute the `loadEmployees` service with two `job` parameters, like this

        servingxml -r resources-insertUpdate.xml loadEmployees
            job=ANALYST job=CLERK

#### Writing rows to a database table as they pass through a pipe, to an XML file (employees-tee)

This example shows how to prepare a resources script that will read rows from an RDBMS, write them to a database table as they pass through the pipe, and finally produce an XML file.

The desired XML file is shown below.

Figure 226. Output XML file employees.xml

    <?xml version="1.0" encoding="utf-8"?>
    <employees>
       <ANALYST>
          <employee employee-no="7902">
             <name>FORD</name>
          </employee>
          <employee employee-no="7788">
             <name>SCOTT</name>
          </employee>
       </ANALYST>
       <CLERK>
          <employee employee-no="7876">
             <name>ADAMS</name>
          </employee>
          <employee employee-no="7900">
             <name>JAMES</name>
          </employee>
          <employee employee-no="7934">
             <name>MILLER</name>
          </employee>
          <employee employee-no="7369">
             <name>SMITH</name>
          </employee>
       </CLERK>
    </employees>

The following resources script does the transformation.

Figure 227. Resources script

    <sx:resources xmlns:sx="http://www.servingxml.com/core">

      <sx:service id="employees">
        <sx:serialize>
          <sx:content ref="employeesDoc"/>
        </sx:serialize>
      </sx:service>

      <sx:recordContent id="employeesDoc">
        <sx:sqlReader>
          <sx:sqlConnectionPool ref="jdbcPool"/>

          <sx:parameter name="jobList">
            <sx:toString value="{$job}" separator=",">
              <sx:quoteSymbol character="'" escapeCharacter="'"/>
            </sx:toString>
          </sx:parameter>

          <sx:sqlQuery recordType = "employee">
            SELECT EMPNO, ENAME AS NAME,JOB FROM EMP WHERE 1=1
            <sx:choose>
              <sx:when test="$jobList">
                AND JOB IN ({$jobList})
              </sx:when>
            </sx:choose>
            ORDER BY JOB,ENAME
          </sx:sqlQuery>
        </sx:sqlReader>

        <sx:recordTee>
          <sx:sqlWriter>
            <sx:sqlConnectionPool ref="jdbcPool"/>
            <sx:sqlUpdate>
                INSERT INTO EMP_LOG(EMPNO, ENAME) VALUES({EMPNO},'{NAME}')
            </sx:sqlUpdate>
          </sx:sqlWriter>
        </sx:recordTee>

        <sx:recordMapping ref="employeesToXml"/>
      </sx:recordContent>

      <sx:recordMapping id="employeesToXml">
        <employees>
          <sx:groupBy fields="JOB">
            <sx:elementMap element="{JOB}">
              <sx:onRecord>
                <employee>
                  <sx:fieldAttributeMap field="EMPNO" attribute="employee-no"/>
                  <sx:fieldElementMap field="NAME" element="name"/>
                </employee>
              </sx:onRecord>
            </sx:elementMap>
          </sx:groupBy>
        </employees>
      </sx:recordMapping>

      <sx:sqlConnectionPool id="jdbcPool"
                            driver="oracle.jdbc.driver.OracleDriver"
                            databaseUrl="jdbc:oracle:thin:@127.0.0.1:1521:dev"
                            user="scott"
                            password="spring"
                            minConnections="2"
                            testStatement="SELECT * FROM DUAL"/>

    </sx:resources>

Executing the `employees` service with two `job` parameters, like this

        servingxml -r resources-employeesTee.xml employees -o output/employees.xml employees
            job=ANALYST job=CLERK

results in the following SQL statement being sent to the Oracle server:

    SELECT EMPNO, ENAME AS NAME,JOB
      FROM EMP
      WHERE 1=1 AND JOB IN ('ANALYST','CLERK')
      ORDER BY JOB,ENAME

Each `EMPNO` and `ENAME` pair is written to the database table `EMP_LOG`, and all rows are ultimately mapped to the XML file `employees.xml`.

## Index

### B

batch
  records, batched XML, batched flat file
  XML, serializing batches, batched XML

### C

CDATA, opdef

### D

date
  convert to ISO 8601 format, trades (positional) to XML
  today, ars (delimited) to XML
date/time
  sx:currentDateTime, optional elements
  sx:formatDateTime, optional elements
delimited field
  max width, field has max width
directory
  reading, books directory to XML, book directory to new format, remote directories
  writing, books directory to XML, book directory to new format, document subtrees to PDF files
dynamic content, custom SAX reader, dynamic content

### E

edi, edi to XML, Invoice96A
encoding
  ebcdic, ascii-ebcidc, ascii-packed

### F

field, transaction
field delimiter
  comma, countries-default
  numerical entity references, tab-delimited to XML
  pipe, family_data (multi-valued) to XML
  tab, tab-delimited to XML
field substitutions, field substitutions
file integrity
  crc, checked books
  size, checked books
filter
  XSLT, element sequence to flat
flat file
  CSV, countries-default, countries-custom, countries-validated, tasks (CSV) to XML, timesheets (CSV) to XML (grouped), books (XML) to flat file (CSV)
  delimited, family_data (multi-valued) to XML, ars (delimited) to XML, exotics (delimited) to XML (grouped), books (XML) to flat file (delimited), books (XML) to listing, book hierarchy
  Java properties, properties to XML, properties to XML (grouped), properties (muti-valued) to XML (grouped)
  multi-valued fields, family_data (multi-valued) to XML, books (XML) to flat file (delimited), books (XML) to listing
  multiple readers, multiple-readers
  multiple record format, exotics (delimited) to XML (grouped), trades (positional) to XML, non-delimited trades to XML
  no end-of-line delimiters, non-delimited book orders to XML, non-delimited trades to XML
  positional, trades (positional) to XML, conv (positional) to XML, hot (preliminaries), hot (positional) to XML (grouped), books (XML) to flat file (positional), ungrouping, all, blocks, block subsequences, books (positional) to books (delimited), books (positional) to books (positional), declarative record filter, checked books, book directory to new format, batch records
    variable width fields, variable field widths
  record composition, record composition
  segment concatenation, smf
  single record format, conv (positional) to XML, non-delimited book orders to XML
flat file field
  binary, special char to XML, XML to special char
  integer, smf
  packed decimal, packed decimal to XML, smf
flat file trailer
  recordCount, books (XML) to flat file (positional)
flat file writer
  default, employees csv
flat to XML
  repeat header in XML, repeat header in XML
fo, document subtrees to PDF files, document subtrees to mail
FpML
  FRA, FRA
  Swap, Swap
FTP
  sink, FTP sink
  source, remote directories

### G

group
  by, tasks (CSV) to XML, timesheets (CSV) to XML (grouped), ars (delimited) to XML, exotics (delimited) to XML (grouped), properties to XML (grouped), properties (muti-valued) to XML (grouped)
  inner, conv (positional) to XML, hot (positional) to XML (grouped)
  outer, hot (positional) to XML (grouped)

### I

inverse record mapping
  header-detail records, XML-flat header-detail
  match, swath (XML) to flat file (delimited), XML-flat header-detail
  multi-valued fields, swath (XML) to flat file (delimited)

### M

mail
  attachment, document subtrees to mail, XML to mail

### O

optional elements, optional elements

### P

parameter
  default value, trades (positional) to XML
  within groups, repeat header in XML
pdf, document subtrees to PDF files, document subtrees to mail

### R

record
  composite, aggregates over records, two records to one element
record delimiter
  continuation, properties to XML, properties to XML (grouped)
  start/end, opdef, transaction
record filter
  custom, hot (preliminaries)
  schema validation, countries-validated, hot (preliminaries)
  sx:replaceRecord, declarative record filter
record filters
  sx:choose, books (XML) to flat file (delimited), books (XML) to listing
record mapping
  default, countries-default
record reader
  custom, custom record reader, multiple output files
record writer
  multiple output files, multiple output files
records
  reordering, multiple summaries
regular expressions
  match and replace, books directory to XML, trades (positional) to XML, declarative record filter, book directory to new format
  search and replace, ars (delimited) to XML
repeating groups, edi to XML, Invoice96A, segments
  delimited, students-delim, invoice
  fixed, students, all
  nested, invoice
repeating segment, transaction

### S

special char, special char to XML, XML to special char
SQL
  batch, Batched inserts, Batched prepared inserts
  INSERT, SQL inserts, Batched inserts, SQL prepare, Batched prepared inserts
    master/detail, Master Detail
  INSERT/UPDATE, insert/update
  PREPARE, SQL prepare
  prepare, Batched prepared inserts
SQL query, employees csv
SQL reader
  chained, Using multiple readers to join data across different data sources (chained readers)

### T

tee
  record, employees tee
  tag, document subtrees to PDF files

### U

UTF-8
  double byte, UTF-8 double byte

### X

X12, X12 XML to Flat File
XML files
  default namespace, multiple default ns
  multiple, multiple XML book files, multiple default ns
XML input
  multiple documents, aggregates over records, eobclaims to flat file
XML output
  multiple documents, eobclaims to XMLs
XSLT extensions
  using, hot (positional) to XML (grouped)