# Upgrading to 1.0

## Upgrading from versions 0.8 and 0.9

### Breaking Changes to Markup

#### sx:wrap

The sx:wrap element is used to extract a sequence of subtrees from XML documents and wrap them with enclosing tags. It may, for example, be used to extract the fiction titles from a book catalog file and produce the output

    <envelope>
      <header mode="xml">
        <creationDate>2008-12-29T23:42:59.168-05:00</creationDate>
      </header>
      <body>
        <myns:books xmlns:myns="http://mycompany.com/mynames/"
                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                    xsi:schemaLocation="url2">
          <myns:book categoryCode="F">
            <myns:title>Factotum</myns:title>
            <myns:author>Charles Bukowski</myns:author>
            <myns:price>22.95</myns:price>
            <myns:isbn>0876852630</myns:isbn>
          </myns:book>
          <myns:book categoryCode="F">
            <myns:title>The Night Watch</myns:title>
            <myns:author>Sergei Lukyanenko</myns:author>
            <myns:price>17.99</myns:price>
            <myns:isbn>0434014125</myns:isbn>
          </myns:book>
          <myns:book categoryCode="F">
            <myns:title>Mr Mee</myns:title>
            <myns:author>Andrew Crumey</myns:author>
            <myns:price>22.00</myns:price>
            <myns:isbn>0312282354</myns:isbn>
          </myns:book>
        </myns:books>
      </body>
      <trailer>This is a trailer element</trailer>
    </envelope>

The `sx:wrap` instruction to produce this content is shown below for earlier versions and for 1.0.

Versions 0.8-0.9 | Version 1.0
--- | ---
      <sx:wrap>
        <sx:xsltSerializer/>
        <sx:document/>
        <envelope>
          <header>
            <sx:attribute name="mode" value="xml"/>
            <creationDate>
              <sx:currentDateTime/>
            </creationDate>
          </header>
          <body>
            <myns:books xmlns:myns="http://mycompany.com/mynames/"
                        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                        xsi:schemaLocation="url2">

              <!-- Subtrees of sx:document declared above -->
              <sx:processSubtree path="/myns:books/myns:book">
                <sx:choose>
                  <sx:when test="@categoryCode='F'">
                    <sx:transform/>
                  </sx:when>
                </sx:choose>
              </sx:processSubtree>

            </myns:books>
          </body>
          <trailer>This is a trailer element</trailer>
        </envelope>
      </sx:wrap>
      <sx:wrap>
        <sx:xsltSerializer/>
        <envelope>
          <header>
            <sx:attribute name="mode" value="xml"/>
            <creationDate>
              <sx:currentDateTime/>
            </creationDate>
          </header>
          <body>
            <myns:books xmlns:myns="http://mycompany.com/mynames/"
                        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                        xsi:schemaLocation="url2">

              <!-- Content to be wrapped -->
              <sx:transform>
                <sx:document/>
                <sx:processSubtree path="/myns:books/myns:book">
                  <sx:choose>
                    <sx:when test="@categoryCode='F'">
                      <sx:transform/>
                    </sx:when>
                  </sx:choose>
                </sx:processSubtree>
              </sx:transform>

            </myns:books>
          </body>
          <trailer>This is a trailer element</trailer>
        </envelope>
      </sx:wrap>

The earlier version is a bit mixed up, with the `sx:document` element that loads the books catalog document appearing outside the tags that will surround its subtrees. The 1.0 version is more natural, with the enclosing tags completely surrounding the content to be enclosed.

### Non-Breaking Changes to Element and Attribute Names

The names of a number of elements and attributes have changed over the 0.8 and 0.9 releases, these are collected in the table below. Note that the old names are retained as aliases to the new names, so these are non-breaking changes.

Version Changed | Old Element Name | Ver 1.0 Element Name | Old Attribute Name | Ver 1.0 Attribute Name
--- | --- | --- | --- | ---
1.0.0 | `sx:xmlRecordReader` | sx:subtreeRecordReader |  |
1.0.0 | `sx:defaultFieldMapping` | sx:defaultFieldElementMap |  |
1.0.0 | `sx:spannedFlatRecordType` | sx:vbsFlatRecordType |  |
1.0.0 | `sx:escapeVariables` | sx:escapeSubstitutions |  |
1.0.0 | `sx:runService` | sx:runService | `service` | `serviceRef`
0.9.5 | `sx:subrecordMapping` | sx:subrecordMapping | `field` | `repeatingGroup`
0.9.4 | `sx:composeRecord` | sx:composeRecord | `compositeRecordType` | `recordType`
|  |  | `repeatingGroupField` | `repeatingGroup`
0.9.4 | `sx:decomposeRecord` | sx:decomposeRecord | `compositeRecordType` | `recordType`
|  |  | `repeatingGroupField` | `repeatingGroup`
0.9.3 | `sx:segmentMapping` | sx:subrecordMapping |  |
0.9.3 | `sx:insertContent` | sx:nestedContent |  |
0.9.2 | `sx:replace` | sx:findAndReplace |  |
0.9.1 | `sx:quoteSymbol` | sx:quoteSymbol | value | character
|  |  | escapedBy | escapeCharacter
|  |  | escapeWith | escapeSequence
0.9.0 | `sx:tagDelimiter` | sx:nameDelimiter |  |
0.9.0 | `sx:taggedDelimitedField` | sx:delimitedNamedField |  |
0.9.0 | `sx:repeatingTaggedField` | sx:repeatingField |  |
0.8.3 | `sx:defaultFieldMapping` | sx:defaultFieldElementMap | except | exceptFields
0.8.3 | `sx:removeEmptyElements` | sx:removeEmptyElements | except | exceptElements
0.8.3 | `sx:batchRecords` | sx:batchedRecordWriter |  |

## Upgrading from versions 0.7

### Changes to Distribution

- Java SE 5.0 or later is now required to build `ServingXML`, and a Java 5.0 or later runtime environment is now required to run `ServingXML`. A number of libraries previously included to fill holes in Java 1.4, including `concurrent.jar`, `xercesImpl.jar`, and `xml-apis.jar`, are no longer required and have been dropped from the distribution.

- Building `ServingXML` requires Apache ANT, which must now be installed separately, it is no longer included in the distribution.

- The default XSLT transformer accompanying the download is now the XSLT 2.0 transformer `Saxon-B 9`, replacing the XSLT 1.0 transformer `Saxon 6.5.5`. This means that all XSLT stylesheets and all XPATH expressions used in ServingXML will by default be processed with an XSLT 2 transformer. It is possible to configure ServingXML to use a different transformer, in particular it is possible to revert to `Saxon 6.5.5`.

- The servingxml-fop extension now wraps Apache FOP version 0.94.

- Previous documentation described running the `ServingXML` console app with Java's -jar option. While still supported, it is now more convenient to use the new batch file `servingxml.bat` (Windows) or shell script `servingxml` (Unix/Linux). These command files are based on `ant`/`fop` models, and build the classpath dynamically from the lib directory.

- In previous versions of servingxml, the binary distribution was built in a directory named `deploy`, this has been changed to `target`.

- In recent versions of servingxml, the samples were placed directly under `deploy`, this has been changed to `target/servingxml`.

### Changes to Markup

#### Breaking Changes

##### sx:fieldElementMap and sx:fieldElementSequenceMap

In previous versions, the `sx:fieldElementMap` element performed two functions: it mapped single-valued fields to a single element, and multi-valued fields to a sequence of elements. The problem is that the desired behaviour for mapping empty values differs in the two cases. When mapping a single-valued field to an element, the desired behaviour is to preserve the empty element. When mapping an empty multi-valued field to a sequence of elements, the desired behaviour is not to emit any elements. For this reason, the `sx:fieldElementMap` element has been restricted to mapping to a single element value, and a new `sx:fieldElementSequenceMap` element has been introduced to map to a sequence of elements.

father_name | mother_name | children
--- | --- | ---
Matthew | Sarah |
Scott |  | Damian;Janet;Paul

Version 0.7.* | Version 1.0
--- | ---
      <sx:recordMapping id="families-to-xml-mapping">
        <families>
          <sx:onRecord>
            <family>
              <sx:fieldElementMap field="father_name"
                                  element="father-name"/>
              <sx:fieldElementMap field="mother_name"
                                  element="mother-name"/>
              <children>
                <sx:fieldElementMap field="children" element="child"/>
              </children>
            </family>
          </sx:onRecord>
        </families>
      </sx:recordMapping>
      <sx:recordMapping id="families-to-xml-mapping">
        <families>
          <sx:onRecord>
            <family>
              <sx:fieldElementMap field="father_name"
                                  element="father-name"/>
              <sx:fieldElementMap field="mother_name" element="mother-name"/>
              <children>
                <sx:fieldElementSequenceMap field="children"
                                  element="child"/>
              </children>
            </family>
          </sx:onRecord>
        </families>
      </sx:recordMapping>
    <families>
      <family>
        <father>Matthew</father>
        <mother>Sarah</mother>
        <children></children>
      </family>
      <family>
        <father>Scott</father>
        <children>
          <child>Damian</child>
          <child>Janet</child>
          <child>Paul</child>
        </children>
      </family>
    </families>
    <families>
      <family>
        <father>Matthew</father>
        <mother>Sarah</mother>
        <children></children>
      </family>
      <family>
        <father>Scott</father>
        <mother/>
        <children>
          <child>Damian</child>
          <child>Janet</child>
          <child>Paul</child>
        </children>
      </family>
    </families>

Note that in the older version, the element corresponding to the empty `mother` field value is suppressed. In the 1.0 version, it is retained.

##### Empty sx:document element required to receive default stream source as SAX source

In previous versions, in `sx:transform` sections, if no content was explicitly specified, the content defaulted to parsing the default input stream, such as a file specified with the -i option on the console app. As of version 1.0, an empty `sx:document` element must be specified to achieve this result, otherwise content defaults to an empty document. In fact, almost all of the examples in previous ServingXML distributions already used an empty `sx:document` element, the only exceptions being the invoice examples in the XML-to-XML exceptions, these needed to be modified as shown below.

Version 0.7.* | Version 1.0
--- | ---
      <sx:service id="invoices">
        <sx:transform>

          <!-- Here we extract a document subtree from the SAX stream -->
          <sx:processSubtree path="/inv:invoices/inv:invoice">
            <sx:transform>
              ...
            </sx:transform>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>
      <sx:service id="invoices">
        <sx:transform>
          <sx:document/>
          <!-- Here we extract a document subtree from the SAX stream -->
          <sx:processSubtree path="/inv:invoices/inv:invoice">
            <sx:transform>
              ...
            </sx:transform>
          </sx:processSubtree>
        </sx:transform>
      </sx:service>

##### The output of the sx:processSubtree element is no longer the same as the input.

Like previous versions, 1.0 allows you to use an `sx:processSubtree` element to process subtrees of an XML document, and to serialize the subtrees as individual XML files. Previous versions, however, allowed you to simultanously output the entire XML file by enclosing the whole thing inside an `sx:serialize` element, because the original SAX events flowed through the `sx:processSubtree` element. In 1.0, this is no longer the case, and to achieve the same effect you need to wrap the `sx:processSubtree` element inside an `sx:tagTee` element, as shown below.

Version 0.7.* | Version 1.0
--- | ---
    <sx:service id="books">
      <sx:serialize>
        <sx:transform>
          <sx:content ref="books"/>

            <sx:processSubtree path="/myns:books/myns:book">
              <sx:parameter name="isbn" select="myns:isbn"/>
              <sx:serialize>
                <sx:xsltSerializer>
                  <sx:fileSink directory="output"
                               file="book-{$isbn}.xml"/>
                </sx:xsltSerializer>
              </sx:serialize>
            </sx:processSubtree>

        </sx:transform>
      </sx:serialize>
    </sx:service>
    <sx:service id="books">
      <sx:serialize>
        <sx:transform>
          <sx:content ref="books"/>
          <sx:tagTee>
            <sx:processSubtree path="/myns:books/myns:book">
              <sx:parameter name="isbn" select="myns:isbn"/>
              <sx:serialize>
                <sx:xsltSerializer>
                  <sx:fileSink directory="output"
                               file="book-{$isbn}.xml"/>
                </sx:xsltSerializer>
              </sx:serialize>
            </sx:processSubtree>
          </sx:tagTee>
        </sx:transform>
      </sx:serialize>
    </sx:service>

The 1.0 behaviour is that the output of `sx:processSubtree` is whatever tags are written in its content to the default SAX sink. In the example above, that is none, because all the tags are written to explicitly specified files. But if no explicit output is specified, the subtrees themselves will be written to the default SAX sink. Then, if there is an enclosing `sx:wrap` element, the subtrees will be aggregated and output as a single XML document, with new wrapping tags. See the XML-to-XML example "Extracting subtrees and wrapping them in containing tags" for more about the new `sx:wrap` element.

##### Removal of deprecated sx:startGroup and sx:endGroup elements

Up to version 0.6.5, the `sx:innerGroup` and `sx:outerGroup` elements took `sx:startGroup` and `sx:endGroup` child elements to control breaking behaviour, these elements were deprecated in the 0.6.5 release and replaced by `startTest` and `endTest` attributes. As of version 1.0, `sx:startGroup` and `sx:endGroup` have been removed altogether.

Pre 0.6.5 versions | Version 1.0
--- | ---
    <sx:outerGroup>
      <sx:startGroup test="sx:previous//record-type='BFH01'"/>
      <sx:endGroup test="sx:next//record-type='BFT99'"/>
      <!-- group content -->
    </sx:outerGroup>
    <sx:outerGroup startTest="sx:previous//record-type='BFH01'"
                   endTest="sx:current//record-type='BFT99'">
      <!-- group content -->
    </sx:outerGroup>

#### Non-Breaking Changes

- `sx:regexFieldCriteria` has been renamed to `sx:fieldRestriction`, and its attribute `match` has been renamed to `pattern`. For backwards compatability, the old names are still allowed, but deprecated..
- `sx:recordTest` has been renamed to `sx:recordRestriction`. For backwards compatability, the old name is still allowed, but deprecated..

Version 0.7.* | Version 1.0
--- | ---
    <sx:restrictRecordFilter>
      <sx:regexFieldCriteria field="name"
                             match="books.*[.]txt"/>
    </sx:restrictRecordFilter>
    <sx:restrictRecordFilter>
      <sx:fieldRestriction field="name"
                                  pattern="books.*[.]txt"/>
    </sx:restrictRecordFilter>
    <sx:restrictRecordFilter>
      <sx:recordTest recordType="persons"/>
    </sx:restrictRecordFilter>
    <sx:restrictRecordFilter>
      <sx:recordRestriction recordType="persons"/>
    </sx:restrictRecordFilter>

- `sx:removeEmptyElementFilter` has been renamed to `sx:removeEmptyElements`. For backwards compatability, the old name is still allowed, but deprecated..

Version 0.7.* | Version 1.0
--- | ---
      <sx:transform>
        <sx:content ref="families"/>
        <sx:removeEmptyElementFilter elements="children"/>
      </sx:transform>
      <sx:transform>
        <sx:content ref="families"/>
        <sx:removeEmptyElements elements="children"/>
      </sx:transform>

- In `sx:directoryReader` and `edt::ftpDirectoryReader`, the field `parentDir` has been renamed to `parentDirectory`. For backwards compatability, the old name is still retained, but deprecated..

Version 0.7.* | Version 1.0
--- | ---
    <sx:recordStream>
      <sx:directoryReader directory="data"/>
      <sx:processRecord>
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:fileSource directory="{parentDirectory}"
                           file="{name}"/>
    <sx:recordStream>
      <sx:directoryReader directory="data"/>
      <sx:processRecord>
        <sx:recordStream>
          <sx:flatFileReader>
            <sx:fileSource directory="{parentDirectory}"
                           file="{name}"/>

- `sx:wrap` has been renamed to `sx:nestedContent` (since 0.8.1.) For backwards compatability, the old name is still allowed, but deprecated.

Version 0.7.* | Version 1.0
--- | ---
      <sx:recordMapping id="personsAddressesMapping">
        <Persons-Addresses>
          <sx:wrap>
            <sx:recordContent>
              <sx:flatFileReader ref="personsReader"/>
              <sx:recordMapping ref="personsMapping"/>
            </sx:recordContent>
          </sx:wrap>
          <sx:wrap>
            <sx:recordContent>
              <sx:flatFileReader ref="personsReader"/>
              <sx:recordMapping ref="addressesMapping"/>
            </sx:recordContent>
          </sx:wrap>
        </Persons-Addresses>
      </sx:recordMapping>
      <sx:recordMapping id="personsAddressesMapping">
        <Persons-Addresses>
          <sx:nestedContent>
            <sx:recordContent>
              <sx:flatFileReader ref="personsReader"/>
              <sx:recordMapping ref="personsMapping"/>
            </sx:recordContent>
            <sx:recordContent>
              <sx:flatFileReader ref="personsReader"/>
              <sx:recordMapping ref="addressesMapping"/>
            </sx:recordContent>
          </sx:nestedContent>
        </Persons-Addresses>
      </sx:recordMapping>

- Boolean values appearing in ServingXML mark-up were previously specified as `yes`|`no`, that has been changed to `true`|`false`. For backwards compatability, `yes`|`no` are still allowed, but deprecated.

Version 0.7.* | Version 1.0
--- | ---
    <sx:delimitedField name="name"
                       trimLeading="yes"/>
    <sx:delimitedField name="name"
                       trimLeading="true"/>
      <edt:ftpDirectoryReader remoteDirectory="."
                              recurse="yes"
                              maxItems="10">
      <edt:ftpDirectoryReader remoteDirectory="."
                              recurse="true"
                              maxItems="10">
      <sx:sqlQuery recordType="foo" trim="yes">
      <sx:sqlQuery recordType="foo" trim="true">
      <sx:flatRecordType name="invoice" omitFinalRepeatDelimiter="no">
      <sx:flatRecordType name="invoice" omitFinalRepeatDelimiter="false">

Note that in the sx:outputProperty elements that supply output property values to the XSLT serializer, the XSLT names are used so it's still, for example,

    <sx:xsltSerializer>
      <sx:outputProperty name="indent" value="yes"/>
    </sx:xsltSerializer>

### Changes to Java Code

The two most noticable changes to the source code are as follows:

- Methods that previously took the three arguments

        ServiceContext context, Record parameters, Flow flow

  now take only two

        ServiceContext context, Flow flow

- The `ServingXmlException` now extends `RuntimeException`, and method signatures no longer have explicit throw statements for `ServingXmlException`.

Parameters are inherently associated with the `flow` value, they must travel together, and passing them separately means that code that buffers `flow` values must really buffer the pair `parameters, flow`. This code becomes simpler and less error prone by moving `parameters` inside `flow`, and making them accessible with a `getParameters` method.

Regarding the change of `ServingXmlException` from a checked exception to a `RuntimeException`, the author of this software has become convinced by the arguments of Java's checked exceptions were a mistake , Exceptional Java, Exception Safe Java and elsewhere that making `ServingXMLException` a checked exception was a mistake. The biggest problem was when `ServingXMLException` had to be thrown from within a `Java` library callback method such as `compare`.

#### Changes to the Embedding API

Version 0.7.* | Version 1.0
--- | ---
    import com.servingxml.io.flow.Flow;
    import com.servingxml.io.flow.FlowImpl;
    import com.servingxml.app.Flow;
    import com.servingxml.app.FlowImpl;
    Flow flow =  new FlowImpl(defaultStreamSource,
                              defaultStreamSink);
    Flow flow =  new FlowImpl(parameters,
                              defaultStreamSource,
                              defaultStreamSink);
    service.execute(context, parameters, flow);
    service.execute(context, flow);

#### Changes to Java Custom Record Filters

Version 0.7.* | Version 1.0
--- | ---
    public void startRecordStream(ServiceContext context,
                                  Record parameters,
                                  Flow flow)
    throws ServingXmlException
    public void startRecordStream(ServiceContext context,
                                  Flow flow)
    public void writeRecord(ServiceContext context,
                            Record parameters,
                            Flow flow)
    throws ServingXmlException
    public void writeRecord(ServiceContext context,
                            Flow flow)
    public void writeRecord(ServiceContext context,
                            Record parameters,
                            Flow flow,
                            Record record)
    throws ServingXmlException
    public void endRecordStream(ServiceContext context,
                                Record parameters,
                                Flow flow)
    throws ServingXmlException
    public void endRecordStream(ServiceContext context,
                                Flow flow)

Version 0.7.* | Version 1.0
--- | ---
    public class HotRecordFilter extends AbstractRecordFilter {

      public void writeRecord(ServiceContext context,
                              Record parameters,
                              Flow flow,
                              Record record)
      throws ServingXmlException {

        ...

        super.writeRecord(context, parameters,
                          flow, newRecord);
    }
    public class HotRecordFilter extends AbstractRecordFilter {

      public void writeRecord(ServiceContext context,
                              Flow flow) {

        Record record = flow.getRecord();
        Record parameters = flow.getParameters();

        ...

        Flow newFlow = flow.replaceRecord(context, record);

        super.writeRecord(context, newFlow);
      }