Login | Register
My pages Projects Community openCollabNet

Frequently Asked Questions

DataObjects
Hospitals
JDK1.1.x
Log4j
MS SQLServer
Oracle
Sybase
MySQL
Windows NT Services
FTP
XML
...
Sundries

 
 

DataObjects

What are DataObjects?

"DataObjects" is the name for the DOXML message payload used by openadaptor

The key points to note about DOXML are:

  • it is not pure XML
  • it has nothing to with objects (!)
  • all openadaptor components downstream of the sources expect to process arrays of dataobjects - this is how the plug ‘n’ play paradigm is supported.

For further information on DataObjects see: http://www.openadaptor.org/docs/do_tutorial.htm

Why does openadaptor use DateHolder and DateTimeHolder instead of the java Date object?

We experienced lots of problems when loading date time strings into a java Date object, it always seems to check the timezone of the box and convert the value that it is parsing.

Why does my Source fail to process some incoming DataObjectXML, I get "Bad value for date attribute"?

If you are subscribing to information and you receive this error:

[03/08/11 13:56:06.193] FATAL: Error on sourcePoll for [C1],Unexpected exception
constructing DataObjects from XML, Line 21: Bad value for Date attribute Att6: '2004-04-01 GMT'

The problem is that "Date" is not supported as a data type in DataObjects v1. However, it is supported in v2, so the solution is to switch the publishing application to use DataObjects v2.

Set the following property on the sink component: MessageWriter.MessageFormat = V2

Are there any tools available for viewing DataObject Arrays in a window?

The InspectorSink in the org.openadaptor.adaptor.display package is a crude component that will display the payload details of each DataObject that is passed to it. You will need to set the following Controller property ExitWhenSourceAreDry = false if you want to use this (otherwise the adaptor will exit as soon as the Source terminates).

Why does XMLStringWriter output each DataObject in a message as a separate XML document? I want to output all the DataObject in a single message array as one document!

Prior to openadaptor version 1.3.0, the XMLStringWriter output each DataObject in a message as a separate XML document.

In v1.3.0 there is a new Property of the XMLStringWriter, called WriteBatchAsRecord, which can be set to 'true' or 'false'. If this is set to 'true', XMLStringWriter outputs all the DataObjects in a single message array as one document. Note that you must also tell the Source's Reader to set its batch size to zero if you want all the records in your file to be read in as one message, and thus output in one document. If you do not set the batch size, it defaults to one, meaning that each line of the file is sent down the openadaptor pipeline as an individual message, and consequently each record would still appear as a single XML document.

So in summary:

- Use the latest version of openadaptor

- In your source, set BatchSize = 0

- In your sink, set WriteBatchAsRecord = true

 

Sybase

I am using SybaseSource and I get a "failure to connect to server" error.?

The Sybase Jdbc drivers (jConnect) rely on some stored procedures in the sybsystemprocs database. Normally these are installed by default. Please ask your DBA to load these. They can be found in any jConnect distribution? Version 5.2 is popular. Try Sybase

I am using a SybaseSource and it's failing with:

FATAL: Error retrieving results from database: Stored procedure 'sp_myproc' may be run only in unchained transaction mode. The 'SET CHAINED OFF' command will cause the current session to use unchained transaction mode.

Unchained is the default mode for stored procs, and your adaptor is running in chained mode ( not the default ). This is due to historic confusion amongst the database components about CHAINED mode. These will be resolved shortly, but in the meantime you can get round this by changing the mode of the sp thus...

1> sp_procxmode sp_myproc, "anymode"

And things will just work.

 

MySQL

Is there support for the BCG message structure?

No. Until MySQL supports stored procedures, the BCG message structure and methods can not be easily implemented. Therefore the only way to get data from or insert data into a MySQL database is to use the SQLSource/SQLSink components. See the cookbook examples for details.

What version does openadaptor work with?

Tested using the version 3.0.9 of the MySQL JDBC driver and versions 4.0.16 and 3.23.58 of the MySQL database. For more details, see:
http://www.mysql.com/documentation/connector-j/index.html
http://www.mysql.com/documentation/index.html

I get "SQL syntax near 'CHAINED OFF'"

The openadaptor JDBC components make use of the JdbcConnectionParams class to configure the JDBC connection. One of the properties it uses is UsingChainedTransactions. This works in "reverse" fashion as if it is not set then a "SET CHAINED OFF" command is issued. MySQL does not support this command so make sure that you avoid it by using the following line in your properties file:

A.C2.UsingChainedTransactions = true

Note that the "A.C2" prefix needs to be modified to suit your adaptor.

I get an access denied error when I run the cookbook examples

In order to run the Cookbook examples you must get the database administrater to create an account for oa_user by running the following command:

GRANT USAGE ON [db].* TO oa_user@[machinename] IDENTIFIED BY '[password]';

where [db] is replaced by the name of the database created when installing the InstallDB.sql and TestInstall.sql scripts, [machinename] is the name of the database server and [password] is the users password. This command will create an account using the supplied details. A unique identifier is composed of two parts - the username and the machinename. For example, tim@host1 and tim@host2 are different.

What is the format of the JDBC connection string?

The JDBC URL format for MySQL Connector/J is as follows, with items in square brackets ([, ]) being optional:

jdbc:mysql://[host][,failoverhost...][:port]/[database][?propertyName1]
[=propertyValue1][&propertyName2][=propertyValue2]...

If the hostname is not specified, it defaults to '127.0.0.1'. If the port is not specified, it defaults to '3306', the default port number for MySQL servers. The following list defines the most common properties:

user The user to connect as
password The password to use when connecting
autoReconnect Should the driver attempt to re-connect if the connection dies? (true/false) defaults to false
maxReconnects If autoReconnect is enabled, how many times should the driver attempt to reconnect? defaults to 3
initialTimeout If autoReconnect is enabled, the initial time to wait between re-connect attempts (in seconds) defaults to 2
maxRows The maximum number of rows to return (0 means return all rows) defaults to 0

 

Oracle

Why does my Oracle JdbcSink keep failing with the errors: Error Code 6550 and/or PLS-00201: identifier M0500001 must be declared?

In Oracle the user is allowed to create user-defined Procedures and Functions. The Oracle JdbcSink expects to call an Oracle user-defined function rather than an Oracle user-defined procedure. These error messages indicate that the user-defined function can not be found, so make sure that the user-defined function that you are calling is an Oracle Function and not an Oracle Procedure.

 

MS SQL Server

Are there components for Microsoft SQL Server?

Yes. In the mssql subdirectory of the jdbc package

I'm using MS SqlServer and my adaptor fails with the message "'CHAINED' is not a recognized option"

Um yes. We've got the property the wrong way round! When using MS Server you need to set the UsingChainedTransactions property to true.

This will be rectified. Although probably not soon enough for some of our users.

There is another strand to this however - if you are trying to take advantage of the polling facility given by the SybaseSource class then it doesn't matter what your properties file is set to as "SET CHAINED OFF" is hardcoded. So it won't work at all. Use the more recent MSSqlSource class.

Strings larger than 255 characters are truncated when using the Opta driver.

You are using the SQL Server 6.5 compatible mode. The SQL Server 6.5 supports only strings with a size up to 255 characters. You are using this mode because JDBC subprotocol is set to inetdae or inetdea6.

You need to modify the connection URL to use the JDBC subprotocol inetdae7 or inetdae7a. For older versions you will have to set the JDBC URL option sql7=true.

samples:
jdbc:inetdae7:localhost
jdbc:inetdae:localhost?sql7=true

 

Hospitals

Can I view the contents of a Message Hospital?

openadaptor offers reference implementations of the Message Hospital using both Sybase and MS Sql Server. Erroneous messages are stored in the database and there is a command line tool to look/edit/re-submit the message. Details are available in the openadaptor Java Programmer's Guide.

Breaking news.....the command line tool has been broken since the dawn of openadaptor time due to a package name error! This has been logged as an issue and will be fixed relatively soon

Is it possible for a HospitalPipe to link to another pipe instead of a sink?

No, it's not possible at the moment. Also, in order to extend the HospitalPipe to enable this, the core framework needs to be enhanced to allow better pipeline branch decisions. This will be done in a future release.

 

JDK1.1.x

How do I run my adaptor under 1.1.x JVM?

NB: Since openadaptor version 1.6.0 support for JDK 1.1.x has been removed

 

Log4j

My log4j.properties settings seem to be ignored

By default old style openadaptor logging properties will be used in preference to any log4j.properties or log4j.xml files specified. This is true even if there are no openadaptor logging properties defined in the adaptor property file. In this case the logging will be set up with openadaptor default logging settings and any log4j configuration ignored.

If you wish to use normal log4j configuration then you must explicitly specify this. See http://openadaptor.openadaptor.org/pg/adaptor_logging_levels.htm#ConfiguringWithLog4j for information.

If you choose to configure log4j directly then all the normal log4j configuration rules apply. See http://jakarta.apache.org/log4j for much more information on log4j.

How do I create log files for specific components?

Check out http://jakarta.apache.org/log4 for details about log4j.

Basically, log4j allows you to define multiple appenders for a logger. Each appender can output log messages to various destinations (console, files, GUI components, remote socket servers, JMS, NT Event Loggers, remote UNIX Syslog daemons, ...). You can define an appender for a specific package and all logging information for all classes in that package will be handled by this appender.

The following excerpt shown in both property file format and xml illustrates defining an appender for the package org.openadaptor.adaptor.xml and logging FATAL events only to a file.

This is how it would appear in log4j.properties.

#
# define seperate appender for any class in the org.openadaptor.adaptor.xml package
#
log4j.logger.org.openadaptor.adaptor.xml = FATAL, FatalLog

#
# log FATAL messages to a file
#
log4j.appender.FatalLog = org.apache.log4j.FileAppender
log4j.appender.FatalLog.File = fatal.log
log4j.appender.FatalLog.Append = true
log4j.appender.FatalLog.Threshold = FATAL
log4j.appender.FatalLog.MaxBackupIndex = 5
log4j.appender.FatalLog.MaxFileSize = 10MB
log4j.appender.FatalLog.layout = org.apache.log4j.PatternLayout
log4j.appender.FatalLog.layout.ConversionPattern = %d %-5p %c{2} (%x) - %m%n

The following xml code shows how to define the same behaviour using an xml file (log4j.xml).

<!-- Define the logger for package org.openadaptor.adaptor.xml -->
<!-- and attach the appender to the logger. -->
<logger name="org.openadaptor.adaptor.xml">
    <level value="fatal"/>
    <appender-ref ref="FatalLog"/>
</logger>

<!-- Define the appender. -->
<appender class="org.apache.log4j.FileAppender" name=""FatalLog">
    <param name="File" value="fatal.log"/>
    <param name="Append" value="true"/>
    <param name="Threshold" value="FATAL"/>
    <param name="MaxBackupIndex" value="5"/>
    <param name="MaxFileSize" value="10MB"/>
    <layout class="org.apache.log4j.PatternLayout">
    <param name="ConversionPattern" value="%d %-5p %c{2} (%x) - %m%n"/>
    </layout>
</appender>

 

Windows NT Services

How do I run my adaptor as a Windows NT service?

There is a natty little java package called jsrvany that "uses the java event model to implement the win32 Service Control interface in java. This allows any java application to be run as a service on Windows NT 4 and for the application to respond to all the events triggered by the win32 Service Control Panel - start, stop, pause, continue, terminate and interrogate."

Check out http://jsrvany.sourceforge.net/ for more details.

NB: jsrvany is very sensitive to the classpath and you must make sure that it does not contain any relative paths.

I get a ClassNotFoundException when using jsrvany?

When launching applications using jsrvany the context ClassLoader for the main thread isn't set leading to ClassNotFoundException's. To fix this use the following:

ClassLoader cl = ClassLoader.getSystemClassLoader();
currentThread.setContextClassLoader(cl);

 

FTP Components

Cannot open SunFTP stream

By default the FTP components make use of the org.openadaptor.adaptor.ftp.SunFTP wrapper classes which use of the standard sun.net.ftp.FtpClient and sun.net.TelnetInputStream classes to perform the actual file transfer. depending on the JDK version you are using, you need to make sure that these classes are in your classpath.:

  • JDK 1.4 - there is no problem and the ftp clasess are automatically included
  • JDK 1.3 - you need to ensure that that rt.jar is in the classpath
  • JDK 1.2 - not tested but the classes are in rt.jar so it might work

I only get the first file when using wikldcards in the SourceFile property

The Sun Java libraries are subject to change between JDK releases. A case in point is - if you specify a wildcard in the filename. Under JDK 1.4 the source will retrieve each file that matches the file pattern on subsequent polls pretty much as you would expect. However, if you use JDK 1.3 then it will only retrieve the first file that matches and then empty files for the rest. Upshot, only use JDK 1.3 if you are getting single files!

java.security.NoSuchAlgorithmException: DH KeyPairGenerator not available

When using the SFTP components, you must have Sun's JCE extensions installed in order for the cryptix classes to work. Even then there have been some problems. It's best to use JDK 1.4 if you can.

If you have installed JCE1.2.2 as an extension to your Java 1.3 distribution, but still cannot connect then you have not yet correctly registered the "SunJCE" provider, as described in the jce1.2.2 documentation. Check out http://java.sun.com/products/jce.

 

XML Documents

How to parse simple documents

Given the following very simple XML document with a single level of nesting:

<?xml version="1.0" encoding="iso-8859-1" ?>
<team>
      <developer>
            <name>Russ</name>
            <skills>Java</skills>
      </developer>
      <developer>
            <name>Tim</name>
            <skills>.Net</skills>
      </developer>
      <developer>
            <name>Steve</name>
            <skills>SQL</skills>
      </developer>
</team>

you can very quickly retrieve an array of DataObjects corresponding to the "developer" tags using the XMLStreamReader. Just add the following to the Java properties file:

A.C1.ClassName = org.openadaptor.adaptor.standard.FileSource
A.C1.DOStringReader = org.openadaptor.dostrings.XMLStreamReader
A.C1.InputFileName = some.xml
A.C1.DiscardRootObjectTag = true

 

Sundries

Does openadaptor support Unicode?

Yes.

Er No, we're working on it...

Is there a particular code style I should use when I contribute?

Yes please, see here for details.

Can I use multiple lines in my properties file?

Yes. Java Properties makes provision for this. Simply add a single backslash to the end of the line and the properties will concatinate it with the next line. The most obvious place to use this is when defining large SQL statements. For example:

A.C2.SQLStatement1 = Declare @trader varchar(255) \
Set @trader = '%Trader%' \
If @trader = '' Set @trader=Null \
Insert Into russ (flag,data) values (0, @trader)