FTP adapter issues with EOL in BPEL/ESB with different OS's

In most of the environments there will be multiple systems having different operating systems, and in case if we need to transfer the file between these different OS’s then we need to be extra careful before evaluating the solutions . First thing will be to see what is the type of file that we are dealing with .If it is a txt or similar format then what is the type of encoding the files are using. Based on OS the EOL characters will be different .Based on the EOLs the way the input xsd gets created in adapters also needs to be scrutinized. This write up compares between EOL’s used and best practices that needs to be followed while designing/developing BPEL /ESB processes for file transfer.

Different OS’s, use different characters to represent end-of-lines (EOL). When transferring files between these operating systems, the transferring program(GUI tools like Filezilla, WinSCP) will usually make compensation for these differences and convert the EOL characters respectively, in order for them to be displayed correctly on the target operating system. In case of BPEL/ESB it doesn’t automatically take care of converting the EOL’s based on the OS. This will create an issue with the processes where business users are very particular with the formats and content of the file

OS	EOL chars	Representation
UNIX	line feed	\n
Windows	carriage return + line feed	\r\n

When transferring text /ASCII files between these operating systems, the EOL characters need to be converted during the transfer process. This is done typically by specifying text or ascii mode when transferring the files using FTP, as opposed to binary mode. If text files are transferred using binary mode, no EOL character conversion is done and these characters are retained during the transfer process. In this case, the files are often not displayed correctly on the target system. So there will be extra unwanted characters in the file after the transfer.

The FTP adapter provides mechanisms to ensure that correct end-of-line character conversion is done, if required.

1. Select Ascii, instead of binary

2. when defining the message schema in the wizard, do not define the schema as opaque. While using FTP adapter's schema, ensure that the records are terminated using the ${eol} designator. Using ${eol} tells the FTP adpater to use the operating system-specific end-of-file character in this position .

In case you have no control over the number of characters in a line, you will not be able to define an input XSD for files to convert it into NON-Opaque format .So the only option left will be to use opaque schema and use some java embedding code to do the conversion . I will be writing more on this topic in my coming blogs.