Implementing Correlation sets - issues and workarounds

Correlation is one interesting area which I always wanted to write on. Recently we got to work extensively on implementing correlations. I will list down some of the issues we had while implementing the correlation for complex workflows.

Why we need correlation?

Interaction patterns can be mainly grouped into Synchronous and Asynchronous. In case of Synchronous the request and reply happens using the same port and the pipe is open till the reply is back. But in case for Asynchronous there are separate ports for both request and reply and because of this the message needs to be tracked so that reply goes to the same request. The correlation mechanism helps to track and route messages to appropriate process instances.

So in case of normal BPEL asynchronous calls BPEL Engine takes care of correlating the message to the instance by assigning a unique correlation id. In case of scenarios where process AàBàCàA or A->B->C->D->A (-> = calls) the correlation has to be taken care by the developer by using correlation sets.

Issues /workarounds while using correlations

Scenario: The example scenario I will be taking will be Agile-SAP PIP which has BPEL1(EBF) -> ESB1(ReqEBS1) -> BPEL2(ProvABCS) -> ESB2(RespEBS1) à BPEL1(EBF)

Here correlation sets are created and assigned while invoking the partner services.

In this flow the out of the box PIP supports only processing of Parts which are of type Item. One of the Clients had a requirement to process Parts which are of type documents. For this we added one more ProvABCS and we had 2 parallel flows which will look like below.

1. -> BPEL1(EBF) -> ESB1(ReqEBS1) -> BPEL2(ProvABCS) -> ESB2(RespEBS1) à BPEL1(EBF)

2. -> BPEL1(EBF) -> ESB1(ReqEBS1) -> BPEL3 (New ProvABCS) -> ESB2(RespEBS1) à BPEL1(EBF)

So for correlating this new flow we created one more correlation set in the same scope and tested the flow.

Issue #1

The response message was not able to properly identify the unique correlation set and was failing.

Staktrace snippet

"ORABPEL-03813 Failed to evaluate correlation query. Failed to evaluate the correlationAlias query "/geo:ResponseMessage/header:Header/header:RequestEBMID" on the element "oracle.xml.parser.v2.XMLElement@1473827" Please check your BPEL/WSDL source to make sure that property alias is defined correctly. at com.collaxa.cube.engine.delivery.CorrelationProperty.setMessagePart(CorrelationProperty.java:85) at com.collaxa.cube.engine.delivery.DeliveryHelper.createCorrelationSet(DeliveryHelper.java:148)

Workaround

1. Create separate scopes for both the flows.
2. Create Correlation sets for each of the flow in their specific scopes.

Test it and it should work properly.

Life is never easy. The new requirement was to call the new Prov ABCS process as many times as there are documents in the payload and at a time there will be multiple items and documents. So while implementing this we ran into our second issue.

Issue#2

While invoking the ProvABCS process in a loop BPEL engine threw the error.

“cannot initiate correlation set more than once”. It meant we cannot initiate the same instance of correlation set again and again.

Stack Trace snippet

Correlation Violation - cannot initiate correlation set more than once

Workaround

The issue is with initiate attribute where we have mentioned it “initiate=yes”. When we look at the BPEL specifications there are 3 values possible for the

Initiate attribute of correlation set. They are

· yes. to initiate the correlation set.

· no. This is the default option. Activity should not initiate the correlation set.

· join. If the correlation set is not yet initiated, initialize it.

Join attribute was the apt choice here, but the attribute join is not supported by Oracle BPEL PM. This is because the implementation of initiate attribute is declared as a Boolean value, it supports only yes/no.

So next option was to create a separate inner scope inside the while loop and declare the correlation specific inside the inner scope. By doing that, our intention is to isolate the initialized value and will get nullified once we come out of the inner scope. So in each loop, a new instance of correlation set will get initialized when flow enters the inner scope and dies when the scope gets over. This will eliminate the issue due to initiation of the same correlation set more than once.

Special thanks to Haresh who worked on trying out different options to make correlations work and helping me this blog.

That’s it on correlation, now back to football J

AIA 11g R1 (over SOASuite11.1.1.3) Installation Issues on Linux

I am putting down some of the issues that we had during the AIA installation on a linux box.

Issue#1 :- The AIA intallation failed at the Pre-Installtion step as it was unable to find the task definition for net/sf/antcontrib/antcontrib.properties or net/sf/antcontrib/antlib.xml.

Error StackTrace

No taskdef found for net/sf/antcontrib/antcontrib.properties

Build failed

In the build files there will be this task definition which looks for this specific task file.

Ant build file- snippet

<taskdef resource="net/sf/antcontrib/antcontrib.properties">

………

…………

</taskdef>

<taskdef resource="net/sf/antcontrib/antlib.xml">

………

…………

</taskdef>

Solution

Copy the ant-contrib.jar from Middleware_HOME/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib to the ANT_HOME/lib folder .Then Retry, it should go through smoothly.It needs to be in the classpath so that these properties or xml files and dependent class files are available to the ant.

Issue#2 :- During the Deployment phase there are chances that initial ear deployment may time out. In case you face such an issue increase the timeout value in the server. Since time-out is coming in the ear deployment increase the time-out value at the weblogic domain level.

Error StackTrace

[wlst] Deploying application from AIA_HOME/Infrastructure/Install/applications/AIAHomeApp.ear to targets soa_server1 (upload=true) ...

[wlst] <Jun 2, 2010 6:00:33 AM IST> <Info> <J2EE Deployment SPI> <BEA-260121> <Initiating deploy operation for application, AIAHomeApp [archive: AIA_HOME/Infrastructure/Install/applications/AIAHomeApp.ear], to soa_server1 .>

[wlst] You have an edit session in progress, hence WLST will not

[wlst] block for your deployment to complete.

[wlst] Started the Deployment of Application. Please refer to the returned WLSTProgress object or variable LAST to track the status.

[wlst] Activating all your changes, this may take a while ...

[wlst] The edit lock associated with this edit session is released

[wlst] once the activation is completed.

[wlst] Traceback (innermost last):

[wlst] Exception in thread "main" java.lang.IllegalStateException: Traceback (innermost last):

[wlst] File " AIA_HOME/Infrastructure/Install/AID/lib/py/deployApplication.py", line 39, in ?

[wlst] File "<iostream>", line 376, in activate

[wlst] File "<iostream>", line 1848, in raiseWLSTException

[wlst] WLSTException: Error occured while performing activate : Error while Activating changes. : Timed out waiting for completion

[wlst] Use dumpStack() to view the full stacktrace

Solution

Timeout can be configure in 2 ways on Weblogic domain level .Either it can be done from the console or else from backend by editing the configuration file.

a) Edit the config.xml @ Middleware_home\user_projects\domains\SOADomain\config folder

XML Snippet –config.xml

<?xml version='1.0' encoding='UTF-8'?>

<domain xmlns="http://xmlns.oracle.com/weblogic/domain"

……………..>

<name>soa_domain</name>

<domain-version>10.3.3.0</domain-version>

<security-configuration>

…………..

</security-configuration>

<jta>
<timeout-seconds>600</timeout-seconds>
</jta>

<server>

…

</server>

</domain>

b) Configure the value from weblogic console

Select Services > JTA

Select JTA tab

Change the parameter: Timeout Seconds

That’s it for the day. Have fun. :)

Sensors missing from JDeveloper 11g PS2

From SOA Suite 11g PS2, the Jdeveloper (11gPS2) will have 3 views for a BPEL process.

The 3 views are

1. BPEL normal view
2. Monitor view
3. BPA

There will be an option on the top right corner of the BPEL process view to select the different views. The sensors are not visible anymore as part of the tabs in any of the activities or structure view. This has been moved and consolidated into a view called Monitor which will help you in creating Sensor and sensor events and publish it to different destinations. There are some new features like Business indicators, counters and intervals introduced for process and business monitoring

Refer the link for more on new features on BAM

SOASuite 11g- Implement User-Defined XPath Extension Functions

I had previously written a blog on creating XPATH function in 10g. This write up is on how to achieve the same in 11g.

In 11g the major difference is that XPATH functions has been extended to some more components.

· BPEL

· Mediator

· Human workflow

· XSLT Mapper

The implementation is different only for XSLT mapping. For all other components implementation is same. Functions can be shared across all components and is defined by the xpath-functions-config xml file.

For each component we will define functions in a component specific config file

BPEL - ext-bpel-xpath-functions-config.xml

Mediator - ext-mediator-xpath-functions-config.xml

XSLT Mapper - ext-mapper-xpath-functions-config.xml

Common to all - ext-soa-xpath-functions-config.xml

I will explain the steps involved to create user-defined Xpath function for both BPEL and XSL mapper.

BPEL

For creating user-defined XPATH function, we need to define a java component and a xml component .The Xml component will take care of the configuration where as java component will be the implementation of the function.

We will start with xml component. We will define the function interface in a xml format.

1. Create an xml file - ext-bpel-xpath-functions-config.xml

Add this xml snippet

<soa-xpath-functions version="11.1.1"

xmlns="http://xmlns.oracle.com/soa/config/xpath"

xmlns:geo="http://www.geo-functions.com/HelloWorld">

<function name="geo:helloWorld">

<className>com.geo.xpath.HelloWorldTest</className>

<return type="string"/>

<params>

<param name="value" type="string"/>

</params>

<desc/>

<detail>

<![CDATA[This function returns a HelloWorld message.]]>

</detail>

</function>

</soa-xpath-functions>

here function signature is helloWorld() ,The namespace of the function is xmlns:geo=http://www.geo-functions.com/HelloWorld

2. Create a simple java class to implement the XPATH BPEL function.

The java class should implement oracle.fabric.common.xml.xpath.IXPathFunction interface. You can find this class and related xpath functions in fabric-runtime.jar

Code Snippet HelloWorldTest.java

package com.geo.xpath;

import oracle.fabric.common.xml.xpath.IXPathFunction;

import oracle.fabric.common.xml.xpath.IXPathContext;

import oracle.fabric.common.xml.xpath.XPathFunctionException;

import java.util.List;

public class HelloWorldTest implements IXPathFunction {

public Object call(IXPathContext context, List args) throws XPathFunctionException {

return "Hello world :: "+args.get(0);

}

}

You need to have fabric-runtime.jar in your classpath for compiling this java class.

>javac -classpath lib\ fabric-runtime.jar -d classes com\geo\xpath\HelloWorldTest.java

Here lib points to the location of these jars. –d classes points to the output folder where I will get my class file.

3. Create a jar with the generated class file.

>jar -cvf helloWorld.jar com/geo/xpath/HelloWorldTest.class

The jar will not be complete without the configuration file. Now add the ext-bpel-xpath-functions-config.xml file to the META-INF folder

Development of custom XPATH function is done. Now we will use it.

Usage

In the BPEL process use an assign operation to invoke the function

Code Snippet of .bpel file

<assign name="Assign_1">

<copy>

<from expression="geo:helloWorld(string(bpws:getVariableData('inputVariable','payload','/client:process/client:input')))"/>

<to variable="Invoke_1_execute_InputVariable" part="request"

query="/ns3:singleString/ns3:input"/>

</copy>

</assign>

When you compile the bpel process you will get a compilation error because the namespace-prefix and namespace-uri are not declared in the bpel process. Add the namespace- prefix and namespace-uri from your new function defined in ext-bpel-xpath-functions-config.xml to the process tag of .bpel file

Code Snippet of .bpel file

<process name="XpathBpel" targetNamespace="http://xmlns.oracle.com/XpathBpel"

……….

……. xmlns:geo="http://www.geo-functions.com/HelloWorld"

>

Compile and deploy your bpel process. The output will be the return string of the call method defined in the function.

Runtime usage:

1. Copy the HelloWorld function jar to MIDDLEWARE_Home/user_projects/domains/soa_domain/lib or a subdirectory of lib.

2. Restart the Oracle WebLogic Server.

XSLT Mapper

As we did for BPEL, we need to define a java component and a xml component .The Xml component will take care of the configuration where as java component will be the implementation of the function.

We will start with xml component. We will define the function interface in an xml format.

1. Create an xml file - ext-mapper-xpath-functions-config.xml

Add this xml snippet

<soa-xpath-functions version="11.1.1"

xmlns="http://xmlns.oracle.com/soa/config/xpath"

xmlns:geo="http://www.oracle.com/XSL/Transform/java/com.geo.xpath.HelloWorldXSLTest">

<function name="geo:helloWorldXSL">

<className>com.geo.xpath.HelloWorldXSLTest</className>

<return type="string"/>

<params>

<param name="value" type="string"/>

</params>

<desc/>

<detail>

<![CDATA[This function returns a HelloWorld message.]]>

</detail>

</function>

</soa-xpath-functions>

Here function signature is helloWorldXSL() ,The namespace of the function is xmlns:geo= http://www.oracle.com/XSL/Transform/java/com.geo.xpath.HelloWorldXSLTest". The difference with BPEL XPATH function is the namespace. Here the namespace should be in a specific format. http://www.oracle.com/XSL/Transform/java/ + package name +Class name

2. Create a simple java class to implement the XPATH BPEL function. A point to note is that for each function you need to write a corresponding public static method. The function name and method name must match.

Code Snippet HelloWorldXSLTest.java

package com.geo.xpath;

public class HelloWorldXSLTest {

public static String helloWorldXSL(String input){

return "Hello worldXSL :: "+input;

}

}

Compile this java class.

>javac -d classes com\geo\xpath\HelloWorldXSLTest.java

3. Create a jar with the generated class file.

>jar -cvf helloWorldXSL.jar com/geo/xpath/HelloWorldXSLTest.class

The jar will not be complete without the configuration file. Now add the ext-mapper-xpath-functions-config.xml file to the META-INF folder

Development of custom XSL Mapper XPATH function is done. Now we will use it.

Usage

JDeveloper will support user defined XSL functions .To use it from the wizard

· In Oracle JDeveloper, go to Tools > Preferences > SOA.

· Click the Add button and select your JAR file.

· Restart Oracle JDeveloper for the changes to take effect.

The function will be listed in the user-defined functions under Component Palette. You can directly drag and drop into your XSL.

Code Snippet of .xsl file

<xsl:stylesheet version="1.0"

xmlns:geo="http://www.oracle.com/XSL/Transform/java/com.geo.xpath.HelloWorldXSLTest"

…………

…………..

>

<xsl:template match="/">

<client:processResponse>

<client:result>

<xsl:value-of select="geo:helloWorldXSL(/client:process/client:input)"/>

</client:result>

</client:processResponse>

</xsl:template>

</xsl:stylesheet>

Compile and deploy your bpel/mediator process having your XSL mapping file. The output will be the return string of the method defined in the function.

Runtime usage:

1. Copy the HelloWorldXSL function jar to MIDDLEWARE_Home/user_projects/domains/soa_domain/lib or a subdirectory of lib.

2. Restart the Oracle WebLogic Server.

That’s it J

Configuration and preference properties in BPEL and Adapters @ Composite

Configuration and preference properties are always used extensively in most of the projects. So a developer needs to be aware about the changes that have happened from 10g to 11g.

BPEL

In 10g the property values which had to be updated at runtime from BPEL Console were defined as preference property. Process level configuration settings were defined using the configuration properties in bpel.xml.

Coming to 11g, to define configuration properties and values for a BPEL process, add the properties under the service component section of the composite.xml file.

Code Snippet composite.xml

 <component name="BPELPropertiesTest">

    <implementation.bpel src=” BPELPropertiesTest.bpel"/>

        <property name="bpel.config.completionPersistPolicy">true</property>

  </component>

</component>

How to use preference properties in BPEL is well explained in the Eric Elzinga blog

Define properties in composite.xml under the component tag.

Code Snippet of composite.xml

<component name="PreferencesTest">

<implementation.bpel src="PreferencesTest.bpel"/>

<property name="bpel.preference.id">4402</property>

<property name="bpel.preference.name">Geo Tho</property>

</component>

Access these preferences in Assign activity using the ora:getPreference() XPath function

Code Snippet of .bpel

<assign name="AssignManagerDetails">

……………

<copy>

<from expression="ora:getPreference('id') "/>

<to variable="Manager" part="payload"

query="/client:processResponse/client:id"/>

</copy>

……

</assign>

The difference here is that instead of going to BPEL Console in 10g you will need to explore the System MBean browser to update these preference properties.

You can right click on soa_infra >Administration > System Mean Browser

Navigate to Application Defined MBeans > oracle.soa.config > Server : soa_server1 > SCAComposite > ProjectName > SCAComposite.SCAComponent > BPELProcessName.

Click Properties. It will list you all the properties that you can update.

Adapters

In case of Adapters the properties will be present in both Composite.xml and Adapter JCA file.

In case of File/FTP adapter, most of the properties are present in the jca file that gets generated as part of the File adapter. In case you select the logical directory, then value of the directory should be entered in composite.xml.

Code Snippet of composite.xml

<reference name="FileWrite" ui:wsdlLocation="FileWrite.wsdl">

<interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/file/GenericApp/PropertiesTest/FileWrite#wsdl.interface(Write_ptt)"/>

<binding.jca config="FileWrite_file.jca"/>

<property name="folder" type="xs:string" many="false" override="may">C:\temp</property>

</reference>

Code Snippet of File Adapter jca file

<adapter-config name="FileWrite" adapter="File Adapter" wsdlLocation="FileWrite.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

<connection-factory location="eis/FileAdapter" adapterRef=""/>

<endpoint-interaction portType="Write_ptt" operation="Write">

<interaction-spec className="oracle.tip.adapter.file.outbound.FileInteractionSpec">

<property name="LogicalDirectory" value="folder"/>

<property name="FileNamingConvention" value="bpel_%yyMMddHHmmss%.txt"/>

<property name="Append" value="false"/>

<property name="NumberMessages" value="1"/>

</interaction-spec>

</endpoint-interaction>

</adapter-config>

All these end point properties can be updated at runtime from the Enterprose manager. Navigate to

Soa-infra > Composite process > Service/References Properties > AdapterName

In case of DB adapter, most of the operation related properties are present in the jca file that gets generated as part of the DB adapter. The connection related properties like retry count, retry interval are specified in composite.xml.

Code Snippet of composite.xml

<reference name="DBWrite" ui:wsdlLocation="DBWrite.wsdl">

<interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/db/GenericApp/PropertiesTest/DBWrite#wsdl.interface(DBWrite_ptt)"/>

<binding.jca config="DBWrite_db.jca"/>

<property name="jca.retry.count" type="xs:int" many="false" override="may">4</property>

<property name="jca.retry.interval" type="xs:int" many="false"

override="may">1</property>

<property name="jca.retry.backoff" type="xs:int" many="false"

override="may">2</property>

<property name="jca.retry.maxInterval" type="xs:string" many="false"

override="may">120</property>

</reference>

Code Snippet of DB Adapter jca file

<adapter-config name="DBWrite" adapter="Database Adapter" wsdlLocation="DBWrite.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

<connection-factory location="eis/DB/p2icoeSOADEMO" UIConnectionName="p2icoeSOADEMO" adapterRef=""/>

<endpoint-interaction portType="DBWrite_ptt" operation="insert">

<interaction-spec className="oracle.tip.adapter.db.DBWriteInteractionSpec">

<property name="DescriptorName" value="DBWrite.Employees"/>

<property name="DmlType" value="insert"/>

<property name="MappingsMetaDataURL" value="DBWrite-or-mappings.xml"/>

<property name="DetectOmissions" value="true"/>

<property name="GetActiveUnitOfWork" value="false"/>

</interaction-spec>

</endpoint-interaction>

</adapter-config>

All these end point properties can be updated at runtime from the Enterprise manager. Navigate to

Soa-infra > Composite process > Service/References Properties > AdapterName