Friday, May 6, 2011

SOA-INFRA startup issue

One of the issues which I got involved last week was more related to server administration. The client had a SOA11g environment with Weblogic 10.3.4, SOA 11.1.1.4 & AIA 11.1.1.4. The servers were running fine, but due to some environment issue the soa-infra crashed. As part of the admin process they restarted the SOA managed server. The SOA managed server in Weblogic came up but the soa-infra fails to start with the following error.

Error Snippet @server log

<Apr 21, 2011 12:10:10 AM PDT> <Error> <HTTP> <BEA-101216> <Servlet: "FabricInit" failed to preload on startup in Web application: "/soa-infra".
oracle.fabric.common.FabricException: Error in getting XML input stream: oramds:/deployed-composites/spe/GEOReqABCSImpl_rev1.0/SCA-INF/classes/META-INF/adf-config.xml: oracle.mds.exception.MDSException: MDS-00054: The file to be loaded oramds:/deployed-composites/spe/GEOReqABCSImpl_rev1.0/SCA-INF/classes/META-INF/adf-config.xml does not exist.
                at oracle.fabric.common.metadata.MetadataManagerImpl.getInputStreamFromAbsoluteURL(MetadataManagerImpl.java:280)
                at oracle.integration.platform.common.MDSMetadataManagerImpl.getInputStreamFromAbsoluteURL(MDSMetadataManagerImpl.java:555)

Analysis
 From the log files and stack trace it was pretty evident that one process was holding up the entire soa-infra. The composite process was having a reference to the MDS store and it was missing the adf-config file from the deployed composites folder. The adf-config file usually gets copied to SCA-INF classes folder during the compilation and packaging of the sar (service archive)file.
During deployment, a composite is targeted to a managed server which stores the composite meta-data in the MDS .During startup or on receiving the deployment notification a managed server loads the composite from the MDS.
Below picture gives you the order in which managed server manages soa-infra.
Since the issue is related to the single process, solution was to either provide the adf-config.xml  to the process or un-deploy the process . I am listing the easiest work around that is available so as to resolve the issue.

Workarounds
In 11g there is a deployed-composites.xml in MDS store which has entries related to the composite processes that are deployed to the current soa-infra.
  1. Editing the deployed-composites.xml to remove entries of the composite which is causing the issue.
Since the GEOReqABCSImpl is preventing the soa-infra from starting up, remove the  entries  w.r.t GEOReqABCSImpl process from deployed-composites.xml, After that  you should be able to bring up soa-infra.  For similar issue,have a look at the this blog -- ReynoldsBlog
How to edit the deployed compsites.xml
You can export the whole of MDS from the EM console and then edit the appropriate content and import it back.
From the entry exporting the whole soa-infra MDS partition by using EM->SOA->soa-infra->Administration->MDS Configuration.  Exporting via EM is probably the easiest because it then prepares you to fix the problem later.
2.   2.  Use sca_undeployComposite in WLST to remove the composite which is causing the error.

This WLST command works in offline mode, so no need for soa-infra to be up and running for undeploying the processes, which makes it ideal option for resolving the issue.


Once you apply any of the above listed workaround, you can restart the managed server or bring up the soa-infra .The soa-infra will be up and running. Special thanks to Antony Reynolds who had helped us with this issue.

8 comments:

Raj said...

Hi,

I followed your post on how to edit the deployed-composites.xml file. I needed to remove the composite (that I could not remove via EM) from the repository but when I re-exported the repository to check how it looks like (after the initial export/import), I found that the zipped folder that I uploaded to MDS has been placed under the root MDS folder and its not merged with the existing structure. In other words, it is sitting there on the same folder strucure with a new name. Previously, I saw the following folders:
apps
deployed-composites
soa
Now, I see another folder added:
soa-infra_metadata

Under this folder I find the same folders that are under the root MDS folder i.e.
apps
deployed-composites
soa

These folders represent the changes that I did and uploaded/imported the zip file back and these changes do not reflect in the original MDS folders.

Did you experience this and do you see any issues with the addition of this new folder structure in the MDS?

Please share your thoughts on this.

Thanks,
Rajesh

George Thomas said...

Raj,
How are you doing the export. Are you exporting it to a zip file.

George Thomas said...

Export it as a zip file, make changes using 7-zip, save it and import back.It should work.

Anonymous said...

Hi,
I am getting following error and soa-infra deosnot come up at all.

java.lang.NoClassDefFoundError: com/oracle/bpel/activityguide/instance/model/AGInstance
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getDeclaredMethods(Class.java:1791)
at weblogic.j2ee.dd.xml.BaseJ2eeAnnotationProcessor.getMethods(BaseJ2eeAnnotationProcessor.java:1055)
at weblogic.j2ee.dd.xml.BaseJ2eeAnnotationProcessor.getMethods(BaseJ2eeAnnotationProcessor.java:1043)
at weblogic.ejb.container.dd.xml.EjbAnnotationProcessor.findAnnotatedMethods(EjbAnnotationProcessor.java:1473)
at weblogic.ejb.container.dd.xml.EjbAnnotationProcessor.findAnnotatedTimeoutMethod(EjbAnnotationProcessor.java:1419)
at weblogic.ejb.container.dd.xml.EjbAnnotationProcessor.processSessionAnnotations(EjbAnnotationProcessor.java:627)
at weblogic.ejb.container.dd.xml.EjbAnnotationProcessor.processAnnotations(EjbAnnotationProcessor.java:315)
at weblogic.ejb.container.dd.xml.EjbAnnotationProcessor.processAnnotations(EjbAnnotationProcessor.java:185)

Urgent help needed..

thanks.

Krishna said...

Hi George,
Both the methods mentioned for updating deployed-composites.xml require soa server to be RUNNING.But how to do incase soa server is not coming up due to some composite issue and you want a soa composite removed from deployment so that soa server comes up.

Regards,
Krishna

George Thomas said...

You can try the WLST offline option, where the port will be admin server port..

http://georgie-soablog.blogspot.com/2011/08/executing-wlst-offline-and-online.html

Danilo Schmiedel said...

Hi there,

thanks for your nice post. As written above the commands require a RUNNING soa_server. Under http://inside-bpm-and-soa.blogspot.de/2012/03/undeployment-of-composites-when-soa.html I tried to describe an approach where you can undeploy the composite even when the Server is OFFLINE. See if you find it useful...

Danilo Schmiedel said...

Hi there,

thanks for your nice post. As written above the commands require a RUNNING soa_server. Under http://inside-bpm-and-soa.blogspot.de/2012/03/undeployment-of-composites-when-soa.html I tried to describe an approach where you can undeploy the composite even when the Server is OFFLINE. See if you find it useful...