SOA-INFRA startup issue

One of the issues which I got involved last week was more related to server administration. The client had a SOA11g environment with Weblogic 10.3.4, SOA 11.1.1.4 & AIA 11.1.1.4. The servers were running fine, but due to some environment issue the soa-infra crashed. As part of the admin process they restarted the SOA managed server. The SOA managed server in Weblogic came up but the soa-infra fails to start with the following error.

Error Snippet @server log

<Apr 21, 2011 12:10:10 AM PDT> <Error> <HTTP> <BEA-101216> <Servlet: "FabricInit" failed to preload on startup in Web application: "/soa-infra".

oracle.fabric.common.FabricException: Error in getting XML input stream: oramds:/deployed-composites/spe/GEOReqABCSImpl_rev1.0/SCA-INF/classes/META-INF/adf-config.xml: oracle.mds.exception.MDSException: MDS-00054: The file to be loaded oramds:/deployed-composites/spe/GEOReqABCSImpl_rev1.0/SCA-INF/classes/META-INF/adf-config.xml does not exist.

                at oracle.fabric.common.metadata.MetadataManagerImpl.getInputStreamFromAbsoluteURL(MetadataManagerImpl.java:280)

                at oracle.integration.platform.common.MDSMetadataManagerImpl.getInputStreamFromAbsoluteURL(MDSMetadataManagerImpl.java:555)

Analysis

 From the log files and stack trace it was pretty evident that one process was holding up the entire soa-infra. The composite process was having a reference to the MDS store and it was missing the adf-config file from the deployed composites folder. The adf-config file usually gets copied to SCA-INF classes folder during the compilation and packaging of the sar (service archive)file.

During deployment, a composite is targeted to a managed server which stores the composite meta-data in the MDS .During startup or on receiving the deployment notification a managed server loads the composite from the MDS.

Below picture gives you the order in which managed server manages soa-infra.

Since the issue is related to the single process, solution was to either provide the adf-config.xml  to the process or un-deploy the process . I am listing the easiest work around that is available so as to resolve the issue.

Workarounds

In 11g there is a deployed-composites.xml in MDS store which has entries related to the composite processes that are deployed to the current soa-infra.

1. Editing the deployed-composites.xml to remove entries of the composite which is causing the issue.

Since the GEOReqABCSImpl is preventing the soa-infra from starting up, remove the  entries  w.r.t GEOReqABCSImpl process from deployed-composites.xml, After that  you should be able to bring up soa-infra.  For similar issue,have a look at the this blog -- ReynoldsBlog

How to edit the deployed compsites.xml

You can export the whole of MDS from the EM console and then edit the appropriate content and import it back.
From the entry exporting the whole soa-infra MDS partition by using EM->SOA->soa-infra->Administration->MDS Configuration.  Exporting via EM is probably the easiest because it then prepares you to fix the problem later.

2.   2.  Use sca_undeployComposite in WLST to remove the composite which is causing the error.

How to use Wlst @ gettingStartedCustomWLSTCommands

This WLST command works in offline mode, so no need for soa-infra to be up and running for undeploying the processes, which makes it ideal option for resolving the issue.

Once you apply any of the above listed workaround, you can restart the managed server or bring up the soa-infra .The soa-infra will be up and running. Special thanks to Antony Reynolds who had helped us with this issue.