SOA Software/Akana API Gateway

There is wide adoption of REStful services across enterprise. Initially we had exposed services on OSB even though the json support was not that great with OSB. There used to be lot of custom java codes that needs to be used to maintain these services. All oracle 11g products were fully focused on the XML , so the json support want great. The overhead of maintaining and converting the payload from xml to json and back was bit taxing on the service performance. Other issue we had was onboarding of App developers who are the real consumers for this API.Even though we had WIKI it had to kept updated based on the enhancements that happen during each sprint. In case the process is missed the Wiki will be outdated. The new App developers had to be helped to troubleshoot to resolve integration issues with services, and this steps are repeated for each App. There was no way they could interact with the developer and then use the knowledge base for the issues already documented. That’s when we started at exploring the option of onboarding a API Gateway which should help us in interaction with our consumers. SOA software or now rebranded AKana was chosen. I have been working extensively on SOA Software /Akana API Gateway for past 8 months. The tool had some initial hiccups to be molded into the client’s environment. But once it became stable, it has been really good. The more familiarity you get on the tool the easier it becomes to debug issues.

The architecture is sort made of 3 components. 

•Community Manager - API Enablement Tool /App Developer Portal

•Policy Manager: Database of run-time policies, access contracts, service definitions and related metadata•

•Network Director: The proxy service that receives virtual service requests, queries Policy Manager for run-time instructions as to how to deal with the requests, then sends the requests to the physical service. The physical service responds, and Network Director applies any policies, and then sends the response.

The CM helps App developers to interact easily with the API developer using a Board and ticketing dash board.

 The policy manager helps in adding all the non-functional requirements.-API Security .traffic monitoring, throttling, QOS Management, caching.

Network director acts as the proxy or the gateway which internally uses the PM .

It really simplifies the virtualizing of service. There is a small process engine to do aggregation of services or orchestrating different services. The out of the box REST support,  json and Oauth 2.0 support helps in  exposing the RESTful  services in quick time and keep up to date the new standards.

The Service Life Cycle

The service life cycle is a very important topic when you start considering the governance of all the services developed across the enterprise and SOA landscape. One of the dilemmas is to define on how long a service should be maintained and given support and when should it should be deprecated and retired.

The services can be grouped as a portfolio of services available in an enterprise. Any service defined or designed in the enterprise will need to be published to the portfolio of services.

•        Before you define a service make sure that the service doesn’t exist.

•        Define stage - To define a service, identify the purpose of the service, appropriate name and design the service  

•        Development stage - Once the service design is approved it moves to the development stage

•        Release stage – Once the development /testing is completed the service is released/published to the consumer

•        Upgrade stage - As the time passes, initial release may not be enough to cater to all the consumer requirements, so will need an upgrade to the service. When the upgrade is published, the older service becomes deprecated. A deprecated service will be in production and available for consumers use for a pre-defined time frame, before they need to upgrade to the new service.  

•        Retirement stage - A retired service is no longer available to consumers and is removed from the enterprise.

Life cycle of the service can be tracked as metadata in a service registry or as part of the header information in the SOAP message. For example service in the released stage should provide the date of its release and deprecated service should record the planned date of its retirement. To maintain the lifecycle we can always leverage capabilities of Enterprise service Repositories and Registries.

Upgrade options

1.       Service URL will have the version number so that it will be clean and easy for deprecating the previous versions and can track the number of cycles of evolution. The consumers will need to change the service url when they have to migrate to new version.

2.       The other mechanism is to add a user header with version mentioned in it, and the Router will need to decode the version and route it to the appropriate service implementation. No change in URL for consuming the new service, but the version number in header only gets modified.

I somehow like the first option because it’s easy for maintaining the services and deprecating them. Anyway to use newer version means the consumer is getting additional functionalities for which anyway they need to make changes. So the URL change can be clubbed in with that, for a consumer it’s not mandatory to upgrade to new service, but the constant change in today’s environment is change ,so you always need to change and move on to new things. Migration can always be planned and worked out as enough time would be given to consumers to upgrade their system to move on to new service.

Service Versioning in OSB

             I was looking at different ways in which we can version the OSB services for my customer. It is a dilemma for all when you get requests for OSB changes due to system standardization like change in the proxy URLS to meet new standards/new functionalities where signatures will change of the exposed services. This issue can arise in any enterprise because when you start of initially without properly architecting the SOA services, not laying down proper standards and guidelines and. Going forward, there will be huge increase in services and how to manage all these services and consumers of these services will be always painstaking. In this article I will put down some points on the service versioning that can be done in OSB.

Backward Compatible changes

The types of changes that are backwards compatible are:

1.       Adding  new XML schema types to WSDL document that doesn’t affect existing schema types

2.       Adding new operations to an existing/new port type in WSDL document. These are basically new functionalities so will not affect existing consumers.

Not Backward Compatible changes

1.       Altering/Removing an operation that is currently consumed.

2.       Modifying the structure/parameters of existing data types

Versioning

All the artifacts need to be versioned at

1.       Namespaces - Unique namespaces to define the new version, usually implemented by mentioning version number.

2.       Folder structure - Separate folders for storing artifacts and folder name mentioning version.

On a high level I am listing the available options

         

Option1 :

If the consumer need not be affected and the new service is backward compatible, then we can deprecate the version1.0 and move on to the new version by changing the proxy service

Option2 :

If the consumer need not be affected and if the new service is not backward compatible, then its always better to maintain different versions of proxy service and have new consumers consume the new Version. When the old consumer is ready to upgrade and move to new version we can deprecate the v1.0 and shutdown the proxy service.

Option3:

This is scenario where there are signature changes and new functionalities required by consumer can be provided only  by modifying the signature then the only option is to work together with consumers and get the new version out which will include changes at consumer as well as provider. Its always better to number the releases so that you can track the service versions and easily identify references to deprecated older versions

I prefer the Option 2 it’s more easy and clean and doesn’t affect existing consumers. Option1 and Option3 are quite similar except that in Option3 the consumer is affected due to the endpoint change and consumer is aware about the version of the Service it’s consuming.

References :-

http://www.infoq.com/articles/contract-versioning-comp2

http://www.ibm.com/developerworks/webservices/library/ws-version/

http://rogervdkimmenade.blogspot.com/2009/09/osb-10gr3-service-versioning.html

SOA based Logging Service - part2

In my last blog I gave a brief overview of different factors that cropped up during the initial thought phase on Logging Service. Since this service was one of the core components in SOA based foundation framework, lot of thought was put in before we finalized on some low-level requirements and features of Logging service. I am listing some core functional requirements that the service should satisfy.

• Service should have a straightforward, lightweight interface

• Service should use a standard, extensible schema to represent the information

• Service should have different levels of severity used to differentiate between logs

• Service should have timestamps associated with each log which will help in preserving the order of logged events.

• Service should represent a set of related logs using a single unique identifier which will be helpful in auditing/tracking

• Service should be able to discard log messages which are below the specified severity

• Service should have multiple destination support (Database, JMS, File, Email)

• Service should have option to set business rules to route logs to different destinations

• Service should be able to modify the data passed in from the client on the fly.

• Service should be able to route to a different destination in case of a failure of the defined destination.(failover)

• Service should be exposed as a web service so that it’s easily accessible to all the heterogeneous systems which form part of the SOA environment.

We did a freeze on requirements so that we could start with the design and development phase and see how things workout. I will write more on this and other components of the SOA foundation framework in my coming blogs.

SOA based Logging Service - part1

Finally its november, considered to be a lucky month for me. I thought I will start this month’s blogging with a non-technical article. I was recently working on designing and developing SOA foundation components which should be generic in nature and can be used for multiple clients. Heavy name rite ?? What was the SOA foundation component supposed to do??? It was supposed to be an Error Handling /Logging /Auditing/Notification Service. I really liked this idea since it involved lot of research and development. Lot of late nights/weekends and breaking of head.

So today I will give a brief on some factors that we considered while doing a requirement analysis of the Logging/Auditing feature of the service.

1. What to log – error data, auditing data, system problems data for troubleshooting, This list keeps growing
2. Log volume - Considering multiple systems in the environment there will be too many log messages.
3. Log diversity - logs generated by all look different .So need to standardize the log format across all platforms and systems.
4. Bad logs – bad logs are also a major challenge. Log messages which do not have enough information. Need to handle them properly.
5. Integrating different logging mechanisms – All applications will be having different ways of error handling and logging. So need to zero-in on the best approach.
6. Making sense of logs - Analyzing the logs based severity, system and other details. Use pre-defined standards.
7. Managing the logs – Using Console/Reporting tool get users to analyze and make best use of the logs.This list will keep growing. But for a generic service these are some factors which I can remember

In my coming blog’s I will write on different options we had in mind and what all features we included in our implementation.

Happy November J

Technologies adopted in Oracle SOA

Oracle SOA Suite is a complete set of service infrastructure components for creating, deploying, and managing SOAs. Oracle SOA Suite enables services to be created, managed, and orchestrated into composite applications and business processes.

Oracle SOA Suite consists of Oracle's most popular, best-of-breed technologies including:

• J2EE:

Oracle JDeveloper 10g: a comprehensive integrated SOA development environment for creating and composing applications that also acts as a unified toolset for all components in the Oracle SOA Suite.

• CONNECTIVITY:

o Adapters

o B2B

o SES(Secure Enterprise Search)

• ROUTING & ORCHESTRATION:

o Oracle Enterprise Service Bus: a standards-based multi-protocol bus to virtualize endpoints as services

o Oracle BPEL Process Manager: the first native business process execution language (BPEL) engine for Web services orchestration enabling you to design, define

and execute business processes.

o Oracle Business Rules Engine: enables agile management of business rules.

• GOVERNANCE:

o Oracle Web Services Manager: a single console to secure and manage your Web services.

o UDDI Registry

• MANAGEMENT & MONITORING:

o Oracle Business Activity Monitoring: delivers real-time insight into business operations.

o Oracle Services Registry: a best of breed UDDI v3 registry.

• END-USER DESIGN TOOLS:

o JDeveloper for Developers and Integrators

o BPA Suite for Business Analysts

Oracle SOA Suite is compatible with other middleware platforms including Oracle Fusion Middleware, IBM WebSphere, BEA WebLogic, and JBoss Application Server.