Restful Gateway Architecture in OSB

In one of my customer’s environment I was reviewing the existing OSB processes. One of their main concerns was Security, so they had implemented security mechanism in all the processes. As you know for SOAP services the OWSM policy feature can be leveraged to embed out of the box security implementations or else for custom implementations use custom OWSM policy and attach it to the process. The environment the customer was working on was 11.1.1.5 where only policies for SOAP services were supported by OWSM. Most of the OSB proxy services were RESTful in nature and all were having the security code scattered all over. That’s when I started looking at reusable and repeating functionalities/steps that need to be built into every service flow. I was able to find lot of repeating steps that had to be done in almost all the flows. As part of refactoring and redesigning I was looking at the Gateway architecture where the request comes in through the Gateway request pipeline and reply through the response pipeline.

As per the new flow we will be clubbing all the common calls into a gateway and internally routing the requests to different projects.

There will be a registry(XML file) which will be having details of all the exposed services in REST. It can be GET  or POST.

The registry will decide on routing the different services based on relative URI

https://<hostname>/GEOGateway/Employee/requestAddress?ID=121

The Gateway will use Employee/requestAddress to figure out which Project and which proxy service to invoke.

The rest of the query string à ID=121 will be passed to the respective proxy services.

Advantages

• Standardization of all the exposed URLs.
•     Request Validation happens in the gateway, only if it is successful it will be routed. A new project developer should only develop the flow, not worry about validation, test it completely then plug it in to gateway.
•   Easy to add/modify new security mechanism.
•   Enrichment/manipulation of request data like populating custom headers which can be useful in tracking the request flow as well as error handling.
•   Response can be converted/transformed from XML to JSON/or any other consumer specific format in response pipeline of Gateway. So tomorrow they want to use a different format we can have it added here instead of doing it in all processes
•  Gateway can also be leveraged to send back properly formatted Error Messages in case of errors suppressing the error stack trace and transforming to consumer understandable error messages.
•  Reduces development time for individual services.
•   Ease of testing

Impacts

• The exposed URLS will change so existing users will be affected.
•   No impact if this is taken care at design time at the beginning of the project.

Downside

If the gateway is down all services will be inaccessible. Central point of failure.

                Not a valid case since

•  the OSB will be always up and running
• Any changes to the Gateway will be tested in lower environments before promoting to PROD.

Implementation

        On the implementation side, all the proxies other than gateway need to be changed to local from http. More on how to handle the new custom headers

References:

http://www.oracle.com/technetwork/articles/soa/architecting-restful-services-083025.html

Response based on Service Types in OSB

Restful services can be implemented using either Proxy type- Any XML Service or Messaging Service types.

·         Messaging Service - to create a service that can receive messages of one data type and respond with messages of a different data type. These exchanges can be either request/response or one-way.

·         Any XML Service - to create an XML service that does not have an explicitly defined, concrete interface.

There is a difference in the way the responses are handled in each case. It’s interesting to see how the response looks like if we add a proxy layer before the existing proxies. I was implementing the Gateway approach where all existing proxies need to be plugged in into the Gateway proxy. In Gateway I am converting the response to either JSON or pass through XML based on the response format requested by the consumer.
Messaging Service response

<soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

<![CDATA[<GetStatusResponseMsg xmlns="http://geo.com/v1.0">

 <GetStatusResponse xmlns="http://geo.com/test/v1.0">

<ResponseStatus xmlns="http://geo.com/status">PENDING</ResponseStatus>

</ GetStatusResponse >

</ GetStatusResponseMsg >]]>

</soapenv:Body>>

In case your proxy is Messaging service based then the response will be encapsulated in the CDATA tag. You will need to use fn-bea:inlinedXML($body) to fetch the XML from the response body.
Any XML Service response

<soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

  <GetStatusResponseMsg xmlns="http://geo.com/v1.0">

 <GetStatusResponse xmlns="http://geo.com/test/v1.0">

<ResponseStatus xmlns="http://geo.com/status">PENDING</ResponseStatus>

</ GetStatusResponse >

</ GetStatusResponseMsg>

</soapenv:Body>>

In case of this approach the response will be in proper XML format without any CDATA encoding.

Service Versioning in OSB

             I was looking at different ways in which we can version the OSB services for my customer. It is a dilemma for all when you get requests for OSB changes due to system standardization like change in the proxy URLS to meet new standards/new functionalities where signatures will change of the exposed services. This issue can arise in any enterprise because when you start of initially without properly architecting the SOA services, not laying down proper standards and guidelines and. Going forward, there will be huge increase in services and how to manage all these services and consumers of these services will be always painstaking. In this article I will put down some points on the service versioning that can be done in OSB.

Backward Compatible changes

The types of changes that are backwards compatible are:

1.       Adding  new XML schema types to WSDL document that doesn’t affect existing schema types

2.       Adding new operations to an existing/new port type in WSDL document. These are basically new functionalities so will not affect existing consumers.

Not Backward Compatible changes

1.       Altering/Removing an operation that is currently consumed.

2.       Modifying the structure/parameters of existing data types

Versioning

All the artifacts need to be versioned at

1.       Namespaces - Unique namespaces to define the new version, usually implemented by mentioning version number.

2.       Folder structure - Separate folders for storing artifacts and folder name mentioning version.

On a high level I am listing the available options

         

Option1 :

If the consumer need not be affected and the new service is backward compatible, then we can deprecate the version1.0 and move on to the new version by changing the proxy service

Option2 :

If the consumer need not be affected and if the new service is not backward compatible, then its always better to maintain different versions of proxy service and have new consumers consume the new Version. When the old consumer is ready to upgrade and move to new version we can deprecate the v1.0 and shutdown the proxy service.

Option3:

This is scenario where there are signature changes and new functionalities required by consumer can be provided only  by modifying the signature then the only option is to work together with consumers and get the new version out which will include changes at consumer as well as provider. Its always better to number the releases so that you can track the service versions and easily identify references to deprecated older versions

I prefer the Option 2 it’s more easy and clean and doesn’t affect existing consumers. Option1 and Option3 are quite similar except that in Option3 the consumer is affected due to the endpoint change and consumer is aware about the version of the Service it’s consuming.

References :-

http://www.infoq.com/articles/contract-versioning-comp2

http://www.ibm.com/developerworks/webservices/library/ws-version/

http://rogervdkimmenade.blogspot.com/2009/09/osb-10gr3-service-versioning.html

OAuth Security implementation in OSB/SOA (REST/SOAP) service

I had recently reviewed / implemented custom made OAuth for OSB. The below article takes you through the journey of becoming secure by using Oauth and the different parameters used and steps to generate valid payload which will pass OAuth security.

Why Oauth ?

Good tutorial @ The OAuth 1.0 Guide

I have tried to map it with the different parameters I have used for creating the signature.

Excerpts from that article

OAuth uses digital signatures instead of sending the full credentials with each request. Digital signatures help the recipient to verify that the content of the request hasn’t changed in transit. To do that, the sender uses a mathematical algorithm to calculate the signature of the request and includes it with the request.

                  In turn, the recipient performs the same workflow to calculate the signature of the request and compares it to the signature value provided. If the two match, the recipient can be confident that the request has not been modified in transit. The confidence level depends on the properties of the signature algorithm used. This mechanism requires both sides to use the same signature algorithm and apply it in the same manner.

         A common way to sign digital content is using a hash algorithm. In general, hashing is the process of taking data (of any size) and condensing it to a much smaller value (digest) in a fully reproducible (one-way) manner. This means that using the same hash algorithm on the same data will always produce the same smaller value. Unlike compression which aims to preserve much of the original uncompressed data, hashing usually does not allow going from the smaller value back to the original.

Shared secret (geo-key-id)

By itself, hashing does not verify the identity of the sender, only data integrity. In order to allow the recipient to verify that the request came from the claimed sender, the hash algorithm is combined with a shared secret. If both sides agree on some shared secret known only to them, they can add it to the content being hashed

Timestamp & nonce (geo-expires-by​ & geo-nonce)

         As long as the shared secrets remains protected, anyone listening in on the network will not be able to forge new requests as that will require using the shared secret. They will however, be able to make the same sign request over and over again. If the intercepted request provides access to sensitive protected data, it can be a significant security risk.

To prevent compromised requests from being used again (replayed), OAuth uses a nonce and timestamp. The term nonce means ‘number used once’ and is a unique and usually random string that is meant to uniquely identify each signed request. Since the nonce value is included in the signature, it cannot be changed by an attacker without knowing the shared secret.

Signature(geo-signature - HMAC-SHAxxx)

OAuth defines multiple signature methods to sign and verify requests: We can use HMAC-SHA 1 ,256 ,512, HMAC signature algorithm combined with the SHA hash method. Since these methods are too complex, so it’s better not to write their own implementations, but instead use trusted open source solutions available for most languages.

UserId /Target sys/Source sys/Module/Service(geo-username​ , geo-target-system-name, geo-source-system-name)

The user Id / Target system/Source system is used to create the signature.

These are not mandatory values , but just added to give more information at the receiver side to do more processing with the payload. This information can be encapsulated under a shared key, but kept it separate so that we need to maintain less number of shared keys.

Based on requirement you can have more service level information to be passed from the Consumer, like different levels of maintaining the shared secret.It can be at a Department/Group level, then modules in each group/dept /then at each service level.

How to get started ?

Pre-Requisites to consume the services validated by Oauth.

1.       geo-key-id​ :- Contact the end system and decide upon on a shared key Id and a hashed shared key value

2.       This hashed shared key value will be part of all the requests and should be confidential between end system and consuming system.

3.       geo-username :-The consuming system should get end system user ids created and need to pass this value along with the request. This is mainly for user token generation at the Atomic service level. The user token will be only be generated in case the user has a valid account /permissions in end system.

4.       geo-target-system-name :- Some more additional information on the type of grouped services like Finance /HRMS/OTM etc application that the requester is trying to access.

5.       geo-source-system-name :- Callers domain name , some more additional information to track the requester.

6.       geo-signature:-The algorithm that needs to be used for hashing and creating signature is HMAC-SHAxxx. There are open source utilities available which can be downloaded to sign the payload using HMAC-SHAxxx algorithm. 

7.       geo-nonce:-Nonce value should be unique and needs to use a unique id generator to populate Nonce value.

8.       geo-expires-by :- The timestamp utility to compute the expiration time of the request.

How to consume the services validated by Oauth.

1.       Identify values for main parameters

·         geo-username

·         geo-target-system-name

·         geo-source-system-name

·         geo-key-id

·         geo-nonce

·         geo-expires-by

·         geo-signature   

geo-key-id

      

The shared key id will be mapped to a shared secret hashed value. Will need to fetch that hashed value and use it for signature creation.

geo-expires-by

                 This will be a timestamp in milli seconds created by using current timestamp in milli seconds

Code snippet ::

  expiresBy =200

  expiresByTime = System.currentTimeMillis() + (expiresBy * 1000);

geo-nonce

   Use  a UID generator so that each nonce value is unique.

geo-signature-algorithm (FYI)

   Use HMAC-SHAxxx​ algorithm utilities to generate the signature encrypted in SHAxxx.

How to create signatureParts

         Signature parts are used to create signature along with the shared Key

1.        For GET - QueryString

  Sample businessParameters  = transactionId=10&sequenceId=123&status=COMPLETED

Steps:                         

a.          Concatenate all the business parameters, geo-username, geo-key-id, geo-source-system-name, geo-target-system-name, geo-expires-by, geo-nonce

Code snippet:

signatureParts =  ?businessParameters+"&")+

                                 "geo-username="+userName+

                                 "&geo-key-id="+sharedKey+

                                 "&geo-source-system-name="+source+

                                 "&geo-target-system-name="+target+

                                 "&geo-expires-by="+expiresByLStr

                                 +"&geo-nonce="+nonce;

b.       Use a HmacSignature Utility to create signature passing in the hashing algorithm HashingAlgorithm.HMACSHA512, shared key and signature parts.

 Code snippet:    

            signature = HmacSignatureUtil.getSignature(new HmacSignatureRequest(HashingAlgorithm.512,

                                                    (String)secretKeys.get(sharedKey),

                                                    signatureParts)

                                                     );

c.        Once you create the signature, always append the geo-signature as the last key /value to the request.

d.       Once the URL is created encode the URL using UTF-8 encoding.

Code snippet      

encodedURL = serverUrl + "?" + URLEncoder.encode(signatureParts + "&geo-signature=" + signature, "UTF-8");

Output:

transactionId%3D10%26sequenceId%3D123%26deliveryStatus%3DCOMPLETED%26geo-username%3Dgeo%26geo-key-id%3DGeoSystem%26geo-source-system-name%3DINT%26geo-target-system-name%3DHRMS%26geo-expires-by%3D1366739410429%26geo-nonce%3Dbb123%26geo-signature%3D8S%2BIbF8DuIChek4ArARBYTIsEY5GIG%2BrbD5CCisw8oY%3D

e.        Final step is to prepend the service URL to the encoded URL

Output:

https://geo.com:9111/GEOBus/v1/update?transactionId%3D10%26sequenceId%123%26status%3DCOMPLETED%26geo-username%3Dgeo%26geo-key-id%3DSystemX%26geo-source-system-name%3DINT%26geo-target-system-name%3DHRMS%26geo-expires-by%3D1366739410429%26geo-nonce%3Dbb123%26geo-signature%3D8S%2BIbF8DuIChek4ArARBYTIsEY5GIG%2BrbD5CCisw8oY%3D

             

2.        POST – HttpHeader

Steps:                         

a.          Concatenate all the business parameters, geo-username, geo-key-id, geo-expires-by, geo-source-system-name, geo-target-system-name, geo-nonce.

         Code snippet:

    httpHeader = "geo-username="+userName+

                            ",geo-key-id="+sharedKey+

        ",geo-source-system-name="+source+

                            ",geo-target-system-name="+target+

                            ",geo-expires-by="+expiresByLStr+

                            ",geo-nonce="+nonce;

signatureParts = httpHeader + businessParameters;

b.       Use a HmacSignature Utility to create signature passing in the hashing algorithm HashingAlgorithm.HMACSHAxxx, shared key and signature parts.

Code snippet:    

            signature = HmacSignatureUtil.getSignature(new HmacSignatureRequest(HashingAlgorithm.HMACSHA512,

                                                    (String)secretKeys.get(sharedKey),

                                                    signatureParts)

                                                     );

c.        The request header should start with “GEO-Auth:“ ,append the signature parts and then  append the geo-signature as the last key /value as the last parameter.

Code snippet:    

                                                                       

Request Header = "GEO-Auth: "+httpHeader+",geo-signature="+signature;

Output:

GEO-Auth: geo-username=geo,geo-key-id=GEOSystem, geo-source-system-name=INT,

geo-target-system-name=HRMS,geo-expires-by=1366736660196,geo-nonce=aa303,geo-signature=nlw2QyHfEmTgti8ZIZRX3rx8utJn3wY2xJsmOiB7Ds8=

That’s it; you can use the same mechanism in consumer system and end system so that both are in sync.

Implementation in OSB

For rest services, you will need to call the service using java callouts in the OSB to recreate the signature and validate it and for SOAP –OSB/SOA you can do it in Custom OWSM authentication using OWSM policy.

You are all set to be secure J .

References:-

http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iii-security-architecture/