How to use OSB Reporting Service with BAM -Part1

            The Reporting Service will be a generic service which will listen to Reporting queue of OSB, log or record transaction or exception data, or to raise a notification or alert to an end user or technical support group. It will persist report event data to a database or other similar sink so that it will be available for reporting and auditing purposes.

OSB Requesters will access the Reporting service via a non-blocking, asynchronous call, passing a simple XML log event object containing various service identification and log description data. On the service provider side, log events will be processed sequentially.

 An event can be defined as an action capable of causing more than one action to fire – so a serious exception may result in the data being persisted to database and an email notification being sent to the appropriate support team. This can be designed and developed by leveraging the functionalities of Oracle BAM tool.

Requirements ::   Reporting / Analytics

• Ability to report service consumption per service per application
• Calls per sec/min/hour/day
•  Error Counts
•   Performance Metrics
• Performance monitoring on service level.

  Approach

                        OSB defines an extensible Reporting Framework that permits adding custom providers. This OSB/Oracle BAM integration approach consists of developing a custom reporting provider that enables report information to be sent out to Oracle BAM. This is accomplished by configuring the reporting action inside an OSB service flow, and Oracle BAM picking up the message using Enterprise Message Sources.

Oracle BAM defines a basic component called a data object. A data object represents a data entity whose information is to be stored in Oracle BAM. Each new message to be integrated into Oracle BAM must be validated against an underlying XML schema specific for the selected data object

Data Model

      Define a Reporting data structure. This will hold all the data that is defined to hold information that can help in monitoring as well as reporting.

Sample data structure

Element	Description
PROJECT_NAME	Project Name
MODULE_NAME	Module name
SERVICE_NAME	getEmployee
SERVICE_TYPE	SOAP or REST
HTTP_METHOD	GET or POST
USER_NAME	User related name
USER_ID	User related id
START_TIME	Start time of the request
SYSTEM_ID	External system Id
CORRELATION_ID	Uniquely identify the record ,so that status updates can be related to the same record
MSG_ID	Uniquely identify the record
REPOSITORY_NAME
QUERYSTRING	For rest calls to store query string
PAYLOAD	XML payloads
STATUS	Success /Error
PRIMARY_ID	Any Ids that we want to track like EmployeeID
TOTAL_TIME_SEC	Time difference between Start /End time
END_TIME	End time of the request
PROXY_ID
ERROR_TEXT	Error Messages
PROCESS_FLOW_STAGE	Request ,Response, service stage-Transcoding
TECH_COMPONENT	OSB, BPEL, Java

Sample XSD

<?xml version="1.0" encoding="windows-1252" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"

            xmlns:report="http://geo.com/report/header"

            targetNamespace="http://geo.com/report/header"

            elementFormDefault="qualified">

  <xsd:element name="REPORT_DATA">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="PROJECT_NAME" type="xsd:string"/>

        <xsd:element name="MODULE_NAME" type="xsd:string"/>

        <xsd:element name="SERVICE_NAME" type="xsd:string"/>

        <xsd:element name="SERVICE_TYPE" type="xsd:string"/>

        <xsd:element name="HTTP_METHOD" type="xsd:string"/>

        <xsd:element name="USER_NAME" type="xsd:string"/>

        <xsd:element name="USER_ID" type="xsd:string"/>

        <xsd:element name="START_TIME" type="xsd:string"/>

        <xsd:element name="SYSTEM_ID" type="xsd:string"/>

        <xsd:element name="CORRELATION_ID" type="xsd:string"/>

        <xsd:element name="MSG_ID" type="xsd:string"/>

        <xsd:element name="REPOSITORY_NAME" type="xsd:string"/>

        <xsd:element name="QUERYSTRING" type="xsd:string"/>

        <xsd:element name="PAYLOAD" type="xsd:string"/>

        <xsd:element name="STATUS" type="xsd:string"/>

        <xsd:element name="QUERY_KEY" type="xsd:string"/>

        <xsd:element name="PRIMARY_ID" type="xsd:string"/>

        <xsd:element name="TOTAL_TIME_SEC" type="xsd:string"/>

        <xsd:element name="END_TIME" type="xsd:string"/>

        <xsd:element name="PROXY_ID" type="xsd:string"/>

        <xsd:element name="ERROR_TEXT" type="xsd:string"/>

       <xsd:element name="PROCESS_FLOW_STAGE" type="xsd:string"/>

        <xsd:element name="TECH_COMPONENT" type="xsd:string"/>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

</xsd:schema>

Implementation

These options are useful in case you have custom OSB Reporting Data Handler which will be listening to the OSB JMS Provider QUEUE. OSB will perform reporting using the OSB JMS Provider QUEUE and message payload will contain ReportData and ReportMetadata. These variables can be XML elements or Strings. These will need to be retrieved in the custom OSB reporting handler and processed further.

More on implementation in jvoggels blog

BAM Dashboards

OSB has inbuilt features to monitor service’s health using metrics and alerts; however, this monitoring is mostly oriented toward technical users.    Oracle Business Activity Monitoring is the preferred product to give business users the functional monitoring on their business services and processes in the enterprise

·         BAM can be leveraged to build reports based on the data in the Reporting table.

·         The BAM dashboards are customizable and can define new dashboards as required.

·         It will be constrained by the way we define the data model, so will need to be sure we capture all the information required into the data model.

Notes

•  These can be implemented at Gateway level to capture the request and response so all processes can be monitored.
•  All process developers must be following naming standards in endpoint URI creation, so that we can identify the Module /service.
•  The correlation Id needs to propagate across the proxies so that in case of errors the same record can be updated with the Error status /error message.
•  Correlation Id can be transmitted using custom user headers in transport headers or as part of SOAP headers.

Tips on reporting in blog

References :

• http://jvzoggel.wordpress.com/2012/02/15/oracle-service-bus-logging-tracing-iii-create-custom-report-provider/
• http://jvzoggel.wordpress.com/2012/01/18/osb_tracing_report_action/
• http://blog.grummt.org/?cat=4
• http://eelzinga.wordpress.com/2009/07/17/oracle-service-bus-xquery-and-optional-parameters/
• http://docs.oracle.com/cd/E21764_01/doc.1111/e15867/reporting.htm
• http://www.oracle.com/technetwork/middleware/bam/osb-bam.pdf

Capturing Business Events @Oracle BPM 11g

        This blog will be the first in series of blogs I will be writing on Oracle BPM suite. In this blog I will look at different options available in Oracle BPM for tracking and monitoring the business process. So how do we get started on monitoring part? In BPEL we used to have sensors and it can be monitored on BAM dashboards, AIA had audit logging mechanisms which we had to invoke to get the information logged. In BPM we have BAM Data Objects as well as BPM Process Cubes. So what/how do we feed these monitoring mechanisms? That’s what we will be focusing on in this blog.

Capturing the business events

To get started with this the analyst need to follow the below steps

1.     Identify KPI’s and process metrics
2.  Create Business indicators to capture metrics
3.    Identifying Sampling points for capturing business indicators
4.   Enable publishing using BAM  or BPM Process Cubes
5.    Validate and deploy

  I will be focusing on the initial 3 steps, as step 4 is on enabling the BAM Data objects and BPM Process cubes, step5 on deployment.

1.    Identify KPI’s and process metrics

First step will be to identify the KPI’s and process metrics. KPI’s help an organization define and measure progress toward organizational goals. They are quantifiable measurements, agreed to beforehand, that reflect the critical success factors of an organization. Process metrics are measurements of the steps of a process. Steps generally have completion milestones, and processes are generally measurable in business terms such as timeliness and cost. 

2.    Create Business indicators to capture metrics

Business Indicators are project data objects that are used to store the value of the KPI’s of your process. i.e they can store the value of an indicator you want to measure in your process. Oracle BPM gives you 3 options to define business indicators, and it needs to chosen based on the information type of the value.

a.    Dimension

It defines specify how the data need to be picked. Dimension store the value of a KPI that you can use to group the values of the measure business indicators in your process.

eg:-region,ordertype

b.    Measure

Measure will help you capture numerical data which signifies a value which will help you in analytics

Eg:-salestotal

c.    Counters

Counters are type of measures used for no.of occurrence, keeps track of the number of times an instance completes a certain activity

3.    Identifying Sampling Points for capturing business indicators

                At runtime when the BPMN Service Engine runs the activity in the process it stores data about the process to the BPM Cubes and Oracle BAM Data Objects. This data comes from the sampling points defined in the project.

          Sampling points are broadly divided into two.

 a.    Build-in Sampling Point

-          configured  at project, process and activity level

o    generate sampling for all user tasks

o    generate sampling for start/end of process

o    generate sampling for start/end of activity

-          Dimensions are always captured

-          Sampling point settings can be overridden at different levels.

b.     User defined Sampling Point

- Measurement marks

Measurement marks store the following data into the Process Analytics databases:

• The value of the process default measures
• The value of the measure business indicators associated to that measurement mark
• The value of the dimensions defined in the process

(i)    Single Measurement

To sample business indicators at a specific point in process

(ii)   Counter Mark

It is used to track the number of times the instance runs the particular activity

(iii)                Interval(start & end)

This will measure a business indicator in a section of your process Always use these measurement marks to monitor critical sections of your process

 What Happens @ Measurement Marks

The BPMN Service Engine stores the values of the measure business indicators associated with the pair of measurement marks to the Process Analytics databases. It also stores the values for the dimensions and default measures for that process section.

That’s pretty much for today on the capturing of business events in BPM.

References:- Using Process Analytics in BPM