An introduction to OSB 12c Maven plug-ins

Recently I started exploring the possibilities with the OSB Maven plug-in that is provided with the 12c product. At my current client we are slowly moving towards a more CI/CD environment and using a Maven plug-in to package OSB projects and resources is an important step in this. When the Jenkins pipeline kicks off, one of the stages is to package the OSB project using this plug-in. But I was surprised by how limited configuration settings are.

The plug-in can be found in %ORACLE_HOME%\osb\plugins\maven\com\oracle\servicebus\plugin\oracle-servicebus-plugin\
We are currently on version:

<groupId>com.oracle.servicebus.plugin</groupId>
<artifactId>oracle-servicebus-plugin</artifactId>
<version>12.2.1-3-0</version>

The accepted parameters for packaging projects are as follows:

<parameters>
 <parameter>
  <name>excludes</name>
  <type>java.lang.String[]</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies a list of files to exclude from the project. Use this to exclude things like versioning system files.</description>
 </parameter>
 <parameter>
  <name>oracleHome</name>
  <type>java.io.File</type>
  <required>true</required>
  <editable>true</editable>
  <description>Specifies the location to the Oracle Fusion Middleware home directory. You can specify this value as an expression.</description>
 </parameter>
  <parameter>
  <name>project</name>
  <type>org.apache.maven.project.MavenProject</type>
  <required>true</required>
  <editable>false</editable>
  <description></description>
 </parameter>
 <parameter>
  <name>system</name>
  <type>boolean</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies whether the resources being packaged are system resources, which are shared by multiple projects within a Service Bus application. The default value is false. You must set this value to true when packaging system resources.</description>
 </parameter>
</parameters>

An example of usage (this piece of code is inserted in the project's pom.xml file):

<plugin>
   <groupId>com.oracle.servicebus.plugin</groupId>
   <artifactId>oracle-servicebus-plugin</artifactId>
   <version>12.2.1-3-0</version>
   <extensions>true</extensions>
   <configuration>
      <oracleHome>${oracle_home_12c}</oracleHome>   
      <project>${project.artifactId}</project>
      <system>false</system>
      <excludes>
         <exclude>*/.idea/**</exclude>
         <exclude>*/Test/**</exclude>
         <exclude>*/properties/**</exclude>
         <exclude>*/Version/**</exclude>
      </excludes>
   </configuration>
</plugin>

The configuration settings are used by the plug-in to create a ConfigJarSettings file. It determines what resources are included and excluded in the package. The plug-in uses the offline export tool (configuration jar) to package Service Bus resources.

In this version of the plug-in I'm missing the ability to control whether projects are exported on project or resource level. The latter is important for specific OSB projects where you only need an incremental to be installed and not an entire project to be overwritten, for example DVM's where values could differ between environments. If this is your requirement in version 12.2.1-3-0 you might want to take a look over here: https://whitehorsesblogarchive.wordpress.com/tag/osb/ On their Bitbucket you will find a customized plug-in which allows you to build projects on resource level using Maven.

Or you could consider upgrading to:

<groupId>com.oracle.servicebus.plugin</groupId>
<artifactId>oracle-servicebus-plugin</artifactId>
<version>12.2.1-4-0</version>

This version allows you to control the exportLevel and which resources need to be in- or excluded. The parameters are as follows:

<parameters>
 <parameter>
  <name>excludes</name>
  <type>java.lang.String[]</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies a list of files to exclude from the project. Use this to exclude things like versioning system files.</description>
 </parameter>
 <parameter>
  <name>exportLevel</name>
  <type>java.lang.String</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies the export level either RESOURCE or PROJECT. You can specify this value as an expression.</description>
 </parameter>
 <parameter>
  <name>includes</name>
  <type>java.lang.String[]</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies a list of files to include from the project. Use this to include things like versioning system files.</description>
 </parameter>
 parameter>
  <name>oracleHome</name>
  <type>java.io.File</type>
  <required>true</required>
  <editable>true</editable>
  <description>Specifies the location to the Oracle Fusion Middleware home directory. You can specify this value as an expression.</description>
 </parameter>
 <parameter>
  <name>project</name>
  <type>org.apache.maven.project.MavenProject</type>
  <required>true</required>
  <editable>false</editable>
  <description></description>
 </parameter>
 <parameter>
  <name>resources</name>
  <type>java.io.File</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specify the resources, which points towards a configuration file. This can be used to specify which files need to be included and excluded.</description>
 </parameter>
 <parameter>
  <name>skipAll</name>
  <type>boolean</type>
  <required>false</required>
  <editable>true</editable>
  <description>Setting skipAll as true will skip depolyment of sbar.</description>
 </parameter>
 <parameter>
  <name>system</name>
  <type>boolean</type>
  <required>false</required>
  <editable>true</editable>
  <description>Specifies whether the resources being packaged are system resources, which are shared by multiple projects within a Service Bus application. The default value is false. You must set this value to true when packaging system resources.</description>
 </parameter>
</parameters>

More information regarding the plug-in can be found on the following Oracle page: https://docs.oracle.com/en/middleware/soa-suite/service-bus/12.2.1.4/develop/using-oracle-service-bus-development-maven-plug1.html#GUID-F3308599-234C-48FF-B74B-4F479100ABA0

Reactive vs pre-emptive HTTP authentication

Seeing double requests in your access log for HTTP authentication with status codes 401 and 200? Read further..

In this blog I will show you how to control the basic HTTP authentication flow with Oracle SOA Suite.

When we implement some sort of HTTP authentication, like the commonly used Basic Authentication, we have to deal with the standard HTTP Challenge/Response Authentication Framework. This means that whenever a client request is sent to the server, the server can challenge the client by sending a request back to provide authentication details. This request is accompanied by an HTTP status 401 Unauthorized and a WWW-Authenticate header which includes at least one authentication challenge. The client then repeats the request providing the credentials in the Authorization header to the server. If the credentials are correct, the server responds with the requested data and an HTTP status code 200 OK.

The diagram below shows how this works:

Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication

We can actually see this flow in practice using SoapUI to call a web service that is secured with basic authentication. Using an HTTP traffic sniffing tool like Fiddler we can see the data being sent back and forth. Setup your SoapUI first by making sure the check box "Authenticate Preemptively" is not checked.

When submitting a request make sure you've added the credentials for Basic Authorization. Because we disabled the "Authenticate Preemptively" feature just now, we will select the "Use global preference" option.

In Fiddler we can now identify two responses when a request is submitted.The first one is the 401:

We can see that along with the status code 401 Unauthorized the WWW-Authenticate header is being sent. It tells us that Basic Authentication is used and the security realm is "owsm".
In the next request we provide these details in the header: "Authorization: Basic <base64 encoded credentials>". As these credentials are correct, an HTTP status code 200 OK is returned.

If you don't see anything logged in Fiddler, try adding the following proxy settings to SoapUI:

In the access log of the HTTP server we can see these responses as follows:

This kind of authentication is also called "reactive" authentication. Now that we know how HTTP authentication works, we can imagine that those extra calls to the server brings extra load on the network. Luckily, as opposed to reactive authentication, there is also something called "pre-emptive" authentication. In this type of authentication the authentication challenge is skipped. This means the credentials are provided directly in the first request to the server. This also means a successful request with HTTP status code 200 HTTP is not preceded by a status code 401 Unauthorized. This way we can reduce the number of requests to the server.

We can easily implement this in SOA Suite. Just add the following property oracle.webservices.preemptiveBasicAuth to the binding of your external service and set it to "true". This will enable pre-emptive authentication.

This property can also be changed run-time via the Enterprise Manager. To do so, navigate to your SOA Composite and then to the Service/Reference Properties for your service. In the right upper corner you can change the settings.

When our BPEL service now calls the external service with Basic Authentication no challenge is needed and the Authorization header is sent directly:

And the access log doesn't contain the 401 results anymore, keeping it clean and tidy:

Throttling JMS adapters in SOA Suite

When working with JCA adapters that need to process huge amounts of transactions per second it's important to have some sort of control over the processing. Failing to do so can lead to memory issues and stuck threads on your servers. In this blog I will show you how you can implement a throttling mechanism on inbound JMS adapters. This way you can control the number of transactions processed per time interval. I will also show you how you can adjust the properties run-time in the Enterprise Manager.

The following two properties will be used for our JMS adapter:

1) adapter.jms.receive.threads: Specifies the number of poller threads that are created when an endpoint is activated. The default is 1.
2) minimumDelayBetweenMessages: Default is False, or no delay. Inbound-only. This property is configured in milliseconds. Ensures that there at least will be MILLI_SECONDS delay between two consecutive messages posted to the downstream composite application.

Note that minimumDelayBetweenMessages is effective per adapter polling thread. If you have configured multiple adapter polling threads, this setting controls the delay between messages processed by each thread only. 

Source: https://docs.oracle.com/middleware/1221/adapters/develop-soa-adapters/GUID-2BB20502-6F62-462F-A490-3AF301AB6089.htm#TKADP2128

I'm working with SOA Suite 11g but these properties exist for 12c too as seen in the Oracle documentation.

These properties are set in the composite.xml as part of the service definition that represents your JCA adapter, as follows:

When setting the values you have to think about what number of transactions per second is desirable. You also need to take the number of managed servers in your environment into account. The following formula could be helpful:

(Number of Managed Servers x thread count) / (minimumDelayBetweenMessages / 1000) 

Calculation example:
-There are two managed servers.
-Each adapter will handle two threads at the same time (set via property adapter.jms.receive.threads = 2).
-Each thread waits at least half a second before processing a new transaction (set via property minimumDelayBetweenMessages = 500ms).

When the formula is applied that means (2 x 2) / (500  / 1000) = 8 transactions per second will be processed per JMS queue.

It's possible to adjust these values run-time. This is done via the Enterprise Manager. Navigate to your SOA Composite and then the Service/Reference Properties:

Adjust the values and click 'Apply'.

Let's do some testing now. To view the results unambiguously, I will just use one active managed server with one queue and one thread per adapter so we can clearly see the time interval.

I will use the following values for the properties: 

<property name="adapter.jms.receive.threads">1</property>
<property name="minimumDelayBetweenMessages">2000</property>

Now according to the formula (Number of Managed Servers x thread count) / ( minimumDelayBetweenMessages / 1000) that means we have (1 x 1) / (2000 /1000) = 0,5 transaction per second. Or 1 transaction every 2 seconds per queue.

I will dump a batch of messages on one JMS queue. Then we will check the logs how the JMS adapter has processed those messages. Of course we would like to see that indeed 1 message was processed every 2 seconds.

First we need to adjust the Diagnostic Logging Level. This is done via the Log Configuration in the Enterprise Manager. Set the level for the oracle.soa.adapter logger to TRACE. This is the logging for JCA adapters.

 
When the batch of messages is imported on the JMS queue to be processed we can check our logging:

 
If we look at the Time column we can verify that the JMS adapter indeed processed 1 message every 2 seconds, great!