IIB (v9 & v10)-ContinuousIntegration-Maven-Jenkins

[UPDATE]:- This article has been updated on 08/24/2018 to support build of IIB 10 projects. Wherever is mentioned v9 only, read it as v9 & v10. .
V10 plugin allows you to make Shared libraries as maven projects & build them independently. Note that the plugin code has been updated to make sure that shared libraries are not included in the bar files of referring projects to avoid unplanned changes in shared library going into runtime.

You can leave your comment or reach out to me at anand.awasthi@gmail.com if need help



In this article I will explain how to build an IIB (IBM Integration Bus v9 & v10) project and deploy the bar file to target Integration Server using Maven and Jenkins. I have not found very useful information around this, so thought to put it together. Here I will take a basic scenario and will explain all steps. You should be able to use this information to build the CI infrastructure for IIB in your enterprise.

Scenario:
             -- Build IIB (v9 & v10) projects
            -- Create BAR file with all referenced libraries/projects/jars
            -- Overwrite BAR file Node properties and User Defined Properties using Properties file
            -- Deploy this BAR file on target Integration Server

Environment:
                   IIB v9,v10
                   Maven 3.3.3
                   Jenkins 1.632
                   IIB-Maven-Plugin (from github)
                   Maven Eclipse Plugin (m2e) v1.4.1
                   TFS (Team Foundation Server) 2012
Operation System:
                  Windows 7

Step 1: Make sure that IIB v9 runtime and Toolkit are installed on your Build server. I assume this is already in place.

Step 2: Install Maven 3.3.3 or latest version. I have used version 3.3.3 here. If you are new to Maven, don't worry. It's a very simple installation. Below are steps to do it:
   -- Download Maven 3.3.3 Binary Zip archive from https://maven.apache.org/download.cgi
   -- Extract distribution archive in any directory
   -- Ensure JAVA_HOME environment variable is set and points to your JDK installation
   -- Add the bin directory of the created directory apache-maven-3.3.3 to the PATH environment variable
   -- Confirm with mvn -v in a new shell. The result should look similar to

-- Optionally you can change the local repository location in maven settings.xml . You can find settings.xml in 'conf' directory inside your maven installation directory.

Step 3:
          Install IIB-Maven-Plugin:        
Below is the v9 plugin source code:
v9 plugin

Below is v10 plugin source code:

v10 plugin

After you download this plugin, unzip the file in any directory. You can install this plugin in central repository of your enterprise. Here we will install it locally.
Open command prompt and go to the directory where you extracted it. Navigate to the directory where POM file is present and run the command "mvn clean install"

This will install the iib-maven-plugin in your local repository. If installed successfully, you will see 'BUILD SUCCESS' message as below:

Step 4:
    Convert IIB Project to Maven Project: IIB projects are nothing but Eclipse projects. You can convert them to maven project using command line or you can use m2e plugin to use IIB toolkit for project conversion.We will use here m2e plugin. Follow below steps to install m2e plugin on your IIB toolkit:
          -- Go to Help --> Install New Software
          -- If not already there, add the site http://download.eclipse.org/technology/m2e/releases
          -- Make sure you uncheck the check-box 'Show only the latest versions of software available'. Install m2e-Maven Integration for Eclipse 1.4.1 and m2e-slf4j over logback logging v 1.4.1

Complete the installation and restart IIB toolkit.

Now change settings in your toolkit to refer to custom maven installation instead of Embedded installation. Go to Window-->Preferences-->Maven-->Installations. Add the maven installation directory.

Now checkout the IIB project into your IIB toolkit. Right click on the project --> Configure --> Convert to Maven project.

Provide the groupId, artifcatId and Version and select packaging as 'pom'.

Click on Finish.

Now we need to make configurations in POM file. Here I have put central repository info in POM; ideally we should put it in settings.xml. Below is a complete sample POM for the project I created:
Sample POM

Note that I have changed packaging to 'iib-bar'.
<groupId>com.endur.esb.dummy</groupId>
<artifactId>GeniusStatus_Dummy_App</artifactId>
<version>1.0</version>
<packaging>iib-bar</packaging>

You can update the central repository info or remove it. By default it will go to internet.

Now look at the Configuration block.
toolkitInstallDir: Path of the IIB toolkit installation directory
barName: Absolute path for bar file
workspace: Absolute path for workspace
versionString: Will be appended to Application name in bar file
cleanBuild: default value is 'true'
deployAsSource: default value is 'true'
esql21: default value is false. If you are compiling mqsi2.1 esql file then it should be set to true.
applicationName: Name of applications to be included in bar file separated by space
configPropFileDirectory: Absolute path for properties file to override bar file (UDPs and Node properties)
defaultPropertiesFile: Absolute path for properties file where the plugin dumps all the properties for internal use
debugWorkspace: Make this 'true'

Keep other settings same as that of in the sample POM.
Now your IIB Maven project is ready to compile. Make sure that the referenced projects are present in 'workspace'.

Step 5:
    Setup Jenkins:
Now let us setup Jenkins and trigger IIB Build from Jenkins.
You can just download the Jenkins WAR and use it with Tomcat or any other server. Here I will use Windows installer and setup Jenkins as Windows service.
Download the Windows installer from below url:

http://jenkins-ci.org/content/thank-you-downloading-windows-installer

It's installation is straightforward. Once installation is complete, you can access jenkins using below url:
http://localhost:8080
You can do access configuration in Jenkins using LDAP or AD server or local access setup. This part is not in scope of this article. We can continue without doing this access setup.

Go to 'Manage Jenkins' --> Configure System
Set the Maven Configuration.

Give the path to settings.xml

Save it and exit.

Step 6:
   Setup Jenkins job to build and deploy IIB bar:
Click on 'New Item' and give name to item. Select the check-box 'maven-project' and click 'Ok'

Now the Configuration page is open. We need to provide the configuration here. Please note that here I am setting to check-out only main project from TFS assuming that previous jobs will do a 'checkout only' of referenced projects in workspace. Provide the project paths and credential to checkout from TFS:

Select the appropriate Build Trigger.
Provide the absolute path for ROOT POM of the project.
Set the Goals and Options
Set the deployment command or batch file having deployment command.

Save the configuration.
You can do all other additional configurations as per your requirement.
Now you are all set to trigger your build.

I triggered the jenkins job and it built the bar file, overridden a node property using properties file and deployed this new bar file to target integration server. Attached is the sample properties files:
sample properties file

Below is the output of the build:

Getting version 'C89878' to 'C:\Anand\jenkins\workspace\GeniusStatus_Dummy_App'...
Finished getting version 'C89878'.
Parsing POMs
Established TCP socket on 52558
[GeniusStatus_Dummy_App] $ "C:\Program Files\Java\jdk1.7.0_79/bin/java" -Xms1024m -Xmx4096m -XX:PermSize=1024m -cp "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven32-agent-1.7.jar;C:\Anand\DevOps\apache-maven-3.3.3\boot\plexus-classworlds-2.5.2.jar;C:\Anand\DevOps\apache-maven-3.3.3/conf/logging" jenkins.maven3.agent.Maven32Main C:\Anand\DevOps\apache-maven-3.3.3 "C:\Program Files (x86)\Jenkins\war\WEB-INF\lib\remoting-2.52.jar" "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven32-interceptor-1.7.jar" "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.7.jar" 52558
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven:  -B -f C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\pom.xml -s C:\Anand\DevOps\apache-maven-3.3.3\conf\settings.xml -gs C:\Anand\DevOps\apache-maven-3.3.3\conf\settings.xml clean package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building GeniusStatusDummy 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ GeniusStatus_Dummy_App ---
[INFO] Deleting C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:prepare-bar-build-workspace (default-prepare-bar-build-workspace) @ GeniusStatus_Dummy_App ---
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:validate-bar-build-workspace (default-validate-bar-build-workspace) @ GeniusStatus_Dummy_App ---
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:create-bar (default-create-bar) @ GeniusStatus_Dummy_App ---
[INFO] Creating bar file: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\GeniusDummy.bar
[INFO] executeMqsiCreateBar command:  "C:\Program Files (x86)\IBM\IntegrationToolkit90\mqsicreatebar" -data C:\Anand\jenkins\workspace -b C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\GeniusDummy.bar -cleanBuild -version 1.0 -a GeniusStatus_Dummy_App -deployAsSource -trace -v C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\createbartrace.txt
[INFO] 
[INFO] C:\Anand\jenkins\workspace>"C:\Program Files (x86)\IBM\IntegrationToolkit90\mqsicreatebar" -data C:\Anand\jenkins\workspace -b C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\GeniusDummy.bar -cleanBuild -version 1.0 -a GeniusStatus_Dummy_App -deployAsSource -trace -v C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\createbartrace.txt 
[INFO] Command log file is set to: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\createbartrace.txt
[INFO] BIP0986I Command completed successfully.
[INFO] 
[INFO] 
[INFO] 
[INFO] 
[INFO] Classloaders are not in use. The following jars will be removed from the bar file: **/javacompute_**.jar,**/jplugin2_**.jar
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:validate-configurable-properties (default-validate-configurable-properties) @ GeniusStatus_Dummy_App ---
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\src\main\resources
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\src\test\resources
[INFO] Reading bar file: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\GeniusDummy.bar
[INFO] mqsireadbar complete
[INFO] Writing configurable properties to: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\default.properties
[INFO] Validating properties files
[INFO]   C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.properties
[INFO] Applying properties files as bar file overrides
[INFO]   C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.properties
[INFO] 
[INFO] C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App>"C:\Program Files (x86)\IBM\IntegrationToolkit90\mqsiapplybaroverride" -b C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\GeniusDummy.bar -o C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.bar -p C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.properties -k GeniusStatus_Dummy_App_1.0 -r -v C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\applybaroverridetrace-Genius.txt 
[INFO] BIP1137I: Applying overrides using toolkit mqsiapplybaroverride...
[INFO] BIP1140I: Overriding property GeniusStatus_DummyService_Flow#HTTP Input.URLSpecifier with '/Anand/DataArea/StatusService' in 'GeniusStatus_Dummy_App_1.0.appzip/META-INF/broker.xml' ...
[INFO] BIP1143I: Saving Bar file C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.bar...
[INFO] 
[INFO] BIP8071I: Successful command completion.
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:clean-bar-build-workspace (default-clean-bar-build-workspace) @ GeniusStatus_Dummy_App ---
[INFO] debugWorkspace enabled - workspace will not be cleaned
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:validate-classloader-approach (default-validate-classloader-approach) @ GeniusStatus_Dummy_App ---
[INFO] Reading configurable properties from: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\iib\default.properties
[INFO] 
[INFO] --- iib-maven-plugin:2.1-SNAPSHOT:package-iib-bar (default-package-iib-bar) @ GeniusStatus_Dummy_App ---
[INFO] Building zip: C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\GeniusStatus_Dummy_App-1.0.zip
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:31 min
[INFO] Finished at: 2015-10-08T08:52:11-04:00
[INFO] Final Memory: 25M/982M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\pom.xml to com.endur.esb.dummy/GeniusStatus_Dummy_App/1.0/GeniusStatus_Dummy_App-1.0.pom
[JENKINS] Archiving C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\target\GeniusStatus_Dummy_App-1.0.zip to com.endur.esb.dummy/GeniusStatus_Dummy_App/1.0/GeniusStatus_Dummy_App-1.0.zip
channel stopped
[GeniusStatus_Dummy_App] $ cmd /c call C:\WINDOWS\TEMP\hudson9075319975353138395.bat

C:\Anand\jenkins\workspace\GeniusStatus_Dummy_App>C:\"Program Files"\IBM\MQSI\9.0.0.3\bin\mqsiprofile   && mqsideploy -i hostname -p port -q QMHM1DEV3 -e ClaimsServices -a C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.bar -w 180 

MQSI 9.0.0.3
C:\Program Files\IBM\MQSI\9.0.0.3

BIP1044I: Connecting to the queue manager...
BIP1060I: Connecting to the broker...
BIP1115I: Connected to the broker 'IBHM1DEV3'.
BIP1039I: Deploying BAR file 'C:\Anand\SourceCode\EA.Integration\ClaimsFeedService\GeniusStatus_Dummy_App\Genius.bar' to broker 'IBHM1DEV3' (execution group 'ClaimsServices') ...
BIP1092I: The broker successfully processed the deployment request.
Finished: SUCCESS