Let's build WSO2 Carbon from source code (For newbies to WSO2 Carbon)

WSO2 Carbon is the base platform for all WSO2 Java products.

Built on OSGi, Currently In Carbon there are 4 main SOA functionalities. They are as follows.

1. WSO2 Web Services Application Server (WSAS)
2. WSO2 Enterprise Service Bus (ESB)
3. WSO2 Registry
4. WSO2 Business Process Server (BPS)

Here are some advantages of building a Open Source Software from the source

If you are willing to develop WSO2 Carbon , it is essential to build WSO2 Carbon from source rather than using a binary distribution. Building from source ensures that the source exists and compiles properly, which is the basis of regular open-source development.

By compiling from source, if you find some code you are able to isolate patches to address such problems, without having to wait for upstream.

Although bytecode runs on every vm that supports the version of the bytecode in question it does not mean that the bytecode produced by different compilers is equal. Bytecode can be optimized too. By building from source, we make it easy to switch the compiler used to compile your installed java programs.

Let's start build Carbon from source code

First of all you need Subversion (SVN) which is a version control system widely used in open source software development. You can install SVN from here based on your operating system.

Then also you need to install the build tool called Apache Maven. It is a open source Apache build manager for java projects. You can install Apache Maven from here.

Go to WSO2 Oxygen Tank SVN repository. Here you will see all the SVN repositories for all WSO2 products.

For WSO2 Carbon you have to checkout the following SVN repositories one by one to a specific location in local hard disk.

• https://svn.wso2.org/repos/wso2/branches/carbon-platform/2.0
• https://wso2.org/repos/wso2/trunk/carbon-orbit
• https://wso2.org/repos/wso2/trunk/carbon
• https://wso2.org/repos/wso2/trunk/carbon-components

For that use "svn checkout" command.
e.g. ...../carbon$ svn checkout https://svn.wso2.org/repos/wso2/branches/carbon-platform/2.0
Now there will be a directory structure created as as follows.


Then you will realise that there is a specific directory for each checkout you made. In each of those dirctoriers you will find (at the root level) a file called "pom.xml". It represent the complete Maven project and using that you can build the project from the source code.

Before build the project first of all we need to install the following plugins and modules which enables you to build the above project source codes successfully. Those plugins source codes are also get downloaded when checking out the https://svn.wso2.org/repos/wso2/branches/carbon-platform/2.0/axis2/ repository. You just have to install them using Maven.

For that go to "axis2" directory which is the first checkout you made. In that go to "modules/tool".
There you can see "axis2-aar-maven-plugin" and "axis2-mar-maven-plugin" named directories. In those directories you will see the pom.xml file for each. You have to go to those directories one at a time and build those plugins using Apache Maven with following commands in the console.

mvn clean install
(This is a command in Maven use to build the project with the help of "pom.xml" file and for more information use this Quick Guide)
After successfully install those plugins you can start to build the above checkedout projects one by one.

In console use command "mvn clean install -Dmaven.test.skip=true" in the root level of each directory as follows.

e.g. To build "axis2" go to the "axis2" directory , which is the directory the source is checked out from the repository https://svn.wso2.org/repos/wso2/branches/carbon-platform/2.0/axis2/. In console use the command as follows.

• $current-location/axis2$ mvn clean install -Dmaven.test.skip=true

Such that use the following commands under the given directories with the given sequence.

• To build transports-> $curr.-location/2.0/transports$ mvn clean install -Dmaven.test.skip=true
• To build wss4j-> $curr.-location/2.0/wss4j$ mvn clean install -Dmaven.test.skip=true
• To build rampart-> $curr.-location/2.0/rampart$ mvn clean install -Dmaven.test.skip=true
• To build sandesha-> $curr.-location/2.0/sandesha$ mvn clean install -Dmaven.test.skip=true
• To build savan-> $curr.-location/2.0/savan$ mvn clean install -Dmaven.test.skip=true
• To build synapse-> $curr.-location/2.0/synapse$ mvn clean install -Dmaven.test.skip=true
• To build carbon-orbit-> $curr.-location/carbon-orbit$ mvn clean install -Dmaven.test.skip=true
• To build carbon-> $curr.-location/carbon$ mvn clean install -Dmaven.test.skip=true
• To build -carbon-components> $curr.-location/carbon-components$ mvn clean install -Dmaven.test.skip=true

Note - You have to build those 10 soucre directories one at a time and according to the given sequence as the later project source codes are depend on a prior project source codes. "ode" and "xmlsec" source directories are not required to be built.

After successfully building all 10 projects' source codes WSO2 Carbon is built from the source.

As well you can create IDE(Eclipse, IDEA, NetBeans) based projects using the Maven plugins.

Home made FOSS

This blog is focused on starting a simple FOSS (Free and Open Source Software) project on your own based on the free resources from the Internet.
Let's look at the basic resources associated with an open source project.


Source code repository
Source code repository is used to store the whole collection of source code used to build a particular component or an application. Typically, the source code includes only human-written source code files, and not, e.g., source code files generated by other tools or binary library files. However, it generally does include configuration and property files. A source code repository becomes handy when the project is developed by many people around the world, which is a normal scenario for large open source projects. Source code repository enables the developers to work together on the project as well people who are interested in the project can go through the source code.
- There are many project hosting sites you can use for your project. All most all of them support for source code hosting. Some of them are as follows.

• Google-code
• Source-Forge

You can get more details about project hosting sites here.

There are other project hosting sites such as mozdev.org which provide project hosting services for projects under specific requirements. You can find more details on those project hosting sites here.

- The following snapshot is from Google code where the initial project creation take place.


- Here you can choose the source code license based on you choise. This source code license determines how others can use your code. Read more informations on the open source licenses on Open Source Initiative.

Executable binaries
- The executable binary is created by converting written human readable programs (called "source code" or simply "source" or "code") into machine language in a process that is called "compiling". Source code can be read and modified (if you know the language), which means, if you had unfettered access to it, you can also alter it to suite your needs. So normally executable binaries are released based on the operating platform.
eg -

• Zipped (.zip) archives for Windows platform (for each version of the product)
• Tar balls (.tar.gz) for Linux platform (for each version of the product)

Documentation
- Documentations are also very important for a FOSS project as it either explains how it operates or how to use it, and may mean different things to people in different roles. Types of the documentation are,

• Requirements - Statements that identify attributes, capabilities, characteristics, or qualities of a system. This is the foundation for what shall be or has been implemented.
• Architecture/Design - Overview of software. Includes relations to an environment and construction principles to be used in design of software components.
• Technical - Documentation of code, algorithms, interfaces, and APIs.
• End User - Manuals for the end-user, system administrators and support staff.
• Marketing - How to market the product and analysis of the market demand.

Here is a snapshot of the project's downloads page in Google code. You can see there are executable binaries and some documentations as well.


Nightly build downloads
- This means creating daily builds from the source code. Nightly builds may be used for testing purposes. Sometimes these builds may be used by the users. Ultimately these builds helps to track whether the source code is stable or not up-to the build date.

Project mailing list
- Normally all most all the discussions related to an open source projects are happened via mailing lists. Anybody interested in a particular FOSS project can join to the public mailing lists hosted for that particular FOSS project. Normally there are two public mailing lists knows as Developer mailing list and User mailing list for a project.
Developer mailing list is used for discussions related to project development. User mailing list is used for discussions from user perspective. Some times there may be private mailing lists related to internal discussions.

Project wiki
- Project wiki is a set of wiki pages related to the project details. People can edit create new pages, edit them. So project wiki is some kind of a collaborative thing rather than documentation which is produced by a particular set of people. Here is a snapshot of a project wiki page on Google code.


Issue tracking systems
- For a software project an issue tracking system is an essential resource. It manages and maintains the list of issues related to project. People can create, update, comment, resolve, close etc. the issues regarding the project. Here you can find more details regarding the features of issue tracking systems. Here is a snapshot on project issue tracker page on Google code.


Forums
- On-line forums are also used on discussions regarding the project. Logically forums are organized into a finite set of generic topics (usually with one main topic) driven and updated by a group known as members, and governed by a group known as moderators.


To commence a FOSS project there are many project hosting sites like Google-code, Source-Forge etc. Google-code is one of the most easiest and the flexible project hosting service you can rely on. You can also try others as well.
Google-code provide basic resources required for a basic FOSS project. Here are some other snapshots of the project site hosted on Google-code.

• Project home page

• Project source code details

How to deploy an Axis2 Web service programatically in ODE during the initialization of ODE Runtime

Description on ODE
Apache ODE (Orchestration Director Engine) executes business processes written following the WS-BPEL standard. It has two communication layers. One is Axis2 integration layer and the other one is based on JBI standard. Those integration layers are used by ODE BPEL Engine Runtime for interact with the outside world. Axis2 integration layer supports for communicate via Web Service interactions. JBI integration layer supports for communicate via JBI messages. Here I have discussed about how to deploy an Axis2 service programatically in ODE during the initialization of Apache ODE.

Why deploying services at the initialization
We can deploy Axis2 web services in ODE in various ways. One way is just dumping and AAR (axis2 archive) in the services directory inside the ODE's deployment directory. In this method axis2 service can be deployed and undeployed them as user's choice. But suppose if we need to deploy a web service which provides a common functionality for all deployed business processes in ODE like ODE Management API then those kind of Web services need to be deployed during the initialization of ODE. ODE Management API is a axis2 Web service deployed in ODE during the initiation of ODE Run Time. This service monitor and manage all the business processes deployed in ODE. Another service which is deployed as same before is Deployment service which is used for deploy WS-BPEL processes. So if someone interested in developing a Web service which act as a generic feature for ODE like ODE Management API , that Web service must be deployed programatically during the initiation of ODE run time.

Basic overview of the process
How to deploy an axis2 Web service generally
There are several ways of deploying a axis2 Web service programatically.

1. One method is using the service archive file (AAR distribution).
Here we create an object of service archive file and a pointer to axis2 runtime or configuration context. Using the below code we can create the web service as follows.


This process is is almost same as putting the service archive file in the deployment directory.

2. Another method does need neither a services.xml nor a repository. In this POJO deployment it is only need axis2-1.4.jar and other dependent libraries.


Here MyService is the service implementation class. In this method as a services.xml is not existed, the degree of freedom for configure the service is very much less.

3. Final method does not need a services.xml, but it gives the freedom to configure the service such as by specifing message receivers etc. I prefer this method to deploy a axis2 Web service programatically during the initiation of ODE runtime.


Here when creating the service the message receiver is set and for more information refer org.apache.ode.axis2.service.ManagementService class. In org.apache.ode.axis2.service.ManagementService.DynamicMessageReceiver you may see how the service operations are invoked.

How to deploy the axis2 service in ODE runtime
Let see how to how to deploy an axis2 service programatically in ODE runtime. In this section it contains the ODE specific information which related to the axis2 service deployment.

ODEServer.java
To deploy a service in ODE runtime programatically org.apache.ode.axis2.ODEServer is normally used. Actually this class is the starting point for a execution of a ODE WAR distribution. If you look at the source code you may see almost all the basic resources are initialized in the org.apache.ode.axis2.ODEServer#init method.


In the init() method above syntaxes are used to deploy the Process ans Instance management service which is a basic management service in ODE runtime. In the above enableService() method the service deployment logic is implemented. That implementation can be any method as I described on the previous section.

Writing WSDL and the schemas and put the into the correct modules
Now the required WSDL and the related schemas should be placed in specific places on the ODE source in order to deploy the service.
Location for the WSDl - axis2/src/main/wsdl/
Location for the related schemas - bpel-schemas/src/main/xsd/

These resources should be kept in those specific packages, othervise while creating the WAR distribution required code generations, schema compilations will not be happened.

Creating .xsdconfig file for a schema
As I explained earlier schema files should be kept in a specific location (bpel-schemas/src/main/xsd/) in order to happen the code generation process succesfully. As well for each schema file it is required to created another file with the same name of the schema file and with .xsdconfig extention at the same location where the schema files are existed. This .xsdconfig file is needed for denote the target package for the generated bean classes. For example for pmapi.xsd the .xsdconfig file is pmapi.xsdconfig.

RakeFile Modification
After that, the "Rakefile" located at the root of ODE source should be configured. Rakefile contains executable Ruby code. Using that Ruby code, Apache Buildr will build the ODE source. In Rakefile it is need to mention about the newly added schema files in order to code generations and to populate the schema file in the WAR distribution. At the following places in the Rakefile you need to modify the Ruby code based on the added schemas.


Note: If you add the schema definitions in the WSDL itself, then there is no way of bean geaneration for thoes schemas.

Compilatation of schemas and bean class generation is handled by the following Ruby code snippet.


Rebuilding using buildr
Now re-building the source code using Apache Buildr, a WAR distraibution will be generated with the modifications you made.