Maven inheritance aggregation

Challenge

The challenge is to create a build system (based on maven of course) that will be delivered to the next layer of development. The difficulty here is that the core team will continue to change the delivery system, and release new versions. We do not want (or would like to limit) the adoption teams to always have to merge when there is a new version.

 

Solution

The solution needs to be implemented in more than one single solution and way.  The standard way of maven is inheritance. Any dependency or plugin that you want to add you put it in the parent pom and everyone can inherit this. The problem is that we do not always want to inherit, and we don’t always want the same functionality for all children. To solve this issue you can put any dependencies or plugins in profiles and then activate them in a child module. This solution is not clean since it is very cumbersome and makes the parent pom very big and hard to manage with all the profiles.

BOM (Bill Of Materials)

Since our project consists of multiple jars and versions we do not want the adoption team to have to update all versions for every new release. In addition if we add a new jar we do not want the adoption team to have to add the new jar to their parent pom.

By creating a bom file (more a less a parent pom), the adoption parent pom does not have to inherit your pom, but can import it (http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html). This will allow the adoption to manager the core product with one import on the dependency.

For example (this is put in the dependencyManagement section):

                                                <dependency>

                                                                <groupId>com.xxx.wsf.buildtools</groupId>

                                                                <artifactId>wsf-bom</artifactId>

                                                                <version>${wsf.tools.version}</version>

                                                                <type>pom</type>

                                                                <scope>import</scope>

                                                </dependency>

This will import all the dependency managements from the wsf-bom file.

Just a reminder in the BOM you only have dependencyManagement and not the dependencies themselves. In addition BOM’s are only for dependency management and not plugin management.

Note: When you have a multi module project, you parent pom needs to be the bom so that it is installed first, else no one is installing it since it is imported in the dependencyManagment and not dependencies.

DepChain

A BOM file is a good start, but what do I do if I have multiple projects that need the actual dependencies and not just the versions (dependency management). If it is only one dependency then we can manually add it to the module, but if it is more than one, or even one but in the future it will grow we would like to import the dependencies themselves and not just the dependency management.

A depchain is a pom file that has the actual dependency list and not just the management. Once you have an artifact that is a depchain you can then import it into any module (this is put in the dependencies section:

                                <dependency>

                                                <groupId>com.xxx.wsf.buildtools</groupId>

                                                <artifactId>wsf-depchain-design</artifactId>

                                                <type>pom</type>

                                                <scope>provided</scope>

                                </dependency>              

This will bring in the actual dependency list into your module, notice that the scope is provided.

For more information see: http://books.sonatype.com/mvnref-book/reference/pom-relationships-sect-pom-best-practice.html.

 

Mixins

Mixin is an object oriented paradigm to add functionality to a class but not by inheritance (http://en.wikipedia.org/wiki/Mixin). Maven has toyed with the idea of adding mixins, currently it is on the roadmap for maven 4 (https://cwiki.apache.org/confluence/display/MAVEN/Roadmap). All thought it is not released by maven, maven allows you to create plugins that are extensions to maven. So a developer by the name of ohad david has written a mixin extension for maven (https://github.com/odavid/maven-plugins).

What this extension allows you to do is to create an external pom that will be merged into your pom. So with this you can write a set of plugins that will give you added value, and then you import it into any module you need this functionality (you also have the option to move the content of the pom to a mixin.xml so that when you install the mixin it does not run the plugins).

For example we would like to write a set of plugins that depending on the OS it will change the permissions of a file in a folder. The pom that does this is fairly simple. What we would like is the option to move this functionality from the parent pom into anther pom that other projects can use. Just like the BOM allows you to import all dependencies version of a project, a dep chain will allow you import the dependencies them self’s.

Example of mixin: permission per os (the value of the properties will be defined in the calling pom).

The pom of the mixin might look something like:

<properties>

<folder.location/>

<windows.permissions>-r /s</windows.permissions>

<linux.permissions>777</linux.permissions>

</properties>

<profiles>

<profile>

  <id>windows</id>

  <activation>

      <os>

         <family>windows</family>

      </os>

   </activation>

       <build>

                <plugins>

                    <plugin>

                        <groupId>org.apache.maven.plugins</groupId>

                       <artifactId>maven-antrun-plugin</artifactId>

                      <executions>

                          <execution>

                             <id>run app maven</id>

                                <phase>validate</phase>

                                <configuration>

                                    <target>

                                        <exec dir="${folder.location}" executable="attrib" failonerror="true">

                                            <arg line="${windows.permissions} *.*"/>

                                        </exec>

                                    </target>

                                </configuration>

                                <goals>

                                    <goal>run</goal>

                                </goals>

                         </execution>

                     </executions>

                </plugin>

         </plugins>

    </build>

</profile>

<profile>

     <id>linux</id>

         <activation>

             <os>

                    <family>unix</family>

                </os>

         </activation>

        <build>

            <plugins>

                   <plugin>

                       <groupId>org.apache.maven.plugins</groupId>

                       <artifactId>maven-antrun-plugin</artifactId>

                       <executions>

                            <execution>

                                    <id>set permission</id>

                                    <phase>process-test-resources</phase>

                                    <goals>

                                         <goal>run</goal>

                                   </goals>

                                  <configuration>

                                       <target>

                                            <chmod dir="${folder.location}" perm="${linux.permissions}"/>

                                       </target>

                                </configuration>

                          </execution>

                     </executions>

             </plugin>

        </plugins>

    </build>

</profile>

The usage of this plugin in another pom would then be:

<properties>

                <folder.location>${basedir}/src/test/resources/asm</folder.location>

</properties>

<plugin>

                <groupId>com.github.odavid.maven.plugins</groupId>

                <artifactId>mixin-maven-plugin</artifactId>

                <extensions>true</extensions>

                <configuration>

                                <phase>process-classes</phase>

                                <mixins combine.children="append">

                                                <mixin>

                                                                <groupId>com.abc</groupId>

                                                                <artifactId>wsf-mixin-files-permission</artifactId>

                                                                <type>mixin</type>

                                                </mixin>

                                </mixins>

                </configuration>

</plugin>                           

Why mixins still don’t make it:

Although mixins do give us a lot of functionality, they do not really wrap a full set of functionality to be used in other poms. For example is you write an assembly mixin, all the extra files for the assembly cannot be put in the mixin but need to be put into the calling pom. So we don’t actually have a full encapsulation solution, but a step in the right direction.

For more information on this see:

http://rethinkingswd.blogspot.co.il/2014/09/mixin-maven-plugin-reusable-project.html

Maven Profiles - per module

Maven Profiles

Activation per module

Maven profiles are very powerful (http://maven.apache.org/guides/introduction/introduction-to-profiles.html).

One of the hardest parts of maven, in my opinion, for someone coming from a programing background, is that the flow of maven is not a method call.

In a lot of cases we would like to create a batch process (group of plugins) that do something, and then invoke them in different modules. So what you would expect to do is to create a profile with all the plugins that you want, and then to active it on a module level.

For instance a simple example is to download all dependencies of the project to the target directory. In this case we will define the profile in the parent pom as follows:

<profile>

  <id>trace</id>

  <build>

    <plugins>

          <plugin>

               <artifactId>maven-dependency-plugin</artifactId>

               <groupId>org.apache.maven.plugins</groupId>

               <executions>

                 <execution>

                       <phase>package</phase>

                       <goals>

                         <goal>copy-dependencies</goal>

                       </goals>

                       <configuration>

                         <outputDirectory>${project.build.directory}/lib</outputDirectory>

                       </configuration>

                 </execution>

               </executions>

          </plugin>

         </plugins>

  </build>

</profile>

Now the problem is that when I invoke this profile it will be active on all modules. In this case it might be nice, but what if I have multiple modules and want to invoke the profile from only some of the modules.

This is where profile activation comes in. There is an option to activate a profile based on a system parameter like:

  <profile>

    <activation>

      <property>

        <name>debug</name>

      </property>

    </activation>

  </profile>

This will activate the profile if the system property of “debug” exists. Of course you can also add the option to check the value of the parameter:

  <profile>

    <activation>

      <property>

        <name>environment</name>

        <value>test</value>

      </property>

    </activation>

    ...

  </profile>

What is missing is that this feature only works with system parameters and not maven parameters. So you cannot set a parameter per module with a different value, and have maven evaluate it.

Another issue that you must always remember: when running maven, maven will compile the pom and create an effective pom (you can run mvn “effective-pom” to view it) and then run it. This means that you cannot do any manipulation during runtime that will affect the run of maven.

Solution – files

The solution that I found to run profiles per module is the use of file activation. Let’s say that I have some test modules. Sometimes I want to run a tomcat before running the tests and others not (just junit). In the standard way I will need to create a “tomcat” profile in each pom, and copy all the code that configures and deploys the pom.

I want to write the profile once in the parent pom, and then activate it only in those modules that need it. To do this I create the profile in the parent pom, and add the following activation:

<profile>

        <id>tomcat</id>

        <activation>

               <file>

                       <exists>profile.tomcat.active</exists>

               </file>

        </activation>                  

        ...

</profile>

Now for each module that I want to activate this profile, I need to have the file profile.tomcat.active in the pom directory. This will activate the profile, and the file needs to be there before you run the maven command, so you cannot copy it during the maven run.

This can also be used in a Jenkins scenario, where in the Jenkins job before running the maven, you add the files or delete them and the run the maven via Jenkins.

linux autocomplete - maven

One of the very nice feature of linux is the option for auto complete on the command line. Yes windows has the power shell, but it is not the same as linux.
For those in windows and want a linux shell you can install Cygwin (http://www.cygwin.com/).

For those of us that use maven, think of how nice it would be to have auto complete for maven commands.
Well this is possible, have a look at:
http://maven.apache.org/guides/mini/guide-bash-m2-completion.html

If you are using Cygwin then just copy the script to \cygwin64\etc\bash.bashrc

Create a Java CLI application with Maven

Creating a CLI application is part of our daily routine. In java it is strait forward; all you need to do is to add a class with the main function:

public static void main(String[] args) {

}

This is nice for debugging from eclipse, but if you need to release this jar with the cli working, you will face the following issues:

1.       You need to add the main class to you manifest file and compile it in the jar.

2.       You need to create a folder with all the jar’s dependencies needed to run this cli.

Main Class

maven-jar-plugin

The first issue can be solved by using the maven-jar-plugin:

<plugin>

       <groupId>org.apache.maven.plugins</groupId>

       <artifactId>maven-jar-plugin</artifactId>

       <configuration>

              <archive>

                     <manifest>

                           <addClasspath>true</addClasspath>

                           <mainClass>com.package.Main</mainClass>

                     </manifest>

              </archive>

              <finalName>jarname-cli</finalName>

       </configuration>

</plugin>

This plugin will create a jar file by the name of jarname-cli.jar, will a manifest file with the following entry:

Main-Class: com.package.Main

This will allow us to run from the command line:

java -jar jarname-cli %1 %2.

Jar Dependencies

Though if your jar has dependencies to other jars you will receive the following error when running the command line:

Exception in thread "main" java.lang.NoClassDefFoundError:

This is because we still need to package all the dependency jars into the same folder.

To solve this you need two more plugins.

maven-dependency-plugin (http://maven.apache.org/plugins/maven-dependency-plugin/)

The first plugin will calculate all the dependencies and copy them to a directory (cli):

<plugin>

       <artifactId>maven-dependency-plugin</artifactId>

       <executions>

              <execution>

                     <phase>install</phase>

                     <goals>

                           <goal>copy-dependencies</goal>

                     </goals>

                     <configuration>

       <outputDirectory>${project.build.directory}/../cli/</outputDirectory>

                     </configuration>

              </execution>

       </executions>

</plugin>

This plugin will copy all dependencies even though you don’t need all of them (it will include testing and jars not necessarily in use). You can start to manage you dependencies with the exclude, but this way you will constantly need to check that new jars are not added. The other option is to tell the plugin only the jars you want added:

                     <configuration>

                           <includeScope>runtime</includeScope>

                           <includeArtifactIds>commons-cli,spring-core,commons-logging,freemarker</includeArtifactIds>

                           <outputDirectory>${project.build.directory}/../cli/</outputDirectory>

                     </configuration>

This way any new dependencies will not be copied to the directory.

Of course if there are dependencies that you need for the execution of your cli, you will need to add them. You can use either the includeArtifactIds, or includeGroupIds whichever is easier.

This plugin is nice since it will get all the dependencies, but what it does not do is copy current jar to your cli directory. For this you need the ant plugin to copy the current jar

maven-antrun-plugin (http://maven.apache.org/plugins/maven-antrun-plugin/)

<plugin>

       <groupId>org.apache.maven.plugins</groupId>

       <artifactId>maven-antrun-plugin</artifactId>

       <executions>

              <execution>

                     <id>Copy Main Jar To Cli Directory</id>

                     <phase>install</phase>

                     <configuration>

                           <tasks>

                                  <copy todir="${project.build.directory}/../cli/">

                                         <fileset dir="${project.build.directory}/" includes="*.jar"

                                                excludes="*sources*.jar,*javadoc*.jar" />

                                  </copy>

                           </tasks>

                     </configuration>

                     <goals>

                           <goal>run</goal>

                     </goals>

              </execution>

       </executions>

</plugin>

The action is very simple, it is a copy action from ant, that will copy the jar from the target directory.

maven-clean-plugin (http://maven.apache.org/plugins/maven-clean-plugin/)

Since your maven project will create the dir and all the jars needed, you need to add the clean plugin to remove the dir when running clean in maven.

<plugin>

       <artifactId>maven-clean-plugin</artifactId>

       <configuration>

              <filesets>

                     <fileset>

                           <directory>${project.build.directory}/../cli</directory>

                     </fileset>

              </filesets>

       </configuration>

</plugin>

This solution is good for most cases. But sometimes you need more flexibility. Occasionally you might want not to have a directory with all jar’s but you might want to pack them all in one jar (thought licensing must be considered). Also sometimes you might want to add more files or resources to the jar, or maybe create a zip file instead of the jar file. For this you have the assembly plugin. The dependency plugin is actually a subset of the assembly plugin, so you can do all the above and more.

maven-assembly-plugin (http://maven.apache.org/plugins/maven-assembly-plugin/)

We will not go into the whole plugin since you can do a lot with it (for a full set of options see http://maven.apache.org/plugins/maven-assembly-plugin/assembly.html).

                     <plugin>

                           <artifactId>maven-assembly-plugin</artifactId>

                           <configuration>

                                  <finalName>wsf_${version}</finalName>

                                  <appendAssemblyId>false</appendAssemblyId>

                                  <descriptors>

                                         <descriptor>src/main/resources/assembly.xml</descriptor>

                                  </descriptors>

                                  <outputDirectory>${project.build.directory}/../assembly/</outputDirectory>

                           </configuration>

                           <executions>

                                  <execution>

                                         <id>make-assembly</id>

                                         <!-- this is used for inheritance merges -->

                                         <phase>package</phase>

                                         <!-- bind to the packaging phase -->

                                         <goals>

                                                <goal>single</goal>

                                         </goals>

                                  </execution>

                           </executions>

                     </plugin>

As you can see we can do both creating a manifest file with the mainclass, and adding other files. You can enable enhanced features by assigning an assembly.xml and do for example:

<assembly>

        <id>bundle</id>

        <formats>

               <format>jar</format>

        </formats>

        <includeBaseDirectory>false</includeBaseDirectory>

        <dependencySets>

               <dependencySet>

                       <outputDirectory>/</outputDirectory>

                       <useProjectArtifact>true</useProjectArtifact>

                       <unpack>true</unpack>

                       <scope>runtime</scope>

                       <includes>

                               <include>*:commons-cli:*</include>

                               <include>*:spring-core:*</include>

                               <include>*:commons-logging:*</include>

                               <include>*:freemarker:*</include>

                       </includes>

               </dependencySet>

        </dependencySets>

</assembly>

Here you can notice that we used the option <unpack>true</unpack>. This will unpack all jar’s and then repack them all in one jar. This way you can bundle you cli into one jar only.