Maven Profiles - per module

Maven Profiles

Activation per module

Maven profiles are very powerful (http://maven.apache.org/guides/introduction/introduction-to-profiles.html).

One of the hardest parts of maven, in my opinion, for someone coming from a programing background, is that the flow of maven is not a method call.

In a lot of cases we would like to create a batch process (group of plugins) that do something, and then invoke them in different modules. So what you would expect to do is to create a profile with all the plugins that you want, and then to active it on a module level.

For instance a simple example is to download all dependencies of the project to the target directory. In this case we will define the profile in the parent pom as follows:

<profile>

  <id>trace</id>

  <build>

    <plugins>

          <plugin>

               <artifactId>maven-dependency-plugin</artifactId>

               <groupId>org.apache.maven.plugins</groupId>

               <executions>

                 <execution>

                       <phase>package</phase>

                       <goals>

                         <goal>copy-dependencies</goal>

                       </goals>

                       <configuration>

                         <outputDirectory>${project.build.directory}/lib</outputDirectory>

                       </configuration>

                 </execution>

               </executions>

          </plugin>

         </plugins>

  </build>

</profile>

Now the problem is that when I invoke this profile it will be active on all modules. In this case it might be nice, but what if I have multiple modules and want to invoke the profile from only some of the modules.

This is where profile activation comes in. There is an option to activate a profile based on a system parameter like:

  <profile>

    <activation>

      <property>

        <name>debug</name>

      </property>

    </activation>

  </profile>

This will activate the profile if the system property of “debug” exists. Of course you can also add the option to check the value of the parameter:

  <profile>

    <activation>

      <property>

        <name>environment</name>

        <value>test</value>

      </property>

    </activation>

    ...

  </profile>

What is missing is that this feature only works with system parameters and not maven parameters. So you cannot set a parameter per module with a different value, and have maven evaluate it.

Another issue that you must always remember: when running maven, maven will compile the pom and create an effective pom (you can run mvn “effective-pom” to view it) and then run it. This means that you cannot do any manipulation during runtime that will affect the run of maven.

Solution – files

The solution that I found to run profiles per module is the use of file activation. Let’s say that I have some test modules. Sometimes I want to run a tomcat before running the tests and others not (just junit). In the standard way I will need to create a “tomcat” profile in each pom, and copy all the code that configures and deploys the pom.

I want to write the profile once in the parent pom, and then activate it only in those modules that need it. To do this I create the profile in the parent pom, and add the following activation:

<profile>

        <id>tomcat</id>

        <activation>

               <file>

                       <exists>profile.tomcat.active</exists>

               </file>

        </activation>                  

        ...

</profile>

Now for each module that I want to activate this profile, I need to have the file profile.tomcat.active in the pom directory. This will activate the profile, and the file needs to be there before you run the maven command, so you cannot copy it during the maven run.

This can also be used in a Jenkins scenario, where in the Jenkins job before running the maven, you add the files or delete them and the run the maven via Jenkins.

Hebrew Aleph Bet Pebble Watch

Now with the new version 2 of pebble i have updated my hebrew aleph bet watch.
You can get the version at:
https://dev-portal.getpebble.com/applications/52f945b835591d6295000441

Java Singleton

There are many implementations for a singleton in java.
Here is a full article of different ways and the advantages or disadvantages of each.

http://www.javaworld.com/article/2073352/core-java/simply-singleton.html

What surprised me is that the best solution is not in that blog.
The best solution seems to be using enum:

http://javarevisited.blogspot.co.il/2012/07/why-enum-singleton-are-better-in-java.html

Micro Services

I was at a session for micro services. There were many sessions covering different aspects of micro services (for seeing of the sessions, have a look at: http://www.meetup.com/full-stack-developer-il/events/155911442). I would like to summarize some of the issues raised and to give my two cents on the topic.

There are many advantages and disadvantages with micro services and there is no silver bullet for all scenarios.  Micro services are a reaction to the big monolithic applications. The biggest disadvantages of monolithic application are the following:

1.       Library bloat: with time as the code base grows, the libraries gets very big and conflicts between shared libraries get harder to handle.

2.       Memory/CPU: a single application uses a lot of memory and cpu and there is no simple way to expand horizontally (except to duplicate the application across machine, and maybe disable portions by configuration files).

3.       Initialization time: the initialization time of the application goes up with each added feature to the application.

4.       Deploy Time: the time to deploy goes up with the size of the application.

5.       Testing: testing become more difficult with big application since it is hard to test only one area without building the whole application memory, and sometimes databases.

Solution overview

Of course for each issue there are other solutions than micro services, but in this blog we will focus on the micro services solution. What is the solution of micro services – very simple. We distribute the main application across multiple processes and machines (hence the name micro service). The client application communicates with the main application (thought there are expansions to micro services were the client will also communicate with multiple servers). The main application will notify to all the micro services of the request and will then return mandatory results to the client. The base framework is an async framework so the main application is not busy with all the logic. The main app only handles the information that is mandatory to return to the client and all other functionality is delegated to services.

Architecture view

The solution for most of these problems, and micro servers as a solution, are on two levels. One is the architectural solution, and the other is on a technical level.

As we mentioned above the communication between services is asynchronous. The standard way to implement this is with a messaging framework (for example: JMS, rabbitmq, ActiveMQ , OpenJMS, Kafka  and more). The design here is that the main application does not even need to know what the services are. Each new functionality that we want to add we can add as a service, and plug it in to the server by registering for events. This is very convenient for application that want to add more processing for information coming in but is not part of the main flow of the application. For instance a purchase on a ecommerce app will enter the request and display to the user the request. Then multiple services will process the information, including checking the inventory and updating if new quantities need to be added. Information about the person purchasing can be added and a lookup for credit history can be done. A full flow of data gathering and processing can be done via different services, and can be added with each. In this example the database is the center of the project, but in other scenarios the data can be pin ponged between services.

Of course if a service needs information from another service in a synchronous way, each service can also have an external API of rest or some other protocol.  

See https://speakerdeck.com/kensodev/scaling-extending-and-expanding-your-apps-through-messaging for a full presentation.

Physical View

The obvious gain from this architecture is that each service can run on any machine in the cloud. As to the issues raised above each service is a separate application so the library is local to each service and will not be big. Each service has its own resources so we will not be blocked by cpu consumption or memory. Deployment is easier since it is on the service level and not the whole application. Also testing is easier since the testing is per service and not on the whole application.

One of the biggest advantage that are pushed with the micro services is the option to totally rewrite a service if the requirements have changed since each service is fairly small, and if a functionality need to be remove the service can just be discarded.

Another big advantage is the language barrier. With micro services you can write each service in any language and with any team. Since all interfaces to the app are via message queue or rest, and language can be used.

Draw backs

But as we know there is no free lunch. What are the major issues that need to be addressed with micro services?

·         Integration: Since there is no app, it is hard to determine ahead of time how the application as a whole will run. The information passed via the message queue might need to change over time, and we do not know which services are using which messages, and which ones need to be updated.

·         Latency: Once logic is distributed across multiple machines there is the latency issue. So only application that the latency is not a big issue can use this architecture.

·         Configuration: How are the configurations of each service handled? Do we have a central repository for configurations? There can be hundreds of machines that need to be configured, and each service with a different format (see https://speakerdeck.com/hagzag/docking-your-services-with-docker for some solutions).

·         Transactions: Once we are in an asynchronous framework that crosses machines we lose the option to have all functionality in one transaction. All “commit/rollback” needs to be done on the application level and not database level.

·         Debugging: How do we debug the application as a whole when each service is on a different machine, in a different language using different logs types.

·         Monitoring: How do we monitor all the applications to make sure they are giving the performance we expect.

For the last two issues see http://zohararad.github.io/presentations/micro-services-monitoring/index.html, solutions for them can be flume, logstash, kibana , sensu.

·         Retransmissions: How do we handle issues where the service fails, we need to make sure that the messaging framework supports transaction of deleting the message only after the service finished with it.

Other solutions?

Are micro services the only solution? As we mentioned above there is no silver bullet. The architecture behind the micro service is an old one of messaging.

OSGI

One architecture that has been around, that also address some of the issues raised above is OSGI (http://en.wikipedia.org/wiki/OSGi).  OSGI focus is to solve library and deployment issues. The basic idea is to generate bundles, where each bundle is a single unit in itself.

The concept was good, but there was never a good implementation of it, and the industry never took it up, since it usually caused more trouble than was worth.

Messaging

The idea of messaging is not new. Using message for transferring data between objects was part of the smalltalk language already in the 80’s (http://en.wikipedia.org/wiki/Smalltalk).

Windows OS used the idea of messaging for handling events from the user in a GUI application (https://developer.mozilla.org/en-US/docs/Web/API/Window.postMessage).

Android OS uses Intents to transfer information between windows and applications. Intents are actually simple messages that are broadcasted and receivers can listen to these intents (http://developer.android.com/reference/android/content/Intent.html).

The basic idea behind messaging is that the data is passed between participants is immutable. This way even with multi-threading there is no memory sharing between with processes that need critical sections.

Spring Integration/ Camel

If you do not want to split your application to multiple services and computer, but you do want to gain the architecture of the micro services, you can use enterprise integration patterns. The basic pattern in every enterprise application is that data needs to pass between processes and needs to be transformed from one presentation to another. In the past we had xslt for xmls, but in object framework we still need to transfer and translate object.

The basic architecture of these frameworks (http://docs.spring.io/spring-integration/reference/html/overview.html, http://camel.apache.org/enterprise-integration-patterns.html) allows each feature to be written in isolation and without any knowledge of the other. The flow between the data is defined in a xml configuration file. The actual process of the messages is the same as queuing where via the xml file you wire the flow of the data, where you can not only pass the data but also transform it for the needs of each feature.

Advantages over micro services

·         You have the flexibility and modularity of micro services without all the maintenance of multiple services and machines.

·         Flow of application can be viewed in xml (and visualized in ide).

·         Spring Integration/Camel support transaction barriers across beans (features).

·         All logging and error handling are handles in one place.

·         Testing can be done on both the feature level and the flow level.

Disadvantages

·         In the case of java, you can only write the code in languages that run on the JVM.

·         There is still an issue of library bloat.

·         Memory / CPU are still issues since you have only one application.

Summary:

Using modular frameworks in a monolithic application give you the architecture advantage of micro services but without the flexibility of horizontal expansion via multiple machines and processes.

See features of spring 4 that help us with micro services: https://spring.io/blog/2013/12/12/announcing-spring-framework-4-0-ga-release.

Akka

Another framework that is between spring integration and micro services is akka (http://akka.io).

In a very simplistic overview, akka is an actor model system (http://en.wikipedia.org/wiki/Actor_model). Akka is written in Scala that is a functional language that brings a functional programing view. Again in a simplistic view Akka is similar to messaging frameworks in that all information that is passed between objects is done by messaging. There are a lot of similarities between akka and spring integration and there are big differences. Since they are different systems they can even be used together.

One big feature in akka that does not exist in spring integration, from the micro service view, is the distribution model. One of the goals of Akka is “location transparency” (http://doc.akka.io/docs/akka/2.2.3/intro/what-is-akka.html). This is one of the big draw backs of spring integration in the micro service perspective. Akka out of the box supports the option to install the system on multiple machines without any code change. So by using akka we can still write in a micro service architecture and gain a lot of the build in features of akka that include transactions (http://doc.akka.io/docs/akka/2.0.1/scala/transactors.html) and the option for horizontal expansion.

The main disadvantage of Akka is that you still have the library bloating as your application grows, and the application is still written as a monolithic application and not a diverse application.

For more information on functional programing and micro services see:

http://www.slideshare.net/michaelneale/microservices-and-functional-programming