Wednesday, October 23, 2013

Java Decompiling and Eclipse


Java Decompiling and Eclipse


One of the more frustrating issues are when you debug code in eclipse and want to debug a third party jar but do not have the source.


If the jar is open source and you are using maven you can try to use
mvn install –DdownloadSources=true.
This does not always work, since not all jar’s supply the source files.
In addition there are third part jar’s or sun jar’s that do not have source files. For this we have decompile utilities. The most famous one is jad (http://en.wikipedia.org/wiki/JAD_(JAva_Decompiler)).
So just go to your eclipse market and write jad. Once you have installed the plugin, next time you need to look at the code you will see:





Monday, October 14, 2013

Android – Israel Timezone


Android – Israel Timezone

For all those Israelis that have an android but have the wrong time/time zone since Israel keeps on changing the date of day light savings, this procedure is for you.

Download updated tzdata 


Android adb

Make sure you have the android adb (http://developer.android.com/sdk/index.html)

Connection to your device

To test your connection type in the dos prompt: adb devices.
You should get a list like

Copy the tzdata file to the same folder as the adb.

Run the following commands

adb root
adb remount
adb push tzdata /system/usr/share/zoneinfo/
adb shell chmod 644 /system/usr/share/zoneinfo/tzdata
adb reboot

You should now have the correct time and time zone.



Monday, October 7, 2013

Create a Java CLI application with Maven

Create a Java CLI application with Maven

Creating a CLI application is part of our daily routine. In java it is strait forward; all you need to do is to add a class with the main function:
public static void main(String[] args) {
}
This is nice for debugging from eclipse, but if you need to release this jar with the cli working, you will face the following issues:
1.       You need to add the main class to you manifest file and compile it in the jar.
2.       You need to create a folder with all the jar’s dependencies needed to run this cli.

Main Class

maven-jar-plugin

The first issue can be solved by using the maven-jar-plugin:
<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-jar-plugin</artifactId>
       <configuration>
              <archive>
                     <manifest>
                           <addClasspath>true</addClasspath>
                           <mainClass>com.package.Main</mainClass>
                     </manifest>
              </archive>
              <finalName>jarname-cli</finalName>
       </configuration>
</plugin>

This plugin will create a jar file by the name of jarname-cli.jar, will a manifest file with the following entry:
Main-Class: com.package.Main
This will allow us to run from the command line:
java -jar jarname-cli %1 %2.

Jar Dependencies

Though if your jar has dependencies to other jars you will receive the following error when running the command line:
Exception in thread "main" java.lang.NoClassDefFoundError:
This is because we still need to package all the dependency jars into the same folder.
To solve this you need two more plugins.

maven-dependency-plugin (http://maven.apache.org/plugins/maven-dependency-plugin/)

The first plugin will calculate all the dependencies and copy them to a directory (cli):
<plugin>
       <artifactId>maven-dependency-plugin</artifactId>
       <executions>
              <execution>
                     <phase>install</phase>
                     <goals>
                           <goal>copy-dependencies</goal>
                     </goals>
                     <configuration>
       <outputDirectory>${project.build.directory}/../cli/</outputDirectory>
                     </configuration>
              </execution>
       </executions>
</plugin>

This plugin will copy all dependencies even though you don’t need all of them (it will include testing and jars not necessarily in use). You can start to manage you dependencies with the exclude, but this way you will constantly need to check that new jars are not added. The other option is to tell the plugin only the jars you want added:
                     <configuration>
                           <includeScope>runtime</includeScope>
                           <includeArtifactIds>commons-cli,spring-core,commons-logging,freemarker</includeArtifactIds>
                           <outputDirectory>${project.build.directory}/../cli/</outputDirectory>
                     </configuration>

This way any new dependencies will not be copied to the directory.
Of course if there are dependencies that you need for the execution of your cli, you will need to add them. You can use either the includeArtifactIds, or includeGroupIds whichever is easier.
This plugin is nice since it will get all the dependencies, but what it does not do is copy current jar to your cli directory. For this you need the ant plugin to copy the current jar

maven-antrun-plugin (http://maven.apache.org/plugins/maven-antrun-plugin/)

<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-antrun-plugin</artifactId>
       <executions>
              <execution>
                     <id>Copy Main Jar To Cli Directory</id>
                     <phase>install</phase>
                     <configuration>
                           <tasks>
                                  <copy todir="${project.build.directory}/../cli/">
                                         <fileset dir="${project.build.directory}/" includes="*.jar"
                                                excludes="*sources*.jar,*javadoc*.jar" />
                                  </copy>
                           </tasks>
                     </configuration>
                     <goals>
                           <goal>run</goal>
                     </goals>
              </execution>
       </executions>
</plugin>

The action is very simple, it is a copy action from ant, that will copy the jar from the target directory.

maven-clean-plugin (http://maven.apache.org/plugins/maven-clean-plugin/)

Since your maven project will create the dir and all the jars needed, you need to add the clean plugin to remove the dir when running clean in maven.
<plugin>
       <artifactId>maven-clean-plugin</artifactId>
       <configuration>
              <filesets>
                     <fileset>
                           <directory>${project.build.directory}/../cli</directory>
                     </fileset>
              </filesets>
       </configuration>
</plugin>

This solution is good for most cases. But sometimes you need more flexibility. Occasionally you might want not to have a directory with all jar’s but you might want to pack them all in one jar (thought licensing must be considered). Also sometimes you might want to add more files or resources to the jar, or maybe create a zip file instead of the jar file. For this you have the assembly plugin. The dependency plugin is actually a subset of the assembly plugin, so you can do all the above and more.

maven-assembly-plugin (http://maven.apache.org/plugins/maven-assembly-plugin/)

We will not go into the whole plugin since you can do a lot with it (for a full set of options see http://maven.apache.org/plugins/maven-assembly-plugin/assembly.html).
                     <plugin>
                           <artifactId>maven-assembly-plugin</artifactId>
                           <configuration>
                                  <finalName>wsf_${version}</finalName>
                                  <appendAssemblyId>false</appendAssemblyId>
                                  <descriptors>
                                         <descriptor>src/main/resources/assembly.xml</descriptor>
                                  </descriptors>
                                  <outputDirectory>${project.build.directory}/../assembly/</outputDirectory>
                           </configuration>
                           <executions>
                                  <execution>
                                         <id>make-assembly</id>
                                         <!-- this is used for inheritance merges -->
                                         <phase>package</phase>
                                         <!-- bind to the packaging phase -->
                                         <goals>
                                                <goal>single</goal>
                                         </goals>
                                  </execution>
                           </executions>
                     </plugin>

As you can see we can do both creating a manifest file with the mainclass, and adding other files. You can enable enhanced features by assigning an assembly.xml and do for example:

<assembly>
        <id>bundle</id>
        <formats>
               <format>jar</format>
        </formats>
        <includeBaseDirectory>false</includeBaseDirectory>
        <dependencySets>
               <dependencySet>
                       <outputDirectory>/</outputDirectory>
                       <useProjectArtifact>true</useProjectArtifact>
                       <unpack>true</unpack>
                       <scope>runtime</scope>
                       <includes>
                               <include>*:commons-cli:*</include>
                               <include>*:spring-core:*</include>
                               <include>*:commons-logging:*</include>
                               <include>*:freemarker:*</include>
                       </includes>
               </dependencySet>
        </dependencySets>
</assembly>


Here you can notice that we used the option <unpack>true</unpack>. This will unpack all jar’s and then repack them all in one jar. This way you can bundle you cli into one jar only.

Sunday, August 18, 2013

Multithreading queries with hibernate & spring

There are times that we have a process that needs to run multiple requests from the db, and then with the result of all of them, to combine the results in a specific way. Of course you can run each query sequentially, but to boost performance you want to run them in parallel.
At first this seems simple since you just need to run each query in a separate thread using a callable, and then wait on the future results to get them (for a good article on thread and concurrency see: http://www.vogella.com/articles/JavaConcurrency/article.html).  
As we will see this approach has a lot of problems, and does not work simply with hibernate.

Session and Threads

The problem is what information do you pass to the thread, and what is the result from the thread. The first naïve thought was to create the query object in the main thread (so the thread can be generic and not deal with the query and parameters), and then pass to each new thread the query. Then all the thread needs to do is call query.uniqueResult() or query.list() and return the result.
The problem with this approach is that hibernate session is not thread safe (http://www.javalobby.org/java/forums/t104442.html, http://pveentjer.wordpress.com/2007/02/19/sharing-hibernate-entities-between-threads/), so you cannot share the session between threads. Hibernate to simplify the implementation assumes that all queries run on a thread are using the same session. This means that you are not allowed to pass the session between threads.
So the next option is to pass the dao to the thread and then call a specific function on the dao. This makes more sense from a design point of view since this is the default design of a service layer application. In a web app, each request has it’s own thread, and then the calls to the dao are all stateless and will have their own session per thread that is created by hibernate.
The downside to this solution is that we do not have a generic way to run queries asynchronous. A possible solution is to pass a reference to the dao, and an object that describes the function and parameters, and the within the thread to use reflection to find the function and to call it with the parameters sent.

Transactions

The next part that you need to think about is transactions. By default each thread will have its own transaction (needs to be defined by annotations or xml). It is not simple to have transactions cross threads and is not considered to be a good design practice. But if for whatever reason you must have it, global JTA is the way to go.

Results

The next part that needs to be taken into consideration is the result from the thread. The problem is that if we return the entity as is, and it has lazy objects, accessing these objects on the main thread will cause a Hibernate Lazy Load Exception. Since hibernate tries to load the data from the database when there is a lazy proxy, and the call to the object is from a different thread, hibernate cannot load the information and a lazy exception is thrown (for more information see http://blog.xebia.com/2009/02/07/hibernate-and-multi-threading/).
There are multiple ways to solve this issue. One is to use dto’s and convert the entity to a pojo and then it will not be associated with hibernate session. Another solution is to use an open source lib by tikal (http://tikalk.com/chop-hibernate-lazy-associations) that will chop any entities and replace the proxies by a null according to an aop pointcut.

Summary:

What seemed like a simple idea in the beginning to run multiple queries in multiple threads, although a need sometimes, the implementation must be carefully done to take into account all the issues raised above.




Monday, July 29, 2013

Nested Hibernate Filters

Nested Hibernate Filters

Hibernate added a feature that is not part of the JPA, and this is filters.
When you want to load information from the database but want to filter the data, you can use HQL, @Where or filters.
HQL is good if your filter is limited to the query that you need. If you always want to filter the class you can add to the class the @Where. In the cases that you sometimes want to filter and sometimes not, or you need to parameterize the filtering then you use the @Filters.
It feels like all the features of filtering were an afterthought by hibernate. The reason is that hibernate does not use the field names for the filtering or @Where but uses the database field name. The reason they do this, is that it is easy to add to the sql statement the where statement, and they do not have to parse the class objects.
Another oversight on the filter issue is nested filters. Hibernate does not care how many times you enable the filter, one disable will disable it in any case. Also if you change the parameter of a filter that is already open hibernate has no problem and will update the fiter.

Take the following case:

Enable filter, date: 1/1/2000
Load data
Enable filter, date: 1/1/1999
Load data
                Disable filter
Load data
Disable filter

In this scenario we have two problems. Once the disable filter is called there is no open filter so the third load data will run with no filter at all. Even if the filter was not closed we would still have the problem that the date of the filter is incorrect, since it is set for 1999 and not 2000.
To solve this issue we need to implement a stack of filters per filter name and store them on the local thread (so that our genericdao is thread safe).

FilterInfo Class

The first class will hold the parameters of the filter and what is the nested count (how many times has the enable been called for the same filter with the same parameters):
public class FilterInfo {
       private int nestedCount;
       private Map<String, Object> parameters = new HashMap<String, Object>();
      
             
       public FilterInfo(Map<String, Object> parameters) {
              super();
              this.parameters = parameters;
       }

      
       public boolean areParametersTheSame(Map<String, Object> currentParameters) {
              if (parameters.size() > 0) {
                     // check old parameters, see if changed
                     for (String paramName : parameters.keySet()) {
                           for (String currentparamName : currentParameters.keySet()) {
                                  if (paramName.equals(currentparamName)) {
                                         Object newValue = parameters.get(paramName);
                                         Object oldValue = currentParameters.get(paramName);
                                         if (!newValue.equals(oldValue)) {
                                                return false;
                                         }
                                  }
                           }
                     }
              }
              return true;
       }

       public int decNestedCount() {
              nestedCount--;
              return nestedCount;
       }

       public int incNestedCount() {
              nestedCount++;
              return nestedCount;
       }

       public Map<String, Object> getParameters() {
              return parameters;
       }

       public int getNestedCount() {
              return nestedCount;
       }

}

FilterStack Class

This class will store a stack of the parameters per filter so that we can re enable the filter with the correct parameters once a nested filter is disabled.
public class FilterStack {
       private String filterName;
       private Stack<FilterInfo> filterStack = new Stack<FilterInfo>();

       public FilterStack(String filterName) {
              this.filterName = filterName;
       }

       public FilterInfo setParameters(Map<String, Object> parameters) {
              boolean areParametersTheSame = false;
              FilterInfo filterInfo = null;
              if (filterStack.size() > 0) {
                     filterInfo = filterStack.peek();
                     areParametersTheSame = filterInfo.areParametersTheSame(parameters);
              }
              if (!areParametersTheSame) {
                     // add a new stack
                     filterInfo = new FilterInfo(parameters);
                     filterStack.push(filterInfo);
              }
              filterInfo.incNestedCount();
              return filterInfo;
       }

       public FilterInfo disableFilter() {
              if (filterStack.size()==0) {
                     throw new InfraException("you are closing a filter that is not open: " + filterName);
              }
              FilterInfo peek = filterStack.peek();
              if (peek != null) {
                     int decNestedCount = peek.decNestedCount();
                     if (decNestedCount <= 0) {
                           filterStack.pop();
                           return filterStack.size()>0 ?  filterStack.peek() : null;
                     }
              }
              return peek;
       }

}

NestedFilterInfo Class

The last class will be stored on the local thread, and will hold all the information for all the open filters.
public class NestedFilterInfo {
       private Map<String, FilterStack> filtersByName = new HashMap<String, FilterStack>();

       public FilterInfo enableFilter(String filterName, Map<String, Object> parameters) {
              // get current filter information
              FilterStack filterStack = filtersByName.get(filterName);
              if (filterStack == null) {
                     filterStack = new FilterStack(filterName);
                     filtersByName.put(filterName, filterStack);
              }

              // check if filter parameters have changed.
              return filterStack.setParameters(parameters);
       }

       public FilterInfo disableFilter(String filterName) {
              FilterStack filterStack = filtersByName.get(filterName);
              return filterStack.disableFilter();
       }
      
       public Set<String> getFilterNames() {
              return filtersByName.keySet();
       }

       public void clear() {
              filtersByName.clear();           
       }
}


GenericDao class

In the generic dao class that actually opens and closes the filter will use these classes to manage all open filters. The function to enable a filter will call:
public void enableFilter(String filterName, Map<String, Object> mapParameters) {
NestedFilterInfo filterStack = getFilterStack();
       FilterInfo enableFilter = filterStack.enableFilter(filterName, mapParameters);
       sessionEnableFilter(filterName, enableFilter.getParameters());
}

private void sessionEnableFilter(String filterName, Map<String, Object> mapParameters) {
       Filter filter = session().enableFilter(filterName);
       Object value;
       if (mapParameters != null) {
              for (Map.Entry<String, Object> entryParameters : mapParameters.entrySet()) {
                     value = entryParameters.getValue();
                     filter.setParameter(entryParameters.getKey(), value);
              }
       }
}

The enabling filter is the easy part. For every enable filter if no parameters have changed we will increment the counter on the FilterInfo class. If the parameters have changed we will create a new FIlterInfo and add it to the stack in the FilterStack.
The magic is done in the disable filter:
public void disableFilter(String filterName) {
       NestedFilterInfo filterStack = getFilterStack();
       FilterInfo filterInfo = filterStack.disableFilter(filterName);
       if (filterInfo == null || filterInfo.getNestedCount() <= 0) {
              session().disableFilter(filterName);
       } else {
              sessionEnableFilter(filterName, filterInfo.getParameters());
       }
}

Here we call the disable and get a FilterInfo class. If the nested count is down to zero then we have finished the stack and called the disable the same amount as the enable so we will close the filter on the session. If not then either the stack count went down by one, or we got the previous filter that was open. In either case we update the session filter with the parameters, and this way we have reverted the filter to the previous one.
This solution support both nested filters with the same parameters (only after the last disable with the filter really be closed) and nested filters with different parameters (with the new parameters the session will be updated, and on the disable filter the previous parameters will be returned to the session).