Showing posts with label Spring Boot. Show all posts
Showing posts with label Spring Boot. Show all posts

Sunday, February 28, 2016

Spring Boot WebJar

Spring Boot WebJar


When creating an end to end application that includes client side code and server side code, we need a full application solution that can be easily built and deployed.
The client side may be written in different frameworks – angular, backbone. Spring Boot supports all of these solutions in multiple ways.

Static Resources

The strait forward way is to copy all resources from the client code (js, html, css...) to the static folder user the resources folder (see https://spring.io/blog/2013/12/19/serving-static-web-content-with-spring-boot).
The problem with this solution is that we are tightly couples between the client and server code. In order to compile the server code we need to copy the client compiled code to our resources folder.
To use this solution we need to create a Jenkins job that takes the client code, compiles it (using grunt or other tools) and then takes the folder that was created and copies it to the resources folder and then compile the server code.
This is definitely not the maven way. The solution that we would want is for the client code to create an artifact for us to add as a dependency in our pom file. The solution that has become a standard is what is called a webjar (www.webjar.com).

WebJar

A webjar is a zip file with the following folder structure:
META-INF\resources\webjars\artifactId\version\
When the client requests a resource from the server, Spring Boot looks in all its resource locations for the resource. The resource locations are: resource\static, resource\public, webjars.
So far the solution looks like it works nicely. The next issue is that webjar technology is not just a server side technology. Webjars come to solve another issue on the client side. When the client needs global functionality the solution for creating modules is to use webjars. What this means is that when the client needs a resource from the server, it needs to specify the artifactId and the version so that the server can retrieve it from the webjar.
For example:
<!DOCTYPE HTML>
<html xmlns:th="http://www.thymeleaf.org">
<head> 
    <title>Getting Started: Serving Web Content</title> 
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <script src="webjars/jquery/2.0.3/jquery.min.js"></script>
    <script type="text/javascript">
        $(document).ready(function() {
            $('p').animate({
                fontSize: '48px'
            }, "slow");
        });
    </script>
</head>
<body>
    <p th:text="'Hello, ' + ${name} + '!'" />
</body>
</html>

As you can see the client code references a version of the modules it wants to use: "webjars/jquery/2.0.3/jquery.min.js". This might be good in most cases, but the problem is that in a lot of cases our client side is a single page solution and does not necessarily need the webjar solution (with the full artifcatId and version). What then happens is that the client requests a resource but the server cannot find it since it needs the artifactid and version to find it within the webjar. We would like the client to request a js file or html file without the prefix of the webjar and for the server to find it and return it to the client.

ResourceHandlers

To do this we need to add our own bean that will supply the lookup for resources for spring.
We are actually going to solve here two other issues that arise with single page applications and Spring Boot.
In addition to the issue that we don’t want the client code to need to reference its own webjar, we would like to have the option to have all the client resources out of our jar, so that we can upgrade the client code without recompiling the server code.
Another issue with single page applications is that the client side navigation uses the url path to do this. So if the client side moves to an admin page, it will usually add to the url “/admin”. The problem with this is that if the use now refreshes his browser, the server will get a url request with “/admin” that the server does not have.
To solve this, we need to create another configuration bean that will inherit WebMvcConfigurerAdapter.  
The default implementation of spring is to search all webjars in the application (see https://spring.io/blog/2014/01/03/utilizing-webjars-in-spring-boot):
@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
    if (!registry.hasMappingForPattern("/webjars/**")) {
        registry.addResourceHandler("/webjars/**").addResourceLocations(
                "classpath:/META-INF/resources/webjars/");
    }
    if (!registry.hasMappingForPattern("/**")) {
        registry.addResourceHandler("/**").addResourceLocations(
                RESOURCE_LOCATIONS);
    }
}

We will change this and override the addResourceHandler as follows:
override def addResourceHandlers(registry: ResourceHandlerRegistry): Unit = {

  if (!registry.hasMappingForPattern("/**")) {

    registry.addResourceHandler("/**").addResourceLocations(

      classpathResourceLocations: _*)

  }

}

This will now allow us to define a string array that will tell spring were to search for resources.
Our basic resource location array will be as follows:
private var classpathResourceLocations = List(

  "classpath:/META-INF/resources/", "classpath:/resources/",

  "classpath:/static/", "classpath:/public/")

This means, for spring it will first search in the resources folder of the classpath, and the in static folder, and lastly in the public folder (standard locations for resources).
But what we want is to add is the option to have a resource folder out of our jar and also to add the location of the webjar within our application without the client having to specific the full path.
classpathResourceLocations = (externalWebsite :: classpathResourceLocations) :+ s"classpath:/META-INF/resources/webjars/my.web-jar/$infoBuildVersion/"

The order that spring boot searches for resources is by the order of the string array for the locations. So the first entry will be a location to an external folder that can be specified in the application.properties in the format of a url (file:\...).
The second entry will be my web-jar with the fullpath of the resources inside the webjar. By doing this the client does not need to send the path, since we have added all the internal resources without the prefix.

Client Side Paths

 The last issue I wish to address is client side paths. The client side is written in angular so navigation in the java script is done by urls. So the client can have a url of : www.server.com\domain\method.
The “method” is not a endpoint in the server side but an endpoint in the client side. All this works fine until the user clicks on the page refresh. What then happens is that the URL is sent to the server, where the serve does not have the endpoint and returns a page not found error.
To fix this error we need to add View Controllers path in our WebMvcConfigurerAdapter as follows:
override def addViewControllers(registry: ViewControllerRegistry): Unit = {

  registry.addViewController("/").setViewName("forward:/index.html")

  registry.addViewController("/administration").setViewName("forward:/index.html")

  registry.addViewController("/license").setViewName("forward:/index.html")

  registry.addViewController("/404").setViewName("forward:/index.html")

  registry.addViewController("/download").setViewName("forward:/index.html")

}

In this method we will add all client side URL’s with a forward to the main page of the client indext.html. This will allow the server to accept the client endpoint and the redirect it back to the client.

Error endpoint

This all works fine unless the client endpoint is /error. Since this is a standard endpoint spring automatically implements this endpoint for us. Since we do not want this we must override spring’s default implementation as follows. In you configuration bean you need to add:
@Bean(name = Array("error"))

def defaultErrorView() : View  = {

  new RedirectView("index.html")

}

This implements the error bean of spring and tells spring to not use the spring implementation but in our case to redirect the error endpoint back to the client.



Monday, November 9, 2015

Spring Boot Curator and ELK

Spring Boot Curator and ELK - JSON Message Log

Application Server

One of the things I love about spring is the simplicity of things. To create an application server that has a health and metric rest interface is very simple. For the application server I use Spring Boot (http://projects.spring.io/spring-boot/). For adding the health and metrics I use the Spring Curator (https://github.com/spring-projects/spring-boot/tree/master/spring-boot-actuator).
Now that I have my application up and running, I wan’t to monitor the application with a nice GUI with history. To do this I use ELK – Elastic Search, LogStash and Kibana. Kibana is the GUI interface. Elastic Search is the Database, and LogStash enters the data.
To implement this I could have added a mechanism to send my health and metrics directly to Elastic Search but this would be a solution specific implementation. I also do not want errors or checking of the Elastic Search Status in my application to run. This is where LogStash comes in. All I need to implement in my application is writing of the log files. Spring Boot comes with slf4j built in (http://docs.spring.io/spring-boot/docs/current/reference/html/howto-logging.html). Now all that I need is to send my health and metrics to different log files for the LogStash to use.

Log Files

First step to create the log files is to use logback as our log implementation. For that you need to add to your resources folder the logback.xml file (http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-logback-extensions). Here you can define the rolling of the log file and the format.
To get the log files into Elastic Search you have two options. The first way is to configure the logstash to use parse the log files of spring boot using grok (https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html).
Since we know that we will be using LogStash it is better to have the log files formatted from the begging in json format for LogStash. To do this we will use the LogStash encoder for logback. For example:
<appender name="LogStashHealthFile" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <
encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
        <
providers>
            <
timestamp/>
            <
version/>
            <
threadName/>
            <
loggerName/>
            <
logLevel/>
            <
logLevelValue/>
            <
context/>
            <
arguments/>
            <
message/>
        </
providers>
    </
encoder>
    <
File>../log/log-stash-health.log</File>
    <
rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
       
<!-- rollover daily -->
       
<fileNamePattern>../log/archive/log-stash-health.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
        <
maxHistory>14</maxHistory>
        <
timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
           
<!-- or whenever the file size reaches 5MB -->
           
<maxFileSize>5MB</maxFileSize>
        </
timeBasedFileNamingAndTriggeringPolicy>
    </
rollingPolicy>
</
appender>

To use the encoder you need to add the following maven dependency:
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>4.5.1</version>
</dependency>

This will create our log file as a json format for LogStash.

The Message Issue

The issue that I had a hard time with, and is the main reason for this blog is the json format of the message. The logback encoder creates a json out of the log entry, where the message is a field in the json. This field is recorded as a text field in the json object and not as part of the full json object.

For example:
{
            "@timestamp": "2015-11-09T10:11:19.026+02:00",
            "@version": 1,
            "logger_name": "com.clearforest.importer.log.health",
            "level": "INFO",
            "message": "{\"status\":\"UP\",\"queue\":{\"status\":\"UP\",\"Queue Name\":\"cmwell.files.ucpa\",\"Data Directory\":\"..\\\\data\\\\ActiveMQ\"},\"CMWell\":{\"status\":\"UP\",\"host\":\"vgilad:9000\"},\"nrg\":{\"status\":\"UP\",\"lastDataReceived\":\"09.11.2015 10:11:18\",\"lastWriteDate\":\"unknown\",\"cachedMessages\":1},\"diskSpace\":{\"status\":\"UP\",\"free\":24298827776,\"threshold\":10485760}}"
}

For Kibana to work we need the message of the log to be part of the full json object. To fix this when using the log in our code we need to add the following import:

import net.logstash.logback.argument.StructuredArguments._

This adds the option to pass arguments to the log file interface. The arguments are then added as a json object and not as a string field. In the code you then use the log as follows:

val res = RequestFactory.getRestContent(s"http://localhost:$serverPort/metrics")
meticLogger.info("",raw("metric",res))

and in the log file we will get

{
            "@timestamp": "2015-11-09T08:59:57.480+02:00",
            "@version": 1,
            "level": "INFO",
            "level_value": 20000,
            "health": {
                        "status": "DOWN",
                        "queue": {
                                    "status": "UP",
                                    "Queue Name": "files.abc",
                                    "Data Directory": "..\\data\\ActiveMQ"
                        },
                        "MyApp": {
                                    "status": "DOWN",
                                    "error": "Connection refused: connect",
                                    "host": "abc:9000"
                        },
                        "diskSpace": {
                                    "status": "UP",
                                    "free": 24000569344,
                                    "threshold": 10485760
                        }
            }
}