Secure images with angular.

Ever needed to deliver images securely? I had a usecase for delivering map tiles only if the user was authenticated - but I was using stateless authentication with Java Web Tokens (JWT). In order to use web tokens I need to place an Authentication header on each web request - easy enough. But that isnt possible to do when using a html img tag - it only allows for a url souce.

So you have two options - extend your JWT token reading service to allow it to read tokens from the url params object too (ie <img src='someurl?token=foo'> ) or instead of using the standard src attribute - use javascript to securely download the image data into a blob, and use the html5 blob data attribute for img tags.

This angular directive will do such a thing. I based my solution on this source code- except noting adding that I use an angular service "Authentication" that has a promise that will resolve only if there is valid non-expired authentication (and will refresh the token if expired).

So I only needed to modify the directive's core method to add my extra waitForAuthenticated() promise.

attrs.$observe('httpSrc', function (url) {
                    revokeObjectURL();

                    if (url && url.indexOf('data:') === 0) {
                        $scope.objectURL = url;
                    } else if (url) {
                        Authentication.waitForAuthenticated().then(function(ok){
                        $http.get(url, {
                                responseType: 'arraybuffer',
                                headers: {
                                    'accept': 'image/webp,image/*,*/*;q=0.8'
                                }
                            })
                            .then(function (response) {
                                var blob = new Blob(
                                    [response.data],
                                    {type: response.headers('Content-Type')}
                                );
                                $scope.objectURL = URL.createObjectURL(blob);
                            });
                        });
                    }
                });

The blue text is the core of the directive, pipeling the securely delivered image data into an HTML objectURL. You can attach headers easily using an injected interceptor to all $http requests.. This wont work on older browsers not supporting the objectURL standard.

A working combination of grails 3, geoDB & hibernate spatial.

Ever since the move to grails 3 - dealing with plugins (for me) has become a pain. Almost all the documentation out there is for the now legacy 2.X branches. Makes me consider just using basic servlets like jetty or tomcat embedded. Anyway!

Hibernate spatial is helpful ORM when working with the Java Topology Suite - and since hibernate 5 is now worked into the core project.

To setup your grails project to use hibernate spatial - and for grails to successfully select the type of the JTS Geometry derviatives when defined in a domain class - you'll need to make sure all your hibernate versions stack up right to avoid no class data found or no class def found, or java.lang.NoSuchMethodError OR cannot resolve name errrors - all symptoms of mismatch java versions and/or class/interface definitions.

in your build gradle file:
(I happen to be using the latest grails 3.2.6)

* Make sure you are using the hibernate5 grails plugin - atleast version 6.0.0
* Make sure the version of hibernate-core & hibernate-ehcache & hibernate-spatial matches - atleast version 5.1.0 - but crucially - not greater than 5.2.0 - 5.2.0+ is currently incompatible with grails.
* Make sure you are using geoDB version 0.7 (for local testing) - but not version 0.8 (incompatible from my testing)

* In your bootstrap, make sure to upgrade the h2 database to contain goeDB features:

 try {
            def sql = new Sql(dataSource)
            sql.executeUpdate('CREATE ALIAS InitGeoDB for "geodb.GeoDB.InitGeoDB"')
            sql.executeUpdate('CALL InitGeoDB()')
        } catch (java.sql.SQLException e) {
            log.debug('', e)
        }

And thats it. After changing the plugins, make sure to do a gradle clean, and you should be able to correctly map classes like:

import com.vividsolutions.jts.geom.Geometry
import com.vividsolutions.jts.geom.GeometryFactory

class Feature {

    UUID id

    double lat
    double lon

    Geometry geometry

    def beforeInsert( ) {
        if ( lat!=null && lon!=null && geometry == null) {
            geometry = (new GeometryFactory()).createPoint( new Coordinate(lon, lat ))
        }
    }

    static mapping = {
        id(generator: "uuid2", type: "uuid-binary") // H2 & Postgres
    }

}

Grails 3 Push Notifications: Spring Web Sockets, AngularJS, JWT.

Apparantly its not very common to want to combine these sets of technologies. But I did, and I wanted to share some of the tips I found, and hopefully combine the knowledge into one place. There are a bunch of different sources that will help you solve several problems you may encounter.

The use-case

I wanted to be able to have my API server push updates to my web-based angular app as interesting things took place. The most common use is that someone else posted new messages in a chat thread - but it can also happen when someone else updates a shared resource or other collaboration. Or perhaps even to populate new items into a news feed without needing a page refresh.

To "push" data to my angular app, I knew I was going to need sockets. Grails 3 has updated spring support that allows for the use of spring websockets - which use the STOMP protocol & SockJS. SockJS is a library that abstracts away a lot of the lower level websocket management in browsers and allows fallbacks to long polling if needed. STOMP makes it easy to create per-user topics and other utility methods.

The websocket pattern integrates really nicely with grails reactor integration. As controllers recieve updates from other clients, events can be raised to push notifications to interested listeners.

Starting Out

Understanding the basics of web sockets are outside the scope of this post - but spring has made integrating web sockets over STOMP easy in recent versions.

There are a couple good sources for starting out. There is an up-to-date grails 3 plugin that allows you to use socket topics easily from within existing controllers.

Defining a controller that takes incoming chat requests over socket is easy.

class ChatController {

    def springSecurityService    def messagingService
    def index = {}

    @ControllerMethod    

    @MessageMapping("/incchat")
    @PreAuthorize("hasRole('ROLE_USER')")
    @SendToUser("/queue/incchat")
    /**     * Take a JSON input and parse from object

     * id: threadid, text: messageText     */    

    String doChat(String jsonIn, Principal principal) {

     ...

Defining a listener to updates from other requests as a service is easy.

@Consumer

class WebChatService {

    SimpMessagingTemplate brokerMessagingTemplate
    @Selector('message.received.chat')
    void sendChatToUser(WebChatTarget target) {
        String data = [message: target.message, thread: target.messageThread] as JSON
        brokerMessagingTemplate.convertAndSendToUser(target.username, "/queue/chat", data)
    }
}

SOCKJS allows http websocket upgrades - so make sure to use a http:// base url if your sockets are using sockjs. Use ws:// otherwise. Make sure to allow origins that would allow your angular app to connect if its running on a different host.

@Override

public void registerStompEndpoints(StompEndpointRegistry stompEndpointRegistry) {
    stompEndpointRegistry.addEndpoint("/stomp")
            .setAllowedOrigins("*")
            .withSockJS() //SOCKJS requires http connection, other use ws://

            .setSessionCookieNeeded(false);
}

Then it gets tricky.

Everything works amazingly up to this point if you are using session based authentication. I'm not doing that in the Yolobe stack. Our api is secured via stateless JWT to make mobile integration easier  -you just need a couple services and an http interceptor to have angular send the correct headers every time. But what about Web sockets - can you send the right headers to have JWT work on websocket? Kind of.

Spring sockets was recently updated to allow JWT type tokens. The provided example will allow basic JWT authentication to work. But user-based topics and queues will stop functionining as the principal user is missing from the session. Youll have to inject it on every call.

A full solution

You'll need to create a pretty extensive configuration bean to integrate spring websocket and spring security rest:

Your configuration bean

webSocketConfig(YolobeWebSocketConfigurationBean) {
    jwtService = ref("jwtService")
}

Will need to parse JWT tokens on each request (inside configureClientInboundChannel )- assign the principal to the session/message, then forward the request to a new principal-aware user registry.

Configuration Bean

@Configuration

@EnableWebSocketMessageBroker

@Order(HIGHEST_PRECEDENCE)
public class YolobeWebSocketConfigurationBean extends AbstractWebSocketMessageBrokerConfigurer {

    public DefaultSimpUserRegistry _userRegistry = new DefaultSimpUserRegistry();
    public DefaultUserDestinationResolver _resolver = new DefaultUserDestinationResolver(_userRegistry)


    JwtService jwtService    

    MessageBrokerRegistry messageBrokerRegistry

    @Bean    

    @Primary    

     public SimpUserRegistry userRegistry() {
        return _userRegistry;
    }

    @Bean

    @Primary

    public UserDestinationResolver userDestinationResolver() {
        return _resolver;
    }

    def tokenPattern = ~/Bearer\s(?<token>\S+)/    

    private Principal getUser(StompHeaderAccessor accessor) {
        String header = accessor.getNativeHeader("Authorization")[0]
        def matcher = (header =~ tokenPattern)
        if (matcher.matches()) {
            try {
                JWT jwt = jwtService.parse(matcher.group('token'))
                return new SocketPrincipal(username:jwt.payload.toJSONObject()["sub"]);

            }
            catch (JOSEException formatError) {
                throw new Exception("Bad Authorization: Bearer - Token")
            }
        }
        throw new Exception("Bad Authorization: Bearer - Token")
    }


    @Override

    public void configureClientInboundChannel(ChannelRegistration registration) {


        registration.setInterceptors(new ChannelInterceptorAdapter() { //authenticate user on connection request            

                @Override

                public Message<?> preSend(Message<?> message, MessageChannel channel) {

                StompHeaderAccessor accessor = MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);

                Principal yourAuth = getUser(accessor)

                if (accessor.messageType == SimpMessageType.CONNECT) {
                    YolobeWebSocketConfigurationBean.this._userRegistry.onApplicationEvent(new SessionConnectedEvent(this, message, yourAuth));
                } else if (accessor.messageType == SimpMessageType.SUBSCRIBE) {
                    YolobeWebSocketConfigurationBean.this._userRegistry.onApplicationEvent(new SessionSubscribeEvent(this, message, yourAuth));
                } else if (accessor.messageType == SimpMessageType.UNSUBSCRIBE) {
                    YolobeWebSocketConfigurationBean.this._userRegistry.onApplicationEvent(new SessionUnsubscribeEvent(this, message, yourAuth));
                } else if (accessor.messageType == SimpMessageType.DISCONNECT) {
                    YolobeWebSocketConfigurationBean.this._userRegistry.onApplicationEvent(new SessionDisconnectEvent(this, message, accessor.sessionId, CloseStatus.NORMAL));
                }


                accessor.setUser(yourAuth);
                accessor.setLeaveMutable(true);
                return MessageBuilder.createMessage(message.payload, accessor.messageHeaders)
            }
        });
    }

    @Override    

    public void configureMessageBroker(MessageBrokerRegistry messageBrokerRegistry) {
        messageBrokerRegistry.enableSimpleBroker("/queue", "/topic");
        messageBrokerRegistry.setUserDestinationPrefix("/user")
        messageBrokerRegistry.setApplicationDestinationPrefixes("/app");
        this.messageBrokerRegistry = messageBrokerRegistry;
    }

    @Override

    public void registerStompEndpoints(StompEndpointRegistry stompEndpointRegistry) {
        stompEndpointRegistry.addEndpoint("/stomp")
                .setAllowedOrigins("*")
                .withSockJS() //SOCKJS requires http connection, other use ws://                .setSessionCookieNeeded(false);
    }

    @Bean

    public GrailsSimpAnnotationMethodMessageHandler grailsSimpAnnotationMethodMessageHandler(SubscribableChannel clientInboundChannel, MessageChannel clientOutboundChannel, SimpMessagingTemplate brokerMessagingTemplate) {
        GrailsSimpAnnotationMethodMessageHandler handler = new GrailsSimpAnnotationMethodMessageHandler(clientInboundChannel, clientOutboundChannel, brokerMessagingTemplate);
        handler.setDestinationPrefixes(["/app","/queue","/topic"]);
        return ((GrailsSimpAnnotationMethodMessageHandler) (handler));
    }

}

Client side

I used this fork of ng-stomp that allows multiple socket connections and disconnection notifications. here is some ugly sample code that shows what is possible. Credentials.decorate() is an angular service that holds stored credentials (persisted to localstorage) and creates a header map.

decorate :  function() {
    return { 'Authorization': this.getType() + " " + this.getToken() };

}

$stomp
    .connect("messaging", platformApiEndpoint.host+'/stomp',  Credentials.decorate() )
    .then(function (frame) {
        console.log(frame);        // notify callback function

        var showResponse = function (res) {
            console.log("RESPONSE! OMG", JSON.stringify(JSON.parse(res.body)));        

        };        

        $stomp.subscribe("messaging", '/user/queue/chat', Credentials.decorate()).then(null, null, showResponse);
        $timeout(function() { //example sending a message to topic            

                $stomp.send("messaging", '/app/incchat', {
                      id: $scope.threadId,                text: "connected VIA SOCK"            

                }, Credentials.decorate() );
        }, 1000);

Urlmappings break weirdly with Grails

Ever started your grails application up and recieved a vomit of stacktraces complaining about being unable to set the URLMappings Bean?

example:

Error creating bean with name 'grailsUrlMappingsHolder'

This is generally the result of the url mappings failing to "compile". Check that all your paths start with a "/" slash and any closures are correctly closed.

Intellij and Grails 3 Command Failure

File this one under "doh, thats obvious" but I completely spaced out as to why my Intellij commands to generate new grails resources - like creating a new taglib
 

Would fail with an error like:

"C:\Program Files\Java\jdk1.8.0_45\bin\java" -XX:+TieredCompilation [...] \lib\grails-rt.jar" org.grails.cli.GrailsCli create-taglib <className> Error |
Command [create-taglib] error: Profile [org.grails.profiles:base:3.1.6] declares and invalid dependency on parent profile [org.grails.profiles:base:3.1.6] (Use --stacktrace to see the full trace)

The solution is to ensure the intellij SDK settings are set to the same level of grails SDK specific within your gradle file. I had upgraded to grails 3.1.6 within my build, but IntelliJ uses its own settings to define which grails binary to use for generating classes.

(Notice how Intellij still thinks my project is using 3.1.8 although I had downgraded to 3.1.6 - another curious feature that may crop up later)

Switching Environments in Grails 3 BootStrap

Over the iterations of grails, the best method to do this has changed. But if you want to execute grails code only within a specific environment in grails 3, you can use the static methods on the grails.util.Environment class.

Eg:

switch (Environment.getCurrent()) {

    case Environment.DEVELOPMENT:
        20.times {
            //do some thing
        }
        break;
    case Environment.PRODUCTION:
        2.times {
            //do some other thing
        }
        break;
}

Gmail Oauth2 with Spring and JavaMail with Grails

Setting up Gmail Email Sending with Grails (2+, 3+) Service

Most Web applications will send some sort of email to users. Lost passwords, invitations to join. Its a good way to reach professional users. Not so great for young people (they never check them) but in order to send emails, you have to get the connection right. I didn';t build a particular groovy way, I used java code within a grails service to make a simple method available to controllers and other serivces:

interface EmailService {
    /**     
    * Send a set of emails and collect the results     
    * @param addresses list of addresses to send to
    * @param subject subject of email    
    * @param text  body of email   
    * @return a list of message IDs returned from the sending service    
    */    
    def send(List<String> addresses, String subject, String text)

    /**     
    * Setup any needed state from config files etc.     
    */    
    void initialize()
}

Most of the default grails java mail setup questions (I'm using spring mail with grails because its mostly already on the classpath) you will find involve turning off enhanced seucirty in your gmail account in order to get around the error message you will recieve on startup:

Authentication failed; nested exception is javax.mail.AuthenticationFailedException: 534-5.7.14 <https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbvO
534-5.7.14 SC261Te39VZ5jtNBz2mvwNtIGtZLxYulCRb8D2u6rGTAg69U2-tQsPDzI1YPgWUbVo1ZQm
534-5.7.14 MlSxYEJHBzyTk-tQKy-6GN5HACShag4XcqNYlxbyWHYvMyMICSTPuwRFzM_Rn2kUOKLcoY
534-5.7.14 hcEmEC6i4DrvVh4h8KTTdK1VxgyDwD6QzfDxWgUa0vM7ZcLRfIURv1CThW4B0G5XvVgG3a
534-5.7.14 t-wJ_9P1uU8YoJK2c-QbrhZe2H9qo> Please log in via your web browser and
534-5.7.14 then try again.
534-5.7.14  Learn more at
534 5.7.14  https://support.google.com/mail/answer/78754 h24sm11972231ioi.17 - gsmtp

Someone just tried to sign in to your Google Account someone@gmailaddress.com from an app that doesn't meet modern security standards." - Oh great, here goes my day

This cryptic error results from using a setup like (in resources.groovy):

emailService(GmailEmailService) {
                template=ref("templateMessage")
                mailSender=ref("mailSender")
            }

templateMessage(SimpleMailMessage) {
                from="email@sendergmail.com"
            }

mailSender(JavaMailSenderImpl) {
                host="smtp.gmail.com"
                port=587
                protocol="smtp"
                username="email@sendergmail.com"
                password="yourpassword"
                javaMailProperties = [
                        "mail.transport.protocol" : "smtp",
                        "mail.smtp.auth" : true,
                        "mail.smtp.starttls.enable" : "true",
                        "mail.smtp.quitwait" : true,
                        "mail.debug" : true
                ]
            }

But what if you wanted to use enhanced security? Well available in the more recent versions of JavaMail is support of Oauth2.

To begin with though, you are going to need to follow the google guidelines here to recieve api credentials for your webserver. You should end up with a set of JSON credentials for your service account. :

{
  "type": "service_account",
  "project_id": "someid",
  "private_key_id": "SOMEID",
  "private_key": "YOURKEY",
  "client_email": "someemail",
  "client_id": "someid",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "someurl"
}

There is a branch here. You can choose to give the service account Delegate domain-wide authority (which I have done) or leave it at user-interactive mode. Since I dont want to have to ask my own company user account for permissions (which makes sense for a back-end webservice making sending email from only one user), I chose  to upgrade the account to domain-wide, which can be done in the console.

Adding Domain Wide Delegation means no prompts, but using an additional secret file

We will need to use these credentials to create oauth tokens that can be sent with each senmail request. To create these tokens (oauthtoken) you can use the Google Java API. (at the time of writing, the latest central maven available was: 'com.google.api-client:google-api-client:1.21.0') . Google has a guide available - but essentially you will be using the builder to setup a trusted account.

GoogleCredential credential = GoogleCredential.fromStream(new FileInputStream(authFile)).createScoped(Collections.singleton("https://www.googleapis.com/auth/gmail.send")) //for SMTP access only

Then email setup can be pretty easy. A lot of the boiler plate of creating a javax.mail.transport can be made easier by using some Java Code provided by Google itself.

If like us, you were using google apps accounts - there is additional information required. As you may get responses like:

DEBUG SMTP: SASL: no response
DEBUG SMTP: SASL authentication failed

Things to check:

* You have allowed the client id access to the same scope you are requesting (Google Scopes listing)
* You are creating your Google credential and access token with a user to impersonate:

GoogleCredential credential = GoogleCredential.fromStream(new FileInputStream(authFile))                .createScoped(Collections.singleton("https://www.googleapis.com/auth/gmail.send"))
credential.serviceAccountUser = "someuser@yourdomain.com" //important

Final Thoughts

After going through all of this trouble, I ended up switching to using the google api client and the gmail api client for java, while still using JavaMail for convience (google has a great guide here)

By the way, I reccomend using scheduling with retries in order to make sure your customers recieve their emails. It can be really annoying when an app fails to send that crucial forgot password email - they will often leave and not come back.

Turns out doing it the right way is a bit more involved - but worth it. One day your email's wont stop randomly working when google finally axes support for the basic authentication.

Grails3 and Spring Security

ERROR org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[grailsDispatcherServlet] - Servlet.service() for servlet grailsDispatcherServlet threw exception
java.lang.IllegalStateException: There was a problem retrieving the current GrailsWebRequest. This usually indicates a filter ordering issue (the 'springSecurityFilterChain' filter-mapping element must be positioned after the 'grailsWebRequest' element when using @Secured annotations).

This issue was bugging me on a new grails3 and spring security web application. It turns out there are more than few active bugs on this issue. I filed my own:

https://github.com/grails-plugins/grails-spring-security-core/issues/412

After a week of noodling on the issue and trying various leads - the root cause seems to be spring security is delegating to the embedded tomcat error pages for routes only covered by static rule mappings (while your application is still using annotations). Instead, the plugin should correctly delegate to your mapped errors pages within the grails system. Included in the above post is my work around, which I will repeat here. This will cause all errors encountered within static routes to use the correct mappings.

You must setup an error controller:

class ErrorController {

    def error() {
        render view: 'error'    }

    def invalid() {
        render view: 'error'    }

    def denied() {
        render view: 'denied'    }

    def notFound() {
        render view: 'notFound'    }
}

And use this controller within your URL mappings:

"400"(controller: "error", action: "invalid")
"500"(controller: "error", action: "denied")
"403"(controller: "error", action: "denied")

"404"(controller: "error", action: "notFound")

And finally disable the spring-secuurity error page within application.groovy

grails.plugin.springsecurity.adh.errorPage = null

FlowJs Upload Reciever for Grails

I chose flow.js (https://github.com/flowjs/flow.js) for my client side file uploading due to its fault tollerance. I use using the ng-flow (https://github.com/flowjs/ng-flow/) angular library that builds an excellent set of directives around the framework. The one downside is the lack of documentation on what it takes to build a server side compliment to this. The particular beckend I am using is Grails3 based (although this would work for earlier versions of grails with little modification).

I built a service that can be plugged into multiple controllers to achieve the effect. While I wont publish the whole service, here is a skelleton that should be enough to get you started.

Simple Scripts for replacing html tags

I recently had to upgrade a tag library we were using. However the latest version had refactored out some of the tags into a separate tag library. I needed to do two things:

1) Add a reference to the new tag library on each page that had previously been using the base library
2) Change the tags using the old library to update to use the new prefix

#Add a new line (using a\ option) to the files containing reference to struts-jquery-tags

find . -type f \( -iname "*.tag" -or -iname "*.jsp" \) -print -exec sed -i '/struts-jquery-tags\"\s%>/ a\
<%@ taglib prefix=\"sjg\" uri=\"\/struts-jquery-grid-tags\" %>' {} \;

#Find and replace references to the old sj:grid to use the new :sjg:grid
find . -type f \( -iname "*.tag" -or -iname "*.jsp" \) -print -exec sed -i 's/sj:grid/sjg:grid/g' {} \;

#Finally strip out some redundant information from header tags
find . -type f \( -iname "*.tag" -or -iname "*.jsp" \) -print -exec sed -i 's/head\suseJqGridPlugin=\"true\"/head/g' {} \;

Script for fast Mysql Dump

This command will create a file named mysqldum.sql in the current dirrectory:


mysqldump DBNAME --add-drop-table -u USERNAME-p > mysqldump.sql

Remember when going to older versions of mysql to do a search and replace for "USING BTREE" as it is not supported in older mysql installs.

This file can then be imported via:

 mysql DBNAME -u USERNAME -p < mysqldump.sql 

This post is mostly for my own information, I keep forgetting the exact order of the arguments!

Flowplayer 201 Error in Commercial Build

Our site had been a long time customer of flowplayer. We have a commercial license. However the person who had purchased said license was no longer at the company and the account was tied to her email address. We didnt get the memo when flowplayer moved its builds to amazon s3.

Luckily my browser remembered the password to the account site - and I changed the location of the flash builds. However they were still using an older version, 3.2.5-0. After changing the links, the player would load up, but report a 201 error,:

201, Unable to load stream or clip file, connection failed, clip: '[Clip]

The fix? I needed to download and use the newer commercial builds found on the account page under "Commercial license. v3.2" - that, along with the new javascript libraries associated (3.2.8 at the time of writing) got things going again.

Detecting environment for Google Web Toolkit

Have a piece of code that you want to run only when running in Hosted Mode? I made a small utility class that uses one method exposed in the Google Web Toolkit API to do just that.


import com.google.gwt.core.client.GWT;


/**
 * Utility Class to help write debugging code
 * References to these function could be searched for and removed when deploying to production for increased speed.
 * @author Jason Lambert
 *
 */
public class Development {
public static boolean isDevelopmentEnvironment() { return !GWT.isScript(); } //detect hosted mode


}

Inspired by how Grails handles development, test and production environments, I  made this quick static utility function.

GWT.isScript detects if the current environment is running in compiled byte code or not. useful for switching to download local resources when perhaps same origin policy (SOP) prevents your code from downloading data from external databases or servers.

Tomcat SEVERE: Error listenerStart - Debugging.

From my travels across Google, this error is common. In Grails applications it is almost always from a bad datasource.groovy file. This error is the non descriptive string you receive when deploying your servlet (Grails, Spring, Otherwise) to tomcat (probably version 5.5 or higher). The servlet probably runs fine in development mode on your machine. How do we debug it? Read on.

Lets start by getting more useful error messages from Tomcat.
Tomcat by default uses java.util.logging, which is giving you the basic error messages along the lines of...


DATETIME org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
DATETIME org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/AppName] startup failed due to previous errors

Lets configure tomcat to use log4j, which will produce a much better call stack to debug. Taken from advice here, I'll paraphrase [read the source material for a full understanding however].

1. Stop Tomcat
2. Install Log4j on your deployment machine. (package install available for a lot of linux distros)
3. Download tomcat-juli-adapters.jar and tomcat-juli.jar from the extras section of the tomcat website (For version 7.0 that extras page is here) and place them into the $CATALINA_HOME/lib folder
4. Replace the $CATALINA_HOME/bin/tomcat-juli.jar with that same tomcat-juli.jar you just placed in lib.
5. Delete $CATALINA_BASE/conf/logging.properties to prevent the old logging system from being used
6. Start Tomcat

In my case, the error stack now looked like the following. As you can see like an idiot I had left the driverClassName to the grails 2.0 default of  org.h2.Driver instead of the com.mysql.jdbc.Driver required to connect to a mysql service. Interestingly my eclipse/tomcat development setup was forgiving me and loading the correct driver anyway - be warned!

SEVERE: Exception sending context initialized event to listener instance of class org.codehaus.groovy.grails.web.context.GrailsContextLoaderListener

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creati$

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

        at java.util.concurrent.FutureTask.run(FutureTask.java:166)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

        at java.lang.Thread.run(Thread.java:679)

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.sp$

        ... 6 more

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Cannot resolve reference to bean 'hibernateProperties' while setting bean property 'hibernateProperties'; nested exception is $

        ... 6 more

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hibernateProperties': Cannot resolve reference to bean 'dialectDetector' while setting bean property 'properties' with key [hibernate.dialect];$

        ... 6 more

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dialectDetector': Invocation of init method failed; nested exception is org.springframework.jdbc.support.MetaDataAccessException: Error while e$

        ... 6 more

Caused by: org.springframework.jdbc.support.MetaDataAccessException: Error while extracting DatabaseMetaData; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of class 'org.h2.Driver' for connect URL '$

        ... 6 more

Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of class 'org.h2.Driver' for connect URL 'jdbc:mysql://xxx.xx.xx.xx:3306/mydatabasename?autoreconnect=true&zeroDateTimeBehavior=convertToNull'

        at org.apache.commons.dbcp.BasicDataSource.createConnectionFactory(BasicDataSource.java:1452)

        at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1371)

        at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)

        ... 6 more

Caused by: java.sql.SQLException: No suitable driver

        at java.sql.DriverManager.getDriver(DriverManager.java:279)

        at org.apache.commons.dbcp.BasicDataSource.createConnectionFactory(BasicDataSource.java:1437)

        ... 8 more

Searching Multiple Data Sources in Grails

Have you got domain classes mapped to multiple datasources? The Searchable plugin will mysteriously not be building indexes for domain classes in your secondary datasources by default. The following explains a way to change the behavior.

This is taken from my personal experience, and information compiled from a helpful thread archived on Grails, found here. We are going to need to redefine some beans.

Step 1:
Open conf/spring/resources.groovy


add in the following bean:


import org.compass.gps.device.hibernate.HibernateGpsDevice //need this import for new datasource
import grails.plugin.searchable.internal.compass.config.SessionFactoryLookup //need this import to get correct sessionfactory class


beans = {


/* ~ your other beans here */



compassGpsDevice(HibernateGpsDevice) { bean ->
bean.destroyMethod = "stop"
name = "hibernate"
sessionFactory = { SessionFactoryLookup sfl ->
sessionFactory = ref('sessionFactory_datasourceName') 
}//replace datasourceName with the name of the datasource as defined in conf/DataSource.groovy
fetchCount = 5000
}
}


Step 2:
You could stop here if this was the only datasource that contained indexable classes, but continue to add each compassGPSdevices for each additional datasource (including the default), making sure they are unique names ie:



anotherUniquecompassGpsDevice(HibernateGpsDevice) { bean ->
bean.destroyMethod = "stop"
name = "unqiueHibernateName"
{ SessionFactoryLookup sfl ->
sessionFactory = ref('sessionFactory_uniquedatasource') 
}
fetchCount = 5000
 }


then finally we need to combine all of these datasources into one SingleCompassGps for searchable. Add this last bean:


import org.compass.gps.impl.SingleCompassGps

compassGps(SingleCompassGps) {
compass = ref('compass')
gpsDevices = [compassGpsDevice, anotherUniqueCompassGpsDevice /* ~add the other sources in here */ ]
}

Thanks to the original thread for guidance. Hopefully this compilation of guidance will help someone else save some time. The major additions beyond the original thread are to use the sessionfactorylookup. You can see these objects in SearchableGrailsPlugin.groovy - we are simply re defining them with new datasources!

Undocumented? Grails scaffolding feature. Widgets

Present in old docs, but weirdly not the latest, you can request a different widget on scaffolded pages for your domain class.

In the domain's scaffold closure:

def constraints = {
        myStringField(widget:'textarea')
}

Make a file list quickly

Need to write a bunch of filenames to a list (say to build a resources list for a Grails plugin or html page) because you can't include just folders? But you don't want to type them all out individually?

Open that good old standby, cmd.exe  or terminal and navigate to the root folder containing the files.

[Windows] command: DIR *.* /s /b > ../yourlistname.LST

[Linux/OSX] command:  find * . > yourlistname


You can easily add extensions to the DIR and find commands to find only a particular kind of file (DIR *.js for example) Open in your favorite text editor (why not try notepad++ or textmate) use some regex-replacing..


Lets try removing directories from the file list (this could also be done with additional filters in the find commands above):

• In notepad++, turn on regular expression searching, and check 'mark line'
• Search for [\w+][\.][\w+], which will bring up every line with a . in it (a valid extension)
• Invert the bookmarking with Search->Bookmark->Inverse Bookmark
• Finally delete the now bookmarked lines without file extensions by Search->Bookmark->Delete Bookmarked Lines

And your done.

Fun fact: Grails GString's require double quotes

Will work:


def element = "<script type=\"text/javascript\" src=\"${g.resource(dir: 'folder', file: 'file.js')}\"></script>"
out <<  element.toString()


Wont work:


def element = '<script type="text/javascript" src="${g.resource(dir: "folder", file: "file.js")}"></script>'
out <<  element.toString()


And by "wont work" the groovy expressions in the GString will not be evaluated, they will be delivered literally. Note the double quotes beginning and ending the GString in the first example.

I knew this. Yet I was dissecting why my unit test was failing for over an hour. Turns out junit can't detect silly human error! Since I re-learnt this lesson, I thought I would share it with you, blogosphere!

Grails Codec file location and extension

If you are just beginning out in Grails development, the creation of custom codecs can greatly increase productivity. Codecs help you code/decode (funny that) strings. Grails ships with a bunch of them already, but when you feel like you want to create you own, greate a groovy class file (extension .groovy), with the name ending in "Codec" in your projects graips-app/utils folder. Remember that if you want to place these files in packages, you will need to create the additional folder structure to match the package structure. A sample codec structure is shown below. This file would be in the folder grails-app/utils/org/mycompany/:


package org.mycompany
import /* imports here */

class SampleCodec{
  static encode = { str ->
   /* Coding data goes here */
  }

static decode = { str ->
   /* Decoding data goes here */
  }

}

Once you start your grails app, Spring will inject the methods encodeAsSample() and decodeSample() to the Java.lang.String class.

Start creating custom hash codes for your passwords now!

JMX and Grails

Developing on windows can sometimes really be a pain. I had found documentation all over the web as to where to place my JVM arguments to make grails accept remote JMX connections while developing in eclipse. Most have you modifying $GRAILS_HOME/bin/grails.bat but I was noticing in Java Visual VM (great tool by the way) that the VM arguments were not changing.

In one of those 'the answer is simple stupid' moments - some clarity was given to me by this page of documentation: http://www.objectpartners.com/2009/05/27/eclipse-setup-for-grails-11-development/

By adding the following string the run configuration for my Grails project:

-Dcom.sun.management.jmxremote -Djava.rmi.server.hostname=<yourHostIPHere> -Dcom.sun.management.jmxremote.port=9004 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
[Note this is obviously not a production setting, you will NEED AUTHENTICATION in production]

I was able to get jconsole to connect to the remote process. Of course local connections had never been a problem as JMX is enabled locally by default.

For anyone curious, the run configuration for a tutorial project is shown below.

A run configuration for JMX with Grails on Windows Eclipse.