Make Groovy’s Grape.grab work when running compiled groovy code with java

5. February 2017Michael Lösch Leave a comment

Groovy’s Grape dependency management is pretty handy if you just want to write Groovy scripts and don’t want to use a full-blown dependency manager (gradle, maven). You just annotate @Grab to your Groovy script and it downloads the defined libs (powered by Ivy).

In some constellations, pain starts when the systemclassloader doesn’t know about the loaded classes/libs.

This can be fixed by setting @GrabConfig(systemClassLoader = true) (see e.g. mrhaki). But it’s getting really ugly when you don’t have a suitable ClassLoader (see groovy.grape.GrapeIvy <isValidTargetClassLoaderClass>) like GroovyClassLoader or RootLoader.

You’ll find this situation when running compiled groovy-code with java (e.g. when deploying your groovy code in a fatjar/shadowjar/uberjar). What you actually can do is to instantiate a GroovyClassLoader and call Grape.grab using this one. But you’ll end up in the same situation when you have to call @GrabConfig(systemClassLoader = true), like I said above. Running compiled groovy code with java means your SystemClassLoader is not a suitable one. You just can’t replace existing loaders by a GroovyClassLoader or inject it somehow somewhere.

I had this situation the other day in my open-source project Grip. I’m deploying with gradle and the shadowJar-plugin, so I run the compiled code with java and NOT groovy. Grip has an Interpreter to execute groovy scripts with some extras like scheduling, easy DB access and so on. And I want to provide grab in the scripts to allow users to import whatever they want.

Due to the fact that you can use Grape.grab with your own GroovyClassLoader, the basic mechanism works (dependencies are getting resolved). But what needs to be done is to transfer the information about the downloaded libs to the existing SystemClassLoader.

This is a hack, of course. But to me there’s no other way to deal with it (except to renounce Grape.grab):

After some research, I found a way to add libs to the SystemClassLoader. See ClassPathHacker.java: http://www.javafaq.nu/java-example-code-895.html

Now having ClassPathHacker, I was able to add the resolved dependencies

def groovyLoader = new GroovyClassLoader()

//grab dependencies using a GroovyClassLoader
Grape.grab(classLoader: groovyLoader, dependencies)

//using org.reflections.util.ClasspathHelper to get the URLs
def urls = ClasspathHelper.forClassLoader(groovyLoader).toArray() as URL[]

//and finally adding the URLs to the SystemClassLoader with ClassPathHacker
def cph = new ClassPathHacker()
urls.each { URL url ->
     if (url.file.endsWith('jar')) {
         cph.addFile(url.file)
     }
}

def groovyLoader = new GroovyClassLoader()

//grab dependencies using a GroovyClassLoader

Grape.grab(classLoader: groovyLoader, dependencies)

//using org.reflections.util.ClasspathHelper to get the URLs

def urls = ClasspathHelper.forClassLoader(groovyLoader).toArray() as URL[]

//and finally adding the URLs to the SystemClassLoader with ClassPathHacker

def cph = new ClassPathHacker()

urls.each { URL url ->

if (url.file.endsWith('jar')) {

cph.addFile(url.file)

}

That’s it. Libs loaded by Grape are now known by the SystemClassLoader.

When calling ClasspathHelper.forClassLoader, some errors appeard in the logfile: could not create Vfs.Dir from url (see https://github.com/ronmamo/reflections/issues/80). Those can be avoided by using this Helper https://gist.github.com/nonrational/287ed109bb0852f982e8

Find the whole implementation in package de.metacode.grip.core.classloaderhack in the project https://github.com/codeeraser/grip

Enhance your Groovy DSL in a plugin-ish way

20. August 2015Michael Lösch Leave a comment

The base of a groovy DSL is the Script class or the DelegatingScript. To provide some functions to your DSL, you define them in this class, implement the missingMethod or enhance the metaclass. No matter which way you choose, it blows up your class with rising amount of functions. So after some time you might want to restructure your code. For some other reason you might want to enable other people to enhance the DSL without touching the base DSL class.

The idea is to provide a class that can be instantiated in you DSL with a create-method.

1. Define a marker interface to find the plugin class (you can do this with annotations as well)

Every class implementing this marker interface will be instantiable in your DSL

interface Instantiable {}

1	interface Instantiable {}

2. A class that enhances your DSL

We’ll have a Car class that has only one field describing the color of our car

class Car implements Instantiable {
    def color = "blue"

    def printColor() {
        println color
    }
}

class Car implements Instantiable {

def color = "blue"

def printColor() {

println color

}

3. The actual DSL-backing class

Now we’ll go to the bone. As mentioned above, there are several ways to enhance the DSL class with new functions. In this sample we’ll add new functions by enhancing DSL’s metaClass.

We’ll have two functions to get an instance of Car: newCar and newCarWith.

newCar is calling the Default-Constructor without arguments. newCarWith calls the “Named” Default-Constructor with a map. To achieve this, there are two closures defined in line 3 and 4. First one with two params (actual class and the map) and one with a single param (only the class to instantiate).

Now we need to find the classes to instantiate. I used the reflections-API to find all classes implementing the Instantiable interface (line 6). In this sample only classes of the package de.metacode.dslplugin are scanned. You can scann the whole known classes as well.

Line 7 and 8 is where the magic happens: Two new methods are added to metaClass. Note the curry-call on the newInstance-closures. What this does is called currying (or Schönfinkeln). It reduces a function with n arguments to a function with n-1 arguments. In this case curry generates a new closure without the first parameter Class clazz. So we have one closure without any parameter and another one with a map.

class DSL {
    static {
        def newInstanceWithArgs = { Class clazz, Map args -> clazz.newInstance(args) }
        def newInstance = { Class clazz -> clazz.newInstance() }

        new Reflections("de.metacode.dslplugin").getSubTypesOf(Instantiable).each { Class clazz ->
            DSL.metaClass."new${clazz.simpleName}With" = newInstanceWithArgs.curry(clazz)
            DSL.metaClass."new${clazz.simpleName}" = newInstance.curry(clazz)
        }
    }
}

class DSL {

static {

def newInstanceWithArgs = { Class clazz, Map args -> clazz.newInstance(args) }

def newInstance = { Class clazz -> clazz.newInstance() }

new Reflections("de.metacode.dslplugin").getSubTypesOf(Instantiable).each { Class clazz ->

DSL.metaClass."new${clazz.simpleName}With" = newInstanceWithArgs.curry(clazz)

DSL.metaClass."new${clazz.simpleName}" = newInstance.curry(clazz)

}

4. Writing some DSL-code

Now we can instantiate the Car without params or with named params:

blueCar = newCar() //blue is the default value
blueCar.printColor()

redCar = newCarWith color: 'red'
redCar.printColor()

blueCar = newCar() //blue is the default value

blueCar.printColor()

redCar = newCarWith color: 'red'

redCar.printColor()

The output is

blue
red

blue

red

That’s it

Here you’ll find executable wired-together code:

https://gist.github.com/codeeraser/31d031cd921e168049ac

You’ll might get some issues with reflections-API when running this code via command-line. To get reflections work like one would expect can be pretty annoying (see stackoverflow…). The shortest way is to run the sample in an IDE.

Groovy DSL: How to split up execution of method calls like gradle does

3. May 2015Michael Lösch Leave a comment

This is how you can implement a gradle-ish behavior of executing a designated method before everything else in a dsl script (gradle executes doFirst{} first). Actually, I have no clue how gradle does it. This approach is about manipulating the AST with an implementation of CompilationCustomizer. I’ll describe two variations. First, how to execute the script twice, e.g. to set up an environment and execute the actual code sometimes later. And than how you can re-order the statements before execution. It’ll be helpful if you know about how AST transformations work (if you don’t, see the links at the end of this article).

Let’s say we have a DSL script like this:

println "this is text that will not be printed out in first line!"

doFirst { println "First things first: e.g. setting up environment" }

doStuff { println "doing some stuff now" }

println "That's it!"

println "this is text that will not be printed out in first line!"

doFirst { println "First things first: e.g. setting up environment" }

doStuff { println "doing some stuff now" }

println "That's it!"

First variation:

We’ll handle the DSL using DelegatingScript. The is the delegate for the first run (only doFirst will be executed):

class DoFirstProcessor {
    def doFirst(Closure c) {
        c()
    }
}

class DoFirstProcessor {

def doFirst(Closure c) {

c()

}

For the second run this is doing everything else:

class TheRestProcessor {
    def doStuff(Closure c) {
        c()
    }

    def methodMissing(String name, args) {
        //nothing to do
    }
}

class TheRestProcessor {

def doStuff(Closure c) {

c()

}

def methodMissing(String name, args) {

//nothing to do

}

At this point we’ve got everything we need to execute the DSL. Now comes the AST transformation part. The following CompilerCustomizer removes every statement from the AST except a designated method. Because there can only be one (method), this is called the HighlanderCustomizer 😉

class HighlanderCustomizer extends CompilationCustomizer {
    def methodName

    HighlanderCustomizer(def methodName) {
        super(CompilePhase.SEMANTIC_ANALYSIS)
        this.methodName = methodName
    }

    @Override
    void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {
        def methods = classNode.getMethods()
        methods.each { MethodNode m ->
            m.code.each { Statement st ->
                if (!(st instanceof BlockStatement)) {
                    return
                }
                def removeStmts = []
                st.statements.each { Statement bst ->
                    if (bst instanceof ExpressionStatement) {
                        def ex = bst.expression
                        if (ex instanceof MethodCallExpression) {
                            if (!ex.methodAsString.equals(methodName)) {
                                removeStmts << bst
                            }
                        } else {
                            removeStmts << bst
                        }
                    } else {
                        removeStmts << bst
                    }
                }
                st.statements.removeAll(removeStmts)
            }
        }
    }
}

class HighlanderCustomizer extends CompilationCustomizer {

def methodName

HighlanderCustomizer(def methodName) {

super(CompilePhase.SEMANTIC_ANALYSIS)

this.methodName = methodName

}

@Override

void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {

def methods = classNode.getMethods()

methods.each { MethodNode m ->

m.code.each { Statement st ->

if (!(st instanceof BlockStatement)) {

return

}

def removeStmts = []

st.statements.each { Statement bst ->

if (bst instanceof ExpressionStatement) {

def ex = bst.expression

if (ex instanceof MethodCallExpression) {

if (!ex.methodAsString.equals(methodName)) {

removeStmts << bst

}

} else {

removeStmts << bst

}

} else {

removeStmts << bst

}

st.statements.removeAll(removeStmts)

}

Bring it all together:

def cc = new CompilerConfiguration()
cc.addCompilationCustomizers new HighlanderCustomizer("doFirst")
cc.scriptBaseClass = DelegatingScript.class.name

def doFirstShell = new GroovyShell(new Binding(), cc)
def doFirstScript = doFirstShell.parse dsl
doFirstScript.setDelegate new AllStuffProcessor()
doFirstScript.run()

// Next run could be sometimes later. This will handle everything else than doFirst:

cc.compilationCustomizers.clear() // no customizer needed for second run
def shell = new GroovyShell(new Binding(), cc)
def script = shell.parse dsl
script.setDelegate new TheRestProcessor()
script.run()

def cc = new CompilerConfiguration()

cc.addCompilationCustomizers new HighlanderCustomizer("doFirst")

cc.scriptBaseClass = DelegatingScript.class.name

def doFirstShell = new GroovyShell(new Binding(), cc)

def doFirstScript = doFirstShell.parse dsl

doFirstScript.setDelegate new AllStuffProcessor()

doFirstScript.run()

// Next run could be sometimes later. This will handle everything else than doFirst:

cc.compilationCustomizers.clear() // no customizer needed for second run

def shell = new GroovyShell(new Binding(), cc)

def script = shell.parse dsl

script.setDelegate new TheRestProcessor()

script.run()

The console output should look like this:

First things first: e.g. setting up environment
this is text that will not be printed out in first line!
doing some stuff now
That's it!

First things first: e.g. setting up environment

this is text that will not be printed out in first line!

doing some stuff now

That's it!

Second variation:

Now the DSL will not be executed in two steps but rather in one single run. We’ll use a CompilationCustomizer again. This time the customizer finds a designated method call and changes the order of execution.

Our Delegate (this time only one is needed):

class AllStuffProcessor {
    def doFirst(Closure c) {
        c()
    }
    def doStuff(Closure c) {
        c()
    }

    def methodMissing(String name, args) {
        //nothing to do
    }
}

class AllStuffProcessor {

def doFirst(Closure c) {

c()

}

def doStuff(Closure c) {

c()

}

def methodMissing(String name, args) {

//nothing to do

}

The ReOrderCustomizer:

class ReOrderCustomizer extends CompilationCustomizer {
    def methodName

    ReOrderCustomizer(def methodName) {
        super(CompilePhase.SEMANTIC_ANALYSIS)
        this.methodName = methodName
    }

    @Override
    void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {
        def methods = classNode.getMethods()
        methods.each { MethodNode m ->
            m.code.each { Statement st ->
                if (!(st instanceof BlockStatement)) {
                    return
                }
                def doFirstStmt
                st.statements.each { Statement bst ->
                    if (bst instanceof ExpressionStatement) {
                        def ex = bst.expression
                        if (ex instanceof MethodCallExpression) {
                            if (ex.methodAsString.equals(methodName)) {
                                doFirstStmt = bst
                            }
                        }
                    }
                }
                if (doFirstStmt) {
                    st.statements.remove(doFirstStmt)
                    st.statements.add(0, doFirstStmt)
                }
            }
        }
    }
}

class ReOrderCustomizer extends CompilationCustomizer {

def methodName

ReOrderCustomizer(def methodName) {

super(CompilePhase.SEMANTIC_ANALYSIS)

this.methodName = methodName

}

@Override

void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {

def methods = classNode.getMethods()

methods.each { MethodNode m ->

m.code.each { Statement st ->

if (!(st instanceof BlockStatement)) {

return

}

def doFirstStmt

st.statements.each { Statement bst ->

if (bst instanceof ExpressionStatement) {

def ex = bst.expression

if (ex instanceof MethodCallExpression) {

if (ex.methodAsString.equals(methodName)) {

doFirstStmt = bst

}

if (doFirstStmt) {

st.statements.remove(doFirstStmt)

st.statements.add(0, doFirstStmt)

}

Bring it all together:

def cc = new CompilerConfiguration()
cc.addCompilationCustomizers new ReOrderCustomizer("doFirst")
cc.scriptBaseClass = DelegatingScript.class.name

def shell = new GroovyShell(new Binding(), cc)
def script = shell.parse dsl
script.setDelegate new AllStuffProcessor()
script.run()

def cc = new CompilerConfiguration()

cc.addCompilationCustomizers new ReOrderCustomizer("doFirst")

cc.scriptBaseClass = DelegatingScript.class.name

def shell = new GroovyShell(new Binding(), cc)

def script = shell.parse dsl

script.setDelegate new AllStuffProcessor()

script.run()

Additionally, one should check that the designated method appears only once in the DSL script.

To understand how the AST looks like, just debug the customizer’s call method.

This article is based on a question I postet on Stackoverflow: http://stackoverflow.com/questions/29967886/groovy-dsl-how-can-i-let-two-delegating-classes-handle-different-parts-of-a-dsl

Links:

Great example of using Groovy to implement a DSL: https://groovy.codeplex.com/wikipage?title=Guillaume%20Laforge’s%20%22Mars%20Rover%22%20tutorial%20on%20Groovy%20DSL’s

About CompilationCustomizer: http://www.jroller.com/melix/entry/customizing_groovy_compilation_process

AST transformation workshop: http://melix.github.io/ast-workshop/

Joe’s series of AST articles: http://joesgroovyblog.blogspot.de

GWT.create 2015

4. February 2015Michael Lösch Leave a comment

Last week, I attended GWT.create in Munich. I made a lot of notes and possibly it may interest somebody who’s working with GWT. So what I want to provide here is a mixture of impressions, facts and my own evaluation of certain issues. If one of the speakers is reading this and discovers something completely wrong, please contact me or leave a post in the comment section.

The vaadin guys recorded every talk: http://gwtcreate.com/videos/index.html

And here’s a link collection of the slides: http://gwtcreate.com/slides/

Before I go into detail, let me first make the following remark: There was a lot of valuable content and I would encourage you to visit GWT.create if you do work with GWT. Sometimes it was hard to decide which talk to hear since there were at least two talks at a time.

Some issues were a constant theme throughout the conference: Webcomponents, JavaScript interoperability and multi platform. At least, this was my impression due to some talks I’ve heard:

First day

Keynote GWT 3 ( Ray Cromwell )

Like one would expect, the Keynote was about building a bridge between past and future of GWT. Impressive to me was to realize how performance of JavaScript engines improved over the last couple of years. As Ray told us, the performance improved about 2200% from the time of Firefox 3.5 being state of the art to Firefox 25 (later on, I’ll tell you about a nice talk about computer vision with GWT, which is possible due to great js performance). On the other side, the GWT compiler was optimized in terms of performance significantly.

Debugging is always a tricky topic because of the expiring DevMode and its substitution by SuperDevMode. Since SuperDevMode made a lot of progress in respect of usability (e.g. source maps), now it’s possible to debug in Eclipse again (IntelliJ will follow).

Elemental 2.0 will be released within GWT 3 and it seems to be a pretty cleverly implementation. Elemental is about being able to use Browser-APIs really close to the metal. To avoid the time gap between availability of new APIs and their support in GWT, Elemental 2.0 will parse the W3C specs to generate the interface and documentation! There’ll be no need to wait for a GWT update anymore.

A really neat thing is when the Googlers give an insight of what and how Google is doing with GWT. In 2013 there was Google Sheet, this year Ray told us about Google Inbox (the new Gmail client). Both are hybrid apps, which means they run on multiple devices with a specific UI, but still sharing the most of their code (he also gave a talk about Inbox – later more). A side-effect of those insights is that people who foresee the dying of GWT can be calmed down. GWT is the base of Sheets, Inbox, Groups and there are some new products in the pipeline.

Another nice new feature which was developed for Inbox is called DeltaJs. It allows to transport only the diffs of your JavaScript file over the network. Taking into account that complex apps can result in large js files, this sounds like a real advantage when working with mobile networks (e.g. the cache.html file of the GWT project of my employer is about 10MB).

There is no release date for GWT 3.0 yet.

Slides: https://drive.google.com/file/d/0B3ktS-w9vr8IbERCZ2VXbjZYeVE

Singular ( Daniel Kurka )

Singular is like AngularJs for Java. At least it’s the same idea but with some nice benefits.

Using AngularJs, you have so called ‘directives’ to bind UI fields with data fields which are kept in sync automatically. This means you’ve got a 2-way data binding out of the box. To do this, AngularJs walks the DOM at runtime. This approach doesn’t scale when the DOM gets bigger.

Singular resolves these binding dependencies at compile time (including validation by the compiler).

So using Singular, you would write your HTML with some directives to bind the UI to your controller/presenter code. The consequence of this is that you need to code HTML and you can’t use your existing widgets (well, you can. But it needs some effort – later more). That might sound a little scary right now. But if you write your HTML using Webcomponents this seems to be a clean way to build stuff. And you are not tied to a web language, because instead of having HTML including directives, you can also have an android XML layout. So you would have specific UI code for every platform but model/logic/controller/presenter code is covered at one place (sounds a lot like what Google did with Inbox).

All these sounds really like everybody would want to have it. Right now. But I’m curious to see how integration in existing projects will work out. Think of existing LayoutPanel hierarchies. And of course your self-made widgets must be available as webcomponents. Nevertheless, if I’d start a new project, I would intensely dealing with Singular.

One drawback exists yet: there’s no i18n support at the moment.

This was one of that kind of talks where you could gain the impression that Widgets are almost dead. With the upcoming Webcomponents they really seem to be out of date. Looks like building apps with GWT moves the engineer closer to the DOM (in case of web apps) or, more abstract, closer to a native UI language.

But there’s a problem with Webcomponents (custom elements): It’s only supported by Chrome & Opera natively ( see what caniuse says: http://caniuse.com/#feat=custom-elements).

This is what the next talk was all about.

Slides: http://daniel-kurka.de/talks/gwtcreate15/singular.pdf

Polymer & Elements ( Chris DiGiano )

This is a hack. Something that fits perfect to some kind of attitude that (in my opinion) is typical of Google. Webcomponents/Custom Elements is poorly supported by most browsers. If the world is a bad place, sometimes Google is going to fix that. This is something that reminds me of the first time I heard about GWT, which took care of all the little nasty things you had to deal with due to different browser’s behavior.

Polyfills enables browsers to support custom elements. You can use custom elements right now just by including a JavaScript library.

It’s also a set of Webcomponents. There’s visual stuff like paper-elements (keyword: material design). And there’s non-visual stuff like core-storage or core-ajax.

State of Polymer is ‘dev preview’. The 0.8 will be the nex release. 1.0 will be there round about middle 2015.

I just hope performance will be fine.

Slides: https://docs.google.com/file/d/0B97Mv38I4V6VUWQtWHZ5U2JqY2c

What’s new with SuperDevMode ( Brian Slesinsky )

Brian Slesinsky gave a detailed view of SuperDevMode’s state. From my point of view, there were two important statements:

IDE-debugging in Eclipse and IntelliJ Ultimate works.
Variables will be formatted in a more readable way.

I think improving SuperDevMode is a very important thing to counter the argument debugging GWT apps is time-consuming and complicated.

Slides: https://docs.google.com/presentation/d/16KvvgpMC5SNjiSS45-Ynm0x5CxlGmgU4HzSGqRSpJa8/edit#slide=id.p

Deep Dive in JS Interop ( Ray Cromwell )

This was a talk with lots of concrete examples how JS Interop will appear in the future. See the detailed examples in the slides. The talk’s core: You’ll need to write less boilerplate to use JS-APIs, exporting GWT-APIs will be pretty easy and that nice way of providing new Web-APIs by parsing the W3C specs (as I mentioned in Keynote section above).

It looks like GWT will be able to serve as the central hub of big web-apps/systems in particular due to the fact that it will probably blend in with its surroundings in a smooth way (JavaScript but also iOS’s ObjectiveC/UiBinder and Android’s Dalivk/ART).

Slides: https://drive.google.com/file/d/0B3ktS-w9vr8IN3d6UUNyWTJIOTQ

Google Inbox: Multi Platform Native Apps with GWT and J2ObjC ( Ray Cromwell )

Ray introduced Inbox as the new Gmail client. He described the process of finding the best architecture due to some must have features like performance, sharing code as much as possible (take a look at the slides) and being able to roll-out on iOS, Android and Web.

In summary, the idea is to write native UIs to achieve best user experience on every device but code everything else only once. The result is that Inbox is based on up to 70% Java code. The server part is fully written in java. The client has a high java proportion.

How does that work:

Web: Export functions with JSInterop for the Gmail JavaScript ninjas
Android: almost nothing to do, it’s java…
iOS: Use the J2ObjC transpiler to get ObjectiveC code. Be aware of some memory pitfalls.

The data is shared using ProtoBufs.

I guess this is a promising compromise. They could have used something like mgwt but that means one would compromise the user experience and had to deal with other problems (e.g. wrap the app to publish in AppStore). In the end, there’s no successful OneSizeFitsAll-UI out there (see Swing and Tom’s quote in the text below).

Here’s a simple sample-project to dive into hybrid apps: https://github.com/Sfeir/jhybrid

Arstechnica about Inbox: http://arstechnica.com/information-technology/2014/11/how-google-inbox-shares-70-of-its-code-across-android-ios-and-the-web/

Slides: https://drive.google.com/file/d/0B3ktS-w9vr8IS2ZwQkw3WVRVeXc

J2ObjC: Sharing Java Code with iOS Applications ( Tom Ball )

Tom Ball started the talk with a story from those days when he was a member of the Swing team at Sun. For the younger ones: This was about having one java UI lib that looks like native UI on several operation systems. Here’s a quote I noted down because I really liked it. It’s about trying to build such a write-once-run-everywhere UI lib:

You can get close enough to sell it, but you can’t get close enough to support it.

This talk provided many technical details. Just some key facts about J2Objc:

J2Objc is focused on shared data models and business logic. The output of this transpiler is pretty ugly and there’s no UI support (lessons learned). As you have JSNI for JavaScript, there’s OCNI for ObjectiveC and it comes with JUnit support. You need to take care of retention cycles that lead to memory leaks, but there’s an annotation driven approach to solve these kind of problems.

Slides: https://docs.google.com/presentation/d/1mW_Plm5jAygELf7qjVK7ZSIFjha0Q3TjOttX4x8oVWY/pub?start=false&loop=false&delayms=3000&slide=id.g6c99c794a_2_26

Second day

Ecosystem Keynote ( Hosted by Joonas Lehtinen )

The Ecosystem Keynote is not what you would expect from a typical Keynote. It’s more like a community action where lots of speakers give a short preview of their following talks and frameworks/libraries/tools. So this is a good point to find out what talks you want to visit this day. And you get an impression about how active the GWT community and 3rd party actually is.

DIY: Computer Vision with GWT ( Alberto Mancini & Francesca Tosi )

By the title, one could think this is just a talk about having fun with CV or to relax from the more ‘serious’ topics. I agree that computer vision is (sadly) not the typical playing field of most software engineers. But what I saw was a very nice example how GWT can act as a hub sticking different libraries and technologies together in an economic way. They had C-libraries containing complex algorithms, a real time computer vision library ( boofcv.org ) written in java and adapted to GWT, Browser-APIs (Web-RTC),…

On the bottom line, it was an inspiring talk about how use existing code to compose a new webapp.

Things that happen when you’re at a conference: When Alberto was talking about finding a workaround to avoid reflection when adapting boofcv to GWT, Nelson James (he gave a talk at gwtcreate as well), who was in the audience, mentioned his recently published GWT refelection library.

Here’s some computer vision fun: http://picshare.jooink.com

Nelson James’s reflection lib: https://github.com/WeTheInternet/xapi/tree/master/gwt/gwt-reflect

Slides: https://docs.google.com/presentation/d/1pfSQkvSn_qttb4fjBWcm3RtubXRLKY9kB8jEK7YJLSE

Speed up your GWT coding with gQuery ( Manuel Carrasco Moñino )

Manuel Carrasco Moñino gave an overview of gQuery’s features and the slides itself where a sample of what you can do with GWT/gQuery. Just see the slides if you keep struggling with stuff like JSNI, RequestBuilder, JSON/XML-processing, FileUpload or nested callbacks. Or if you just like swiss army knifes 🙂 If you don’t want to traverse DOM at runtime, you’re able to define your selectors so work’s done at compile time.

I’m going to apply some of this to my GWT code definitely!

Getting Started: https://code.google.com/p/gwtquery/wiki/GettingStarted

Manuel Carrasco Moñino’s ‘slides’: http://gwtquery.github.io/gwt.create-slides/gwtcreate2015/gwtcreate2015.html#0

Cell Widgets in the Wild ( Chris DiGiano )

This was like giving best practices for using Cell Widgets. Those are used by the Google Groups guys and they had to deal especially with some situations you get in when having a lot of data to visualize by lists or tables and some usability issues.

Avoid string building for more maintainable rendering code
Filling and pre-fetching for flexible, performant scrolling
Custom key handling for the control users expext
Composition utilities for concise Composite Cells

Slides: https://docs.google.com/file/d/0B97Mv38I4V6VLVV1VW5pT0Zodms

Vaadin with Polymer, GWT and AngularJS ( Leif Åstrand & Joonas Lehtinen )

First part of the talk was about AngularJS. I didn’t get why AngularJS was mentioned in that talk, but they came up with an interesting analogy:

AngularJS is for webdesigner what Lotus Notes was for business people.

Lotus enabled the business man to juggle with numbers in a new way, so they where able to write small financial apps on their own. AngularJS gives the webdeveloper more power, because they can build prototypes just by writing HTML (with interactive UI due to the 2-way-binding).

The second part of the talk was about how Vaadin is going to face the upcoming Webcomponents. They’ve got a rich components library tied to Vaadin framework. Webcomponents don’t fit into the existing framework. Thus, Vaadin’s going to port their components so they can be shipped as Webcomponents.

This leads to the question how one builds Webcomponents/Custom Elements at all, especially if you’ve got powerfull widgets like Vaadin’s Grid-Component. They came up with a method that gives you a Panel which is in fact a part of Webcomponent’s shadow DOM. Having that Panel, you just put on it every widget you want. They didn’t tell us how this method works in detail unfortunately. But this looked like a good attempt to rescue existing widgets over to Webcomponents. Perhaps that might be not the best way to get the maximum out of Webcomponents , but this could work to migrate in a silent way.

Slides: https://docs.google.com/file/d/0ByS1wxINeBWjcWxYSHVaTW5BSmc

Panel ( GWT Steering committee )

Like in 2013, the final point on the agenda was the panel. This is where you can ask the steering committee whatever you want about GWT. For me, there where two major points being discussed: Is GWT dead/alive and Widgets/Webcomponents.

In my opinion, GWT is alive, no doubt. Google uses it for prominent and complex products and there’s more to come (that was one of the big arguments pro GWT by the steering comittee). Furthermore, the community grows (see Ray Cromwell’s Keynote) continuously. And the most important thing for the future: Improving GWT keeps moving, the engineers don’t rest on their laurels. I think with JSInterop, Elemental 2.0 and Singular, there’s everything you’ll need when Webcomponents gather momentum. Improvements on SuperDevMode and new approaches like Singular will increase productivity. 3rd party does a good job as well (e.g. Vaadin’s components, Redhat’s Errai, Arcbees GWTP or gQuery).

Some guys complained about the project website being out of date in some points, which gives the impression GWT is dead. That put’s one in a bad position while trying to establish GWT in a company’s new project. So some shoe shining might help. Steering committee requested that people who find obsolete parts might send in a patch to remove/fix that. They want to focus their work on the compiler as much as they can (sounds very reasonable to me).

The other major point was about Widgets and Webcomponents. Daniel Kurka said

Nobody will use Widgets anymore in a couple of years.

For new projects that might be. But I’m pretty sure for a lot of people it’ll be important to be able to use their existing code further. Ray Cromwell added

Widgets will be supported in future, so old stuff will compile. New stuff should be done with Webcomponents.

I think everybody can live with that. For me, it’s just the question of migration. Is it a good practice to wrap old stuff into a shadow DOM (like Vaadin does)? Julien Dramaix gave a talk covering best practices and one of his points was to avoid widgets because they are heavy. They generate something Chris DiGiano called “DIV-soup”. But will that be noticeably better using Webcomponents? The DIVs are hidden in the shadow DOM, for sure. But complex Widgets will become complex Webcomponents. There was another argument against using widgets. Back in 2006, widgets had the job to handle the several browser’s different behavior. These days, browsers act pretty much the same (true, if you don’t have to support IE8/IE9). So this widgets bring some extra complexity where it’s not needed anymore.

Nevertheless, I’m looking forward using Webcomponents. And otherwise than in 2013, using Webcomponents is close at hand thanks to Polymer/Polyfills. When people will start using it, I guess best practices to migrate will come up soon. At the end, new approaches like Singular only make sense when Webcomponents/Custom Elements can be used.

See you there next year (if my boss pays 😉 )!

Strange behavior of Firefox when it comes to range requests of JavaScript files

6. January 2015Michael Lösch Leave a comment

At work we have a GWT application which contains a large ( > 10mb) javascript file. In some cases that file is not executed by the browser. It seems like the application is frozen. A colleague and I observed that pretty strange behavior while using Firefox (didn’t recognize this in IE, Crome, Safari,…):

The js file is cached by the browser usually. In some cases the file is not cached completely. The strange thing happens when we want to reload the application. Due to the incomplete caching of that file, Firefox requests the missing part of the file (range request). But that ends up with the frozen-like behavior.

So we tried to isolate the problem especially without using GWT. In the end we found a way to reproduce the problem:

1. The HTML/Javascript file

We created a HTML file that loads large js-Files and reports their state of execution. The javascript method add will be called by the large js-Files (see later on).

ct.html:

<!DOCTYPE html>
<html>
<head>
    <script type="text/javascript">
      var count = 11;
      function initTable() {
        var p = document.getElementById('table');
        var t = document.createElement('table');
        t.setAttribute('border','1');
        for (i = 0; i <= count; i++) {
          var tr = document.createElement('tr');
          tr.setAttribute('id', 'tr-' + i);
          var td = document.createElement('td');
          td.appendChild(document.createTextNode(i));
          tr.appendChild(td);
          t.appendChild(tr);
        }
        p.appendChild(t);
      }
      function add (idx, text) {
        var td = document.createElement("td");
        td.appendChild(document.createTextNode(text));
        document.getElementById('tr-' + idx).appendChild(td);
      }
      function log (text) {
        var e = document.createElement("div");
        e.innerHTML = text;
        document.getElementById('log').appendChild(e);
      }
    </script>
</head>
<body>
  <div id="table"></div>
  <div id="log"></div>
  <script type="text/javascript">
    initTable();
    log(new Date());
    for (i = 0; i <= count; i++) {
      add(i, 'add script');
      var s = document.createElement('script');
      s.setAttribute('src', 'big' + i + '.cache.js');
      s.setAttribute('type', 'text/javascript');
      document.getElementsByTagName('head')[0].appendChild(s);
    }
  </script>
</body>
</html>

<!DOCTYPE html>

<html>

<head>

var count = 11;

function initTable() {

var p = document.getElementById('table');

var t = document.createElement('table');

t.setAttribute('border','1');

for (i = 0; i <= count; i++) {

var tr = document.createElement('tr');

tr.setAttribute('id', 'tr-' + i);

var td = document.createElement('td');

td.appendChild(document.createTextNode(i));

tr.appendChild(td);

t.appendChild(tr);

}

p.appendChild(t);

}

function add (idx, text) {

var td = document.createElement("td");

td.appendChild(document.createTextNode(text));

document.getElementById('tr-' + idx).appendChild(td);

}

function log (text) {

var e = document.createElement("div");

e.innerHTML = text;

document.getElementById('log').appendChild(e);

}

</script>

</head>

<body>

initTable();

log(new Date());

for (i = 0; i <= count; i++) {

add(i, 'add script');

var s = document.createElement('script');

s.setAttribute('src', 'big' + i + '.cache.js');

s.setAttribute('type', 'text/javascript');

document.getElementsByTagName('head')[0].appendChild(s);

}

</script>

</body>

</html>

2. Generating large js files

We wrote a Groovy script that generates files with three lines (named big${i}.cache.js)

The first line contains add calling the add method with the argument “start”.
The second line contains ~30mb spaces
The third line again contains code calling the add method with the argument “end”.

def size = 31450000
for (i in 0..12) {
    def f = new File("/Users/user/apache/htdocs/big${i}.cache.js")
    f.delete()
    f << "add($i, 'start');\n" << " ".multiply(size) << "\nadd($i, 'end');"
}

def size = 31450000

for (i in 0..12) {

def f = new File("/Users/user/apache/htdocs/big${i}.cache.js")

f.delete()

f << "add($i, 'start');\n" << " ".multiply(size) << "\nadd($i, 'end');"

}

3. Bring it together

Copy ct.html and the big.cache.js files into your webservers htdocs.

Clear your browser cache

Navigate to ct.html and wait till the table looks like this (all files are loaded and executed):

0	add script	start	end
1	add script	start	end
2	add script	start	end
3	add script	start	end
4	add script	start	end
5	add script	start	end
6	add script	start	end
7	add script	start	end
8	add script	start	end
9	add script	start	end
10	add script	start	end
11	add script	start	end

Then reload the page.

What you should see is that some scripts are executed (you see start/end columns) and some are not. You can see the not executed ones in Firebugs network sheet. Those are the ones with status “206 Partial Content”:

We reported that problem to Mozilla but got no response yet ( https://bugzilla.mozilla.org/show_bug.cgi?id=1117763 ).

But if you got the solution already, feel free to post it in the comment section 😎

My environment:

Firefox 34.0.5 on a Mac running OsX 10.10.1 (Yosemite)

Read CGLog with Groovy

11. December 2014Michael Lösch Leave a comment

If an app uses Jemalloc to manage space, there’s a security gap if you run Yosemity. Famouse apps using Jemalloc are e.g. Firefox and Thunderbird. So if you want to know you’re affected or not, just take a look at the /tmp dir and watch out for files like CGLog_Thunderbird* or CGLog_Firefox*.

I’m affacted and I wanted to know what mail content was logged. So here’s an example how sweet and easy groovy can handle jobs like that, including some code that cleans up the un-readable chars.

def f = new File("/Users/username/tmp/CGLog.txt")
new File("/tmp/CGLog_Thunderbird_123456").readLines().each { String line ->
    line.eachMatch(~/KeyDown.*char ([0-9]*);/) {
        f << String.valueOf((char) (it[1] as Integer)).replaceAll("[\uF700-\uF72F]", "").replaceAll('\r','\n')
    }
}

def f = new File("/Users/username/tmp/CGLog.txt")

new File("/tmp/CGLog_Thunderbird_123456").readLines().each { String line ->

line.eachMatch(~/KeyDown.*char ([0-9]*);/) {

f << String.valueOf((char) (it[1] as Integer)).replaceAll("[\uF700-\uF72F]", "").replaceAll('\r','\n')

}

Hackserei

It's about software

Widgets

Search

Make Groovy’s Grape.grab work when running compiled groovy code with java

Enhance your Groovy DSL in a plugin-ish way

1. Define a marker interface to find the plugin class (you can do this with annotations as well)

2. A class that enhances your DSL

3. The actual DSL-backing class

4. Writing some DSL-code

That’s it

Groovy DSL: How to split up execution of method calls like gradle does

First variation:

Second variation:

GWT.create 2015

First day

Keynote GWT 3 ( Ray Cromwell )

Singular ( Daniel Kurka )

Polymer & Elements ( Chris DiGiano )

What’s new with SuperDevMode ( Brian Slesinsky )

Deep Dive in JS Interop ( Ray Cromwell )

Google Inbox: Multi Platform Native Apps with GWT and J2ObjC ( Ray Cromwell )

J2ObjC: Sharing Java Code with iOS Applications ( Tom Ball )

Second day

Ecosystem Keynote ( Hosted by Joonas Lehtinen )

DIY: Computer Vision with GWT ( Alberto Mancini & Francesca Tosi )

Speed up your GWT coding with gQuery ( Manuel Carrasco Moñino )

Cell Widgets in the Wild ( Chris DiGiano )

Vaadin with Polymer, GWT and AngularJS ( Leif Åstrand & Joonas Lehtinen )

Panel ( GWT Steering committee )

Strange behavior of Firefox when it comes to range requests of JavaScript files

1. The HTML/Javascript file

2. Generating large js files

3. Bring it together

Read CGLog with Groovy