Archive for the ‘gwt’ Category

GWT 2.2 and java.lang.IncompatibleClassChangeError

Thursday, March 3rd, 2011

I’ve been updating Gri.pe to the latest versions of the various libraries we use and ran into some trouble while attempting to update GWT to 2.2. There were some incompatible bytecode changes made: the classes in the com.google.gwt.core.ext.typeinfo package were converted to interfaces. Java has a special invoke opcode for interface types, so the classes would fail to load with this obscure error when the verifier tried to load the classes:

java.lang.IncompatibleClassChangeError: Found interface com.google.gwt.core.ext.typeinfo.JClassType, but class was expected

There are updated libraries available for some third-party GWT libraries, but others (like the gwt-google-apis) haven’t been updated yet.

The solution suggested by others was to manually recompile these against the latest GWT libraries to fix the problem. I thought I’d try a different approach: bytecode rewriting. The theory is pretty simple. Scan the opcodes of every method in a jar and replace the opcode used to invoke a method on a class, INVOKEVIRTUAL, with the opcode for invoking a method on interfaces, INVOKEINTERFACE. More info on Java opcodes is available here.

The ASM library makes this sort of transformation trivial to write. Compile the following code along with the asm-all-3.3.1.jar and plug in the file you want to transform in the main method. It’ll spit out a version of the library that should be compatible with GWT 2.2. You might need to add extra classes to the method rewriter, depending on the library you are trying to rewrite:

package com.gripeapp.bytecoderewrite;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;

import org.objectweb.asm.ClassAdapter;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.ClassWriter;
import org.objectweb.asm.MethodAdapter;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;

public class BytecodeRewrite {
  static class ClassVisitorImplementation extends ClassAdapter {
    public ClassVisitorImplementation(ClassVisitor cv) {
      super(cv);
    }

    @Override
    public MethodVisitor visitMethod(int access, String name, String desc,
              String signature, String[] exceptions) {
        MethodVisitor mv;
        mv = cv.visitMethod(access, name, desc, signature, exceptions);
        if (mv != null) {
          mv = new MethodVisitorImplementation(mv);
        }
        return mv;
    }
  }

  static class MethodVisitorImplementation extends MethodAdapter {

    public MethodVisitorImplementation(MethodVisitor mv) {
      super(mv);
    }

    @Override
    public void visitMethodInsn(int opcode, String owner, String name,
        String desc) {
      if (owner.equals("com/google/gwt/core/ext/typeinfo/JClassType")
          || owner.equals("com/google/gwt/core/ext/typeinfo/JPackage")
          || owner.equals("com/google/gwt/core/ext/typeinfo/JMethod")
          || owner.equals("com/google/gwt/core/ext/typeinfo/JType")
          || owner.equals("com/google/gwt/core/ext/typeinfo/JParameterizedType")
          || owner.equals("com/google/gwt/core/ext/typeinfo/JParameter")) {
        super.visitMethodInsn(Opcodes.INVOKEINTERFACE, owner, name, desc);
      } else
        super.visitMethodInsn(opcode, owner, name, desc);
    }
  }

  public static void main(String[] args) throws IOException {
    ZipInputStream file = new ZipInputStream(new FileInputStream("/tmp/gwt-maps-1.1.0/gwt-maps.jar"));
    ZipEntry ze;

    ZipOutputStream output = new ZipOutputStream(new FileOutputStream(("/tmp/gwt-maps.jar")));

    while ((ze = file.getNextEntry()) != null) {
      output.putNextEntry(new ZipEntry(ze.getName()));
      if (ze.getName().endsWith(".class")) {
        ClassReader reader = new ClassReader(file);
        ClassWriter cw = new ClassWriter(0);
        reader.accept(new ClassVisitorImplementation(cw), ClassReader.SKIP_FRAMES);
        output.write(cw.toByteArray());
      } else {
        byte[] data = new byte[16*1024];
        while (true) {
          int r = file.read(data);
          if (r == -1)
            break;
          output.write(data, 0, r);
        }
      }
    }

    output.flush();
    output.close();
  }
}

GWT 2.0 on OSX and the ExternalBrowser RunStyle

Saturday, December 12th, 2009

GWT 2.0 ships with a new HtmlUnit-based testing system that makes testing faster, but isn’t full web-browser under the hood. It can’t perform layout, so any tests that rely on layout measurements won’t succeed.

There are a number of alternate run styles that are useful, however. Manual mode, which outputs a URL that you can connect any browser to, works well for quick testing, but is a pain when you need to keep running it over and over. RemoteWeb and Selenium automate the process of starting and stopping browsers, but require you to start an external server before testing.

There’s another run-style that isn’t well-documented, but I’ve found to be the most useful for testing locally: ExternalBrowser. It requires an executable name in the command-line, the name of the browser to start. On OSX, you can specify the fully-qualified name of the executable beneath the application bundle, or you can use a shell script to launch the browser of your choice.

Place the following script under /usr/bin named ‘safari’ and make it chmod a+x. This allows you (and ExternalBrowser) to launch Safari from the command-line’s PATH. The argument to “open” is the name of any browser that lives under your /Applications/ directory, including subdirectories.

#!/bin/sh
open -W -a Safari $1

Add a Java system property “gwt.args” to your testing launch configuration in Eclipse, Ant script or other IDE. You can then specify the run style like so:

-Dgwt.args="-runStyle ExternalBrowser:safari"

Now, when you run your unit tests, you’ll see GWT launch Safari and navigate to the appropriate URL on its own. Tests will leave windows up on your screen after they complete, but you can safely close them after the run terminates.

My Presentation at Google Campfire One

Saturday, December 12th, 2009

I went down to Mountain View last week to show off the company we’ve been building, DotSpots, plug our brand-new Chrome extension and demo some of the great new features of GWT 2.0.

See it directly on YouTube here: http://www.youtube.com/watch?v=JQpuDB2Jxfg.

Kudos to the Google team for putting together such a great event. You don’t realize how much planning and preparation goes into an hour-long event like that.

Fix for GWT Hosted mode crash with Safari 4.0.4

Monday, November 16th, 2009

My local tests started failing soon after upgrading my machine to Safari 4.0.4. Some research and pointers from the GWT folk pointed me at the root cause, a WebKit bug which is now fixed.

Kelly Norton pointed me at a quick fix:

  1. Download the latest WebKit nightly from here
  2. Save it locally
  3. Add the following environment variable to your testing .launch targets (using the path to your new WebKit.app, of course). Tests run from Ant or the command line will need to use the appropriate tasks or shell commands:
    Name: DYLD_FRAMEWORK_PATH
    Value: /Applications/path-to-your-local-webkit/WebKit.app/Contents/Frameworks/10.5
    
  4. Snow Leopard users will need to use 10.6 in the path above.

The proper fix should arrive in WebKit 4.0.5 at some point, but this will keep you running for now.

Jim Douglas has posted more detailed instructions on how to configure your launch targets on the GWT issue here: Issue #4220: GWT crash (Safari 4.0.4)

A quieter window.name transport for IE

Tuesday, July 28th, 2009

Using window.name as a transport for cooperative cross-domain communication is a reasonably well-known and well-researched technique. I came across it via two blog posts by members of the GWT community that were using it to submit GWT FormPanels to endpoints on other domains.

For our product, I’ve been looking at various ways we can offer RPC for our script when it is embedded in pages that don’t run on servers under our control.  Modern browsers, like Firefox 3.5 and Safari 4.0 support XMLHttpRequest Level 2.  This standard allows you to make cross-domain calls, as long as the server responds with the appropriate Access-Control header.  Internet Explorer 8 supports a proprietary XDomainRequest that offers similar support.

When we’re looking at “downlevel” browsers, like Firefox 2/3, Safari 2/3 and IE 6/7, the picture isn’t as clear. The window.name transport works well in every downlevel browser but IE6 and 7. In those IE versions, each RPC request made across the iframe is accompanied by an annoying click sound. As you can imagine, a page that has a few RPC requests that it requires to load will end up sounding like a machine gun. The reason for this is IE’s navigation sound which plays on every location change for any window, including iframes. The window.name transport requires a POST and a redirect back to complete the communication, triggering this audio UI.

I spent a few hours hammering away on the problem, trying to find a solution. It turns out that IE6 can be fooled with a <bgsound> element that masks the clicking sound. This doesn’t work in IE7, however. My research then lead to an interesting discovery: the GTalk team was using an ActiveX object named “htmlfile” to work around a similar problem: navigation sounds that would play during their COMET requests. The htmlfile object is basically a UI-disconnected HTML document that works, for the most part, the same way as a browser document. The question was now how to use this for a cross-domain request.

The interesting thing about the htmlfile ActiveX object is that not all HTML works as you’d expect it to. My first attempt was to use the htmlfile object, creating an iframe element with it, attaching it to the body (along with an accompanying form inside the htmlfile document) and POSTing the data. Unfortunately, I couldn’t get any of the events to register. The POST was happening, but none of the iframe’s onload events were firing:

if ("ActiveXObject" in window) {
    var doc = new ActiveXObject("htmlfile");
    doc.open();
    doc.write("<html><body></body></html>");
    doc.close();
} else {
    var doc = document;
}

var iframe = doc.createElement('iframe');
doc.body.appendChild(iframe);
iframe.onload = ...
iframe.onreadystatechange = ...

The second attempt was more fruitful. I tried writing the iframe out as part of the document, getting the iframe from the htmlfile and adding event handlers to this object. Success!  I managed to capture the onload event, read back the window.name value and, best of all, the browser did this silently:

if ("ActiveXObject" in window) {
    var doc = new ActiveXObject("htmlfile");
    doc.open();
    doc.write("<html><body><iframe id='iframe'></iframe></body></html>");
    doc.close();
    var iframe = doc.getElementById('iframe');
} else {
    var doc = document;
    var iframe = doc.createElement('iframe');
    doc.body.appendChild(iframe);
}

iframe.onload = ...
iframe.onreadystatechange = ...

I’m currently working on cleaning up the ugly proof-of-concept code to integrate as a transport in the Thrift-GWT RPC library I’m working on. This will allow us to transparently switch to the cross-domain transport when running offsite, without any visible side-effects to the user.

A taste of Firefox Extensions, written in GWT

Monday, July 13th, 2009

UPDATE: It’s live! The open-source project is up on Google Code and I’ve blogged a more about it.

I’m getting closer to having the GWT bindings that we wrote for Firefox ready for public release. What we’ve got is more than enough to write a complex extension. The bindings were even enough to write a prototype of an OOPHM server, itself written in GWT!

For now, just a taste of what extension development is like GWT, complete with strong typing, syntax checks, auto-completion and *hosted mode support*:

protected nsIFile createTempFile() {
    nsIFile file = nsIProperties.getService("@mozilla.org/file/directory_service;1")
        .get("TmpD", nsIFile.iid());
    file.append("logs");
    if (!file.exists()) {
        file.create(nsIFile.DIRECTORY_TYPE, 0777);
    }

    file.append("log.txt");
    file.createUnique(nsIFile.NORMAL_FILE_TYPE, 0666);

    return file;
};

protected void write(String value, nsIFile file) {
    nsIFileOutputStream foStream = nsIFileOutputStream.createInstance("@mozilla.org/network/file-output-stream;1");
    foStream.init(file, 0x02 | 0x08 | 0x10, 0666, 0);
    foStream.write(value, value.length());
    foStream.close();
};

The bindings are all generated from the xulrunner SDK’s IDL files and include documentation, parameter names and constants:

  /**
     * @param file          - file to write to (must QI to nsILocalFile)
     * @param ioFlags       - file open flags listed in prio.h
     * @param perm          - file mode bits listed in prio.h
     * @param behaviorFlags flags specifying various behaviors of the class
     *        (currently none supported)
     */
  public final native void init(nsIFile file, int ioFlags, int perm, int behaviorFlags) /*-{
    return this.init(file, ioFlags, perm, behaviorFlags);
  }-*/;

The final word on Google Eclipse plugin OSX crashes

Sunday, April 19th, 2009

I’ve blogged about the subject of GWT JVM crashes far too much (here and here).  This is, I hope, the final word on the subject.  I spent some time disassembling the Google Eclipse plugin (did I agree not to do that in one of the EULAs? ;) ) and discovered that they are launching the JVM without using the Eclipse infrastructure.  This means that the JVM arguments are effectively hardcoded, except in one case where -startOnFirstThread is passed for OSX clients.  For those interested, the buggy classes of note are GWTDeploymentParticipant and LaunchUtils.

I posted a note on the contributor’s list and received a response from a Googler suggesting that I wrap my Java executable in a shell script.  Not one to shy away from wrapping system executables with scripts, I decided to give it a shot tonight.  My first shell script attempt was a disaster (dear lazyweb: how can you get bash to respect quoted spaces when using $@?), so I gave it a whack in Python.

This is tested and somewhat guaranteed to work for JVM 1.6 under OSX:

cd /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin
sudo mv java java_wrapped
sudo nano java

And the script:

#!/usr/bin/env python
import sys
import os

print sys.argv
cmd = os.path.dirname(sys.argv[0]) + '/java_wrapped'

args = ['',
        '-XX:CompileCommand=exclude,org/eclipse/core/internal/dtree/DataTreeNode,forwardDeltaWith',
        '-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding,<init>',
        '-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedMethodBinding,<init>']

args.extend(sys.argv[1:])

print cmd
print args
print ""

os.execv(cmd, args)

If you have any suggestions on how to improve my Python, I’d be glad to hear them in the comments.

UPDATE: Thanks to Vitali, I got a bash script working as well:

#!/bin/bash
`dirname $0`/java_wrapped \
        "-XX:CompileCommand=exclude,org/eclipse/core/internal/dtree/DataTreeNode,forwardDeltaWith" \
        "-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding,<init>" \
        "-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedMethodBinding,<init>" \
        "$@"

AppEngine+Java+OSX “Invalid memory access” on deploy

Friday, April 17th, 2009

UPDATE, Apr 19 2009: This does not fix all compiler crashes in Eclipse.  This is only useful for the compiler crashes where an Eclipse JVM configuration is used.  Please see my later blog post for a better fix and disregard this information.

This is the same error I hit before (see my earlier post), but it happens in Eclipse during the “deploy to AppEngine” phase.  You’ll see it if you’ve got a project configured with a 1.6 JVM on the build path (even if your .class compatibility is set to 1.5).  It’s not obvious how to fix this at first glance – I had to dig around to find the options that Eclipse passes to the JVM.

The solution

You can fix it by adding the same compiler workarounds we explored before to your JVM default arguments.  IMPORTANT NOTE: Add the args to all of your 1.6 level installed JVMs.  I’ve found that Eclipse doesn’t always choose the selected one if there’s an equivalent also installed (ie: it may use your ’1.6′ JVM if you have ’1.6.0′ selected).

-XX:CompileCommand=exclude,org/eclipse/core/internal/dtree/DataTreeNode,forwardDeltaWith -XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding,<init> -XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedMethodBinding,<init>

Add them to your JVMs under Preferences > Java > Installed JREs by clicking the edit button and filling out “Default VM Arguments”:

jvm-args-to-fix-crash

An example of where to place the arguments

Google eats GWT dogfood

Tuesday, April 14th, 2009

Google’s new profile editor uses GWT, cool!

Looks like they aren’t obfuscating class names yet (a feature that just landed on the tip of trunk, IIRC):

http://www.google.com/s2/gwt/resources/9312606AE1FEAAD063F34A9446584258.cache.js

The name of the project is “com.google.focus” and there are references to codenames: “evergreen” (contact info DB?) and “publicusername”.  They are also pulling some of the code from Google Collections into the final output.

GWT 1.6 crashes (and a fix)

Tuesday, April 14th, 2009

There’s a bug in the latest Java 1.6  that Apple provides that gets tickled by the GWT 1.6 compiler.  It manifests as a reproducible HotSpot compiler crash that looks like this:

    [java] Invalid memory access of location 00000000 rip=01160767

When you look into the crash logs that Java produces (found under ~/Library/Logs/CrashReporter/java_*), you can see that HotSpot was compiling a method when it crashed:

 Java VM: Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57 mixed mode macosx-amd64)
Current thread (0x0000000101843800):  JavaThread "CompilerThread1" daemon [_thread_in_native, id=1756672000, stack(0x0000000168a4b000,0x0000000168b4b000)]
Stack: [0x0000000168a4b000,0x0000000168b4b000]
Current CompileTask:
C2:637      org.eclipse.jdt.internal.compiler.lookup.ParameterizedMethodBinding.<init>(Lorg/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding;Lorg/eclipse/jdt/internal/compiler/lookup/MethodBinding;)V (596 bytes)

The workaround is to disable JIT of some JDT methods using some advanced JVM command-line arguments.  Using Ant, you can tack them on to your GWT compiler task <java> blocks, like so:

<java classname="com.google.gwt.dev.Compiler" classpathref="compileClassPath@{module}" fork="true" failonerror="true">
    ...
    <jvmarg value="-XX:CompileCommand=exclude,org/eclipse/core/internal/dtree/DataTreeNode,forwardDeltaWith" />
    <jvmarg value="-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding,&lt;init&gt;" />
    <jvmarg value="-XX:CompileCommand=exclude,org/eclipse/jdt/internal/compiler/lookup/ParameterizedMethodBinding,&lt;init&gt;" />

By applying those commands, you ask the HotSpot compiler to skip those methods (relying on the interpreter to run them).  You’ll know it’s working if you see this during your compile:

[java] CompilerOracle: exclude org/eclipse/core/internal/dtree/DataTreeNode.forwardDeltaWith
[java] CompilerOracle: exclude org/eclipse/jdt/internal/compiler/lookup/ParameterizedTypeBinding.<init>
[java] CompilerOracle: exclude org/eclipse/jdt/internal/compiler/lookup/ParameterizedMethodBinding.<init>
[java] Compiling module com.dotspots.ExtensionModule

I’m hoping that Apple eventually gets around to releasing an updated Java 1.6 for OSX.  This bug was fixed in mainline Java a very long time ago!