Technology excruciation

Using built-in JavaCompiler with a custom classloader

2009-10-25T06:02:00.000-07:00

Java 6 has a great feature built into the JVM (when it comes with JDK apparently): a programmatical access to javac functionality. There's a great article on developer works that helped me a lot when I was implementing a feature in my application that was using this functionality. In fact I just took the code from that article, applied it to my project and everything worked like magic when I was running locally from my IDE =)

So it was fine until I deployed the app to a real-life server. There the code stopped working completely. Which was not really a big surprise to me but quite frustrating. Looking into details I was able to find out the reason. The application utilising this code was loaded on the server by a custom classloader. I.e. the jvm was started with a single jar in classpath that was capable of loading different versions of the application in separate classloaders (don't ask me for reasons pls). So the compiler was failing unable to resolve dependencies.

Ok, let's do a simple example.
Imagine that this loading jar is called loader.jar and the application itself is contained in a jar app.jar. Among others there is an interface ru.atamur.Compiled there. And finally I'm trying to compile a source of:


package ru.atamur.compiled;

import ru.atamur.*;

public class Generated implements Compiled {
  @Overrides public int sum(int a, int b) { return a+b; }
}

Apparently I need the interface to access my compiled methods, so I have this dependency on classes inside the app.jar.

So why was it running locally?
if you check out the code from the article it contains this line:


JavaFileManager fileManager 
  = compiler.getStandardFileManager(diagnostics, null, null);

Basically JavaFileManager is something that allows javac to ask for classes it can't find in the source code - the dependencies.
compiler.getStandarfFileManager matching its name returns you a standard manager. Probably the one that's used by command line javac. The runtime version uses the jvm classpath to resolve all the FQNs encountered. When I was running locally - my whole app (the app.jar) was on my classpath, so this standard file manager was doing just fine, but when running on the server the only jar on the classpath was loader.jar that didn't contain ru.atamur.Compiled causing a compile error.

This might seem a bit artificial but the same problem will occur in any user code that's run inside a container (app server, web server, ioc container etc).

The most irritating detail was: all the classes the compiler needs are here in current thread's classloader, but there's no way to tell it about this fact!

Browsing google I wasn't able to find a proper solution so here's what I finally came up with: use my very own implementation of JavaFileManager that resolves the user class dependencies using a given classloader and resolves system dependencies (like java.lang) using standard file manager.

The solution might be not enough for every possible application but was just fine for my app.

First we modify that line from CharSequenceCompiler seen above:


    StandardJavaFileManager standardJavaFileManager 
      = compiler.getStandardFileManager(diagnostics, null, null);
    final JavaFileManager fileManager
      = new CustomClassloaderJavaFileManager(loader, standardJavaFileManager);

So this introduces a new class called CustomClassloaderJavaFileManager which takes as a parameter a classloader that knows where application classes reside and the standard file manager that can resolve dependencies to standard java classes.

Here is the definition of CustomClassloaderJavaFileManager:


package ru.atamur.compilation;

import java.io.*;
import java.util.*;
import javax.tools.*;

/**
 * @author atamur
 * @since 15-Oct-2009
 */
public class CustomClassloaderJavaFileManager implements JavaFileManager {
 private final ClassLoader classLoader;
 private final StandardJavaFileManager standardFileManager;
 private final PackageInternalsFinder finder;

 public CustomClassloaderJavaFileManager(ClassLoader classLoader, StandardJavaFileManager standardFileManager) {
  this.classLoader = classLoader;
  this.standardFileManager = standardFileManager;
  finder = new PackageInternalsFinder(classLoader);
 }

 @Override
 public ClassLoader getClassLoader(Location location) {
  return classLoader;
 }

 @Override
 public String inferBinaryName(Location location, JavaFileObject file) {
  if (file instanceof CustomJavaFileObject) {
   return ((CustomJavaFileObject) file).binaryName();
  } else { // if it's not CustomJavaFileObject, then it's coming from standard file manager - let it handle the file
   return standardFileManager.inferBinaryName(location, file);
  }
 }

 @Override
 public boolean isSameFile(FileObject a, FileObject b) {
  throw new UnsupportedOperationException();
 }

 @Override
 public boolean handleOption(String current, Iterator<String> remaining) {
  throw new UnsupportedOperationException();
 }

 @Override
 public boolean hasLocation(Location location) {
  return location == StandardLocation.CLASS_PATH || location == StandardLocation.PLATFORM_CLASS_PATH; // we don't care about source and other location types - not needed for compilation
 }

 @Override
 public JavaFileObject getJavaFileForInput(Location location, String className, JavaFileObject.Kind kind) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public JavaFileObject getJavaFileForOutput(Location location, String className, JavaFileObject.Kind kind, FileObject sibling) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public FileObject getFileForInput(Location location, String packageName, String relativeName) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public FileObject getFileForOutput(Location location, String packageName, String relativeName, FileObject sibling) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public void flush() throws IOException {
  // do nothing
 }

 @Override
 public void close() throws IOException {
  // do nothing
 }

 @Override
 public Iterable<JavaFileObject> list(Location location, String packageName, Set<JavaFileObject.Kind> kinds, boolean recurse) throws IOException {
  if (location == StandardLocation.PLATFORM_CLASS_PATH) { // let standard manager hanfle
   return standardFileManager.list(location, packageName, kinds, recurse);
  } else if (location == StandardLocation.CLASS_PATH && kinds.contains(JavaFileObject.Kind.CLASS)) {
   if (packageName.startsWith("java")) { // a hack to let standard manager handle locations like "java.lang" or "java.util". Prob would make sense to join results of standard manager with those of my finder here
    return standardFileManager.list(location, packageName, kinds, recurse);
   } else { // app specific classes are here
    return finder.find(packageName);
   }
  }
  return Collections.emptyList();

 }

 @Override
 public int isSupportedOption(String option) {
  return -1;
 }

}

So this class implements only a subset of functionality defined by JavaFileManager interface but that appears to be enough to handle the compilation (at least in my case). It also has some not-that-pretty code to handle both custom classloader classes and standard java classes.

There are 2 new classes used here PackageInternalsFinder and CustomJavaFileObject. FIrst one is a utility class that translates JavaFileManager requests to classloader requests and the second one is a custom implementation of JavaFileObject interface. JDK already contains an implementation of JavaFileObject: SimpleJavaFileObject based on URI that seemed suitable for my needs but it heavily depends on the URI having non null path which is not true for JAR URIs, so I had to develop my own version:


package ru.atamur.compilation;

import java.net.URI;
import java.io.*;
import javax.tools.JavaFileObject;
import javax.lang.model.element.NestingKind;
import javax.lang.model.element.Modifier;

/**
* @author atamur
* @since 15-Oct-2009
*/
class CustomJavaFileObject implements JavaFileObject {
 private final String binaryName;
 private final URI uri;
 private final String name;

 public CustomJavaFileObject(String binaryName, URI uri) {
  this.uri = uri;
  this.binaryName = binaryName;
  name = uri.getPath() == null ? uri.getSchemeSpecificPart() : uri.getPath(); // for FS based URI the path is not null, for JAR URI the scheme specific part is not null
 }

 @Override
 public URI toUri() {
  return uri;
 }

 @Override
 public InputStream openInputStream() throws IOException {
  return uri.toURL().openStream(); // easy way to handle any URI!
 }

 @Override
 public OutputStream openOutputStream() throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public String getName() {
  return name;
 }

 @Override
 public Reader openReader(boolean ignoreEncodingErrors) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public Writer openWriter() throws IOException {
  throw new UnsupportedOperationException();
 }

 @Override
 public long getLastModified() {
  return 0;
 }

 @Override
 public boolean delete() {
  throw new UnsupportedOperationException();
 }

 @Override
 public Kind getKind() {
  return Kind.CLASS;
 }

 @Override // copied from SImpleJavaFileManager
 public boolean isNameCompatible(String simpleName, Kind kind) {
  String baseName = simpleName + kind.extension;
  return kind.equals(getKind())
    && (baseName.equals(getName())
    || getName().endsWith("/" + baseName));
 }

 @Override
 public NestingKind getNestingKind() {
  throw new UnsupportedOperationException();
 }

 @Override
 public Modifier getAccessLevel() {
  throw new UnsupportedOperationException();
 }

 public String binaryName() {
  return binaryName;
 }


 @Override
 public String toString() {
  return "CustomJavaFileObject{" +
    "uri=" + uri +
    '}';
 }
}

again I didn't implement methods that are not absolutely required for doing compilation - and that's fine in my case.

Last bit to actually work out URIs for classes needed by the compiler:


package ru.atamur.compilation;

import java.util.List;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.Collection;
import java.util.jar.JarEntry;
import java.io.IOException;
import java.io.File;
import java.net.URL;
import java.net.JarURLConnection;
import java.net.URI;
import javax.tools.JavaFileObject;

/**
* @author atamur
* @since 15-Oct-2009
*/
class PackageInternalsFinder {
 private ClassLoader classLoader;
 private static final String CLASS_FILE_EXTENSION = ".class";

 public PackageInternalsFinder(ClassLoader classLoader) {
  this.classLoader = classLoader;
 }

 public List<JavaFileObject> find(String packageName) throws IOException {
  String javaPackageName = packageName.replaceAll("\\.", "/");

  List<JavaFileObject> result = new ArrayList<JavaFileObject>();

  Enumeration<URL> urlEnumeration = classLoader.getResources(javaPackageName);
  while (urlEnumeration.hasMoreElements()) { // one URL for each jar on the classpath that has the given package
   URL packageFolderURL = urlEnumeration.nextElement();
   result.addAll(listUnder(packageName, packageFolderURL));
  }

  return result;
 }

 private Collection<JavaFileObject> listUnder(String packageName, URL packageFolderURL) {
  File directory = new File(packageFolderURL.getFile());
  if (directory.isDirectory()) { // browse local .class files - useful for local execution
   return processDir(packageName, directory);
  } else { // browse a jar file
   return processJar(packageFolderURL);
  } // maybe there can be something else for more involved class loaders
 }

 private List<JavaFileObject> processJar(URL packageFolderURL) {
  List<JavaFileObject> result = new ArrayList<JavaFileObject>();
  try {
   String jarUri = packageFolderURL.toExternalForm().split("!")[0];

   JarURLConnection jarConn = (JarURLConnection) packageFolderURL.openConnection();
   String rootEntryName = jarConn.getEntryName();
   int rootEnd = rootEntryName.length()+1;

   Enumeration<JarEntry> entryEnum = jarConn.getJarFile().entries();
   while (entryEnum.hasMoreElements()) {
    JarEntry jarEntry = entryEnum.nextElement();
    String name = jarEntry.getName();
    if (name.startsWith(rootEntryName) && name.indexOf('/', rootEnd) == -1 && name.endsWith(CLASS_FILE_EXTENSION)) {
     URI uri = URI.create(jarUri + "!/" + name);
     String binaryName = name.replaceAll("/", ".");
     binaryName = binaryName.replaceAll(CLASS_FILE_EXTENSION + "$", "");

     result.add(new CustomJavaFileObject(binaryName, uri));
    }
   }
  } catch (Exception e) {
   throw new RuntimeException("Wasn't able to open " + packageFolderURL + " as a jar file", e);
  }
  return result;
 }

 private List<JavaFileObject> processDir(String packageName, File directory) {
  List<JavaFileObject> result = new ArrayList<JavaFileObject>();

  File[] childFiles = directory.listFiles();
  for (File childFile : childFiles) {
   if (childFile.isFile()) {
    // We only want the .class files.
    if (childFile.getName().endsWith(CLASS_FILE_EXTENSION)) {
     String binaryName = packageName + "." + childFile.getName();
     binaryName = binaryName.replaceAll(CLASS_FILE_EXTENSION + "$", "");

     result.add(new CustomJavaFileObject(binaryName, childFile.toURI()));
    }
   }
  }

  return result;
 }
}

This code is tested under both windows and linux OS and both from command line and IDE.

Hope someone will find this in time of need =)

2008-12-16T11:32:00.000-08:00

Just a nice image of my fiancee feeding ostriches ...

Seam 2.1.0GA and JBoss 5.0.0CR2

2008-11-01T00:32:00.001-07:00

As seam and jboss are developed by the same team, I would expect them to be compatible. But in reality they are not =)

So if you download the versions I mentioned in the subject and try to generate a default project by seam-gen (including db access) then there wil be a surprise for you. Deploying this project to jboss won't actually work. When it wants to connect to the database it will suddenly go:

 EntityManagerFactory not found in JNDI

Googling that string was quite painful, but after couple of hours I finally got a result!
Someone commented out something from jboss codebase and seam doesn't know about it =)

So if you're looking for a solution, you should be using another way of binding your entity manager, not jndi. To do that in j2ee enviroment (which i suppose you would have running jboss) you need to create a small helper class that will translate the entitymanager from jboss world to seam world:


@Stateless
@Name("entityManagerFactory")
public class EntityManagerFactoryHackBean implements EntityManagerFactoryHackLocal {

 @PersistenceUnit(name="main")
 EntityManagerFactory factory;
 
 @Unwrap
 public EntityManagerFactory getEntityMangagerFactory()
 {
  return factory;
 }
 
}

see, having both stateless and name annotations it serves like a bridge between two worlds.

then you have to use it in your components.xml instead of jndi name:

<persistence:managed-persistence-context name="entityManager"
  entity-manager-factory="#{entityManagerFactory}"
  scope="conversation" auto-create="true" />

Thanks for this thread on jboss forum

There is another thread which has a solution without ugly hacks, ~~but it doesn't work in j2ee env~~ and a small amendment that should make it work even in j2ee env. Btw this second thread tells us that this should be fixed in a consequent release of jboss.

Display a message when GWT app is loading

2008-10-04T01:59:00.000-07:00

Quick search through the internet revealed a way to achieve the subject:
developerlife and google groups

If you're to lazy to read those, here is an extremely short tutorial:

1. Place this div on your html (tip: you can generate your own loading gif here):

<div id="initialLoading">
    <img src="images/ajax-loader.gif" alt="loading ..." />
</div>

2. In your app's entry point add this line to remove the div when the app is actually loaded:

DOM.removeChild(RootPanel.getBodyElement(), DOM.getElementById("initialLoading"));

that's all

Adding custom styles to odd/even rows in a GWT table

2008-09-28T09:27:00.001-07:00

For the sake of better UI we decided to implement a table like gmail one that has differnt colors for even/odd rows. This seems to be quite a simple task, but I had to try several approaches before I managed to implement it.

1. Use custom RowFormatter

RowFormatter interface defines a getElement method which is supposed to be called for every row. I believed that if you override it you can attach special style to a row object based on it's number.

Wrong guess =)

The specific table implementation - FixedWidthGrid - I was using from the gwt incubator project is using some custom/native logic inside prepareCell method so it doesn't call the getElement method =(

2. Directly override prepareCell method - doesn't seem correct, cuz would be called much more often then you need it

3. Finally I got the idea that I'm not able to find any OO way of introducing this functionality and went with a brute force approach: implement a method that applies some special formatting to all the rows and call it every time the table content is updated.

Now I faced a new problem: there is no explicit "onContentUpdate" callback method I could use.

The only way my application now allows the content to be updated is by sorting it, so I implemented an appropriate callback. Here is the final source code packed into a class:

    public static class StripedTable extends FixedWidthGrid {
        private static final String EVEN_STYLE = "even_row";
        private static final String ODD_STYLE = "odd_row";

        public StripedTable() {
            super();
            addSortableColumnsListener(new SortableColumnsListener() {
                public void onColumnSorted(TableModel.ColumnSortList sortList) {
                    applyStripes();
                }
            });            
        }

        public StripedTable(int rows, int columns) {
            this();
            resize(rows, columns);
        }

        private void applyStripes() {
            for (int i = 0; i < getRowCount(); i++) {
                this.getRowFormatter().setStyleName(i, getRowStyle(i));
            }
        }

        private static String getRowStyle(int row) {
            return row % 2 == 0 ? EVEN_STYLE : ODD_STYLE;
        }
    }

hibernate4gwt and processing all class fields

2008-09-28T02:32:00.000-07:00

Today I was stuck working on my project which employed both hibernate and GWT technologies and thus also needed to use hibernate4gwt lib.

This is a great library that makes it possible to integrate two other great framewroks into one project. But as usual it has some caveats =)

It worked great for my model classes passing them from server side to JS client side. With one exception. All model object's ids were dropped. At first I thought this might have to do with @Id or @GeneratedValue annotations ... but that even sounds silly - why would you expect a cloning lib to drop a plain simple Long value because it has an @Id annotation? =)

Half an hour and 15Mb source code downloads later I found out the real cause. The library that hibernate4gwt uses for actual cloning of hibernate-enchanced objects to POJO is called beanlib (specifically beanlib-hibernate) and deep inside, in a method called net.sf.beanlib.provider.BeanPopulator#populate it uses all defined setters on a given class to populate it. Apparently I haven't defined a setter for id field - which caused the beanutil lib to drop it when converting to POJO.

So the moral of this story is: always create both getters and setters for all you fields on a class that you expect to pass to some 3rd party bean handling libs =)