Goals for the next Asset-Pipeline

Posted by David Estes on May 21, 2015

Filed under Grails, Open Source, Spring Boot, Groovy

It's been a bit since I've release a post regarding the asset-pipeline for Grails,Gradle,the JVM in general. I think it's time we talk about what some of the future goals of this plugin should be and what should be prioritized. I hear new feature requests coming in all the time and the world of javascript is a highly volatile community these days. So lets talk about some heavy hitters that need to be done with regards to managing javascript.

ES6 Module Support

This seems like a no brainer to me. Now that there is a spec for modules in Ecmascript 6 and some libraries are already starting to adhere to it, it seems necessary to write a module for the asset-pipeline that is also capable of leveraging this new module syntax. At the time of writing this post there appears to be a module built for grails to support the ES6 to ES5 conversion https://github.com/jurberg/es6to5-asset-pipeline. I have not yet tried this plugin but definitely looks to be promising. There are, however, other issues that need to be addressed when it comes to starting to support these new features. Most of them require a javascript runtime in java to process. For example, this plugin relies on Babel to convert the files to ES5. The asset-pipeline currently relies on Rhino.js to accomplish js related compilation. It is used for LESS (when not using less4j mode), Coffee, handlebars, etc. The downside to this is that Rhino.js is very slow. One suggestion might be to use Nashorn if its available to the runtime (when using Java 8) but this also presents some difficulty.

A Faster Javascript Runtime

After performing many expirements and tests with changing out RhinoJs for something like Nashorn, there seems to be a cost/benefit to going either direction. Rhino is actually faster on a cold start than Nashorn is. It takes Nashorn longer to "warm up" its runtime and JIT compile it. Rhino doesn't do this right off the bat and uses a non JIT runtime mode to start running the javascript immediately. Once Nashorn is up and running it blows away anything else on the Java stack as far as running javascript is concerned but for compiling assets on the fly or during a build, this delay can be a dealbreaker. There are some options such as compiled script casching now available in Nashorn that can definitely help with this slow startup time but the improvement is still not quite there. Nashorn is still fairly young however and it stands to reason that the team will be making great strides to improve the startup performance of the runtime.

Include/Exclude Extensibility

One of the biggest issues/complaints I've seen with the asset-pipeline is managing the include exclude patterns of assets that actually need compiled vs. assets that don't. There seems to be no easy silver bullet solution to this as everyone's code structure is different but there are some things we can do that may help ease the burdon. I would like to introduce a concept of an extensible ScanFilter. A ScanFilter could be registered by any plugin (similar to how AssetFile and AssetProcessor) and added to the pipeline that scans for files to include/exclude from a build.

A great example usecase for this might be a bower aware asset-pipeline module that could read a packages.json file and appropriately omit / include files specified in the json file without having to compile the entire set of source files. This wouldn't even be that hard of an add to asset-pipeline and could facilitate other package managers down the line. Another extensibility feature related to this might also be to allow an AssetFile object to confirm a file match not simply by an extension but also by the contents of the file.

Finish out SourceMap Support

Since the release of the Asset-pipeline 2.0, the new use of Closure Compiler for minification of javascript has allowed for creating sourcemaps making it significantly easier to debug production code. It would be great to fill this functionality out in the add on plugins for Sass, Less, and Coffeescript. Also I am considering the idea of auto bundling assets even in development runtime mode and producing individualized sourcemaps that track back to the source, further increasing development runtime performance without losing the difficulty in tracing down a bug.

Configuration

One of the issues that has cropped up with the newer version (at least in the context of Gradle) is that configuration has to be duplicated between runtime and build time. Finding a solution that moves this to one clear cut place would significantly improve the use of the plugin and reduce confusion.

Gradle Plugin enhancements

The gradle plugin for the asset-pipeline was primarily built to support the release of Grails 3 and still could use some improvements. On that list is forking the AssetCompile process into its own java process. This should relieve some memory pressure running inside the gradle jvm and reduce clutter on the class path.

Framework Support

While there is a prototype early stages spring-boot plugin for the asset-pipeline it could still use some work to improve referencing assets from within the view layer as well as build time injection.

On the ratpack from some headway has been made on using a Guice module with a custom handler to make runtime support run perfectly. This still needs a lot of work to be ready for use by a team on Ratpack and I plan to focus on this very soon.

These are some of the items that need to be addressed in the new asset-pipeline. Contributions are definitely welcome to the project as the code base grows. If you find yourself wanting to contribute things not on this list such as maybe Typescript support or Dart support feel free to build addon libraries using the extensibility guide for the asset-pipeline.