Customize module compiler using closure compiler

                                                                              yiminghe at gmail.com

    Because there rarely exists articles on this topic, I will write in English, this article can also be found at Google Doc . Forgive my chinglish and syntactic error if you are interested.

 

Preface

About how to use closure compiler, you can refer to: 使用closure compiler高级模式

About module, you can refer to : 模块加载 ，模块设计

 

Part 1: Introduction

    In this article, I will show how to customize your own module compiler using closure compiler’s APIs provided.

 

Goals

 

Combiner

 

Scene

 

    There exists some modules, they may depend on each other, but cyclic dependence will be left for later discussion.

For example:

Event module in event.js:

 

KISSY.add(“event”,function(){

  //do your stuff and return event module

   return xx;

},{

 requires:[”event/base”,”event/ie”]

});

 

event/ie module in event/ie.js :

 

KISSY.add(“event/ie”,function(){

    //do your stuff and return event/ie module

    return yy;

     },{

    requires:[“event/base”]

      });

 

     There also exists dom, dom/ie …. , infrastructure module and some UI modules, complicated UI modules may depend on dom, event and infrastructure module.

 

Expected

     This combiner tool can generate file with modules you want exactly, no more and no less. for example : if you want a simple slide ui module ,and this module only depends on dom and infrastructure, it simply shows some effect (no interaction),so event module is not needed ,you can run :

    combiner requires=slide output=slide_standalone.js

     Done! That’s it. slide_standalone.js is generated, and only includes dom/ie, dom, infrastructure, slide in one file .Invoker does not need to know which depends on which . At the same time it can accelerate your application (less http connection);

 

Compiler

 

Scene

    If you look closely at the above part, you will notice some information redundancies, for example: a module file is named as dom.js, and in this file, you have to say

 

  KISSY.add(“dom”,function(){},{…});

 

It would be better: if in dom.js:

 

  KISSY.add(function(){},{…})

 

      The module’s name is same with module’s file name.  If dom.js only contains dom module, it’s fine, some module loaders can solve this problem by attaching file name with module name when loading module files dynamically. But when you combine dom.js with some other module into one file, it’s impossible to associate a single module’s factory function with corresponding module name.

 

Expected

     This tool will automatically transform original module definition code by adding module name which can be obtained from module’s file name.

For example: dom.js contains dom module:

 

KISSY.add(function(){},{requires:[“dom/base”,”dom/ie”]});

 

After transformation, the code will be:

 

KISSY.add(“dom”,function(){},{requires:[“dom/base”,”dom/ie”]});

 

“dom” comes from dom.js

Now you can be free to use combiner to combine multiple modules into one file.

 

Part 2: Using closure compiler

     Closure compiler provides some unpublished APIS, they can be used to customize your own module compiler, it means you do not need to code everything from scratch ,such as lexical, syntactic parser (actually closure compiler does not code from scratch neither, it uses rhino but exports a set of better API) and serialize.

 

Parse

     The first thing is parsing, if you want to do code analysis, you have to parse code to an AST(abstract syntax tree ). In closure compiler it does not provide a simple api (or I missed) ,here is what i do :

 

/**

     * get root node of ast from javascript source string

     *

     * @param {String} code    javascript source

     * @return {Node} root    node of ast corresponding to source code

     */

    public static Node parse(String code) {

        //just want to use Compiler as one parameter for CompilerInput's getAstRoot method

        Compiler compiler = new Compiler();

        CompilerOptions options = new CompilerOptions();

        // Advanced mode is used here, but additional options could be set, too.

        CompilationLevel.WHITESPACE_ONLY.setOptionsForCompilationLevel(

                options);

        compiler.initOptions(options);

        //get a fake file representing input source code

        //CompilerInput need JSSourceFile as type of input for constructor

        JSSourceFile input = JSSourceFile.fromCode("parse.js", code);

        CompilerInput ci = new CompilerInput(input);

        //here we go , finally get root node of ast

        return ci.getAstRoot(compiler);

    }

 

For example (from closure compiler’s wiki)

 

x=’string’

 

AST correspond to the above code :

 

     After you get AST ,you can do anything about this tree ,just as DOM tree with different operation API. In reality closure compiler applies multiple optimization passes to AST and changes it dramatically in the end.

 

Serialize

     Finally AST is ready, you want to serialize it to code file, I still do not find a direct way, it seems closure compiler associates everything with Compiler class. I have to construct a Compiler Object:

 

/**

     * get javascript source from root node of its ast

     *

     * @param {Node} jsRoot root node of javascript source's abstract syntax tree

     * @return {String} corresponding javascript source

     */

    public static String toSource(Node jsRoot) {

        //here,just want Compiler's toSource method

        Compiler c = new Compiler();

        CompilerOptions options = new CompilerOptions();

        //just need option,no real effect

        CompilationLevel.WHITESPACE_ONLY.setOptionsForCompilationLevel(

                options);

        options.prettyPrint = true;

        c.initOptions(options);

        Compiler.CodeBuilder cb = new Compiler.CodeBuilder();

        //finally can get source code by ast

        c.toSource(cb, 0, jsRoot);

        return cb.toString();

    }

 

Part 3: Customize module compiler

This part is short. Because it simply combines the above two parts together:

1. Read code file and transform it to AST, add the missing module name by module’s file name.

2. Track current module’s dependencies and recursively process them.

3. After solve current module’s dependencies, serialize current module’s final AST to file.

4. If you want, you can call closure compiler’s optimization method or just leave it to external invoke later.

 

Part 4: Usage

To Be Continued

Part 5: Summary

     Using closure compiler you do not need to code everything from scratch, you can also process code file deeply according to your own business logic.

     I hope there will be more articles about closure compiler’s implementation. It’s a fantastic library for F2E.