The goal for using JIT (just-in-time compilation) is to speed up interpreted code by compiling it in run time. This is very tricky for dynamic languages such as Matlab (also Python, Perl, etc. ). The main reason is that the variable types are determined at runtime. For example:
if j > 5
a = int8(j);
a = float(j);
What's the type of a? Compiler has to know the variable size and type to emit correct code. One can imagine using an object to represent variables (the object would carry a type and pointer to memory where data is stored) and assignement operator which would assign both value and type. However, such implementation results in code that is not much faster than interpreted code.
However, code like the snippet above is rare. In most cases variables have well defined type at runtime. For such a code we can generate very fast machine code.
Here's the current approach that FreeMat takes is:
- Compile code that would most benefit from speedup (loops, functions)
- If code cannot be compiled fall back to using the interpreter (slow, but at least you always get an answer).
- Check for variable type changes between running JIT compiled code.
- JIT compiled code works on the same data structures as the interpreter.
LLVM was our choice. It is not perfect - the library is really huge and quite hard to compile and use. However, you can generate and optimize code on the fly and get near optimal performance (the only thing you can't do with JIT code is interprocedural optimizations).
We plan to have functional JIT compiler in Freemat 4.