YAJET – JavaScript Template Engine

Table of Contents

1 Yet Another JavaScript Emplate Tengine1

A “template engine” is a tool able to transform some text into another, by interpreting/replacing various patterns in the source text. YAJET is such a tool designed for client-side (JavaScript, in-browser) transformation.

YAJET is a compiler, in the sense that it transforms your template into executable JavaScript code; after compiling a template you get a function which you can call with data required to fill your template, and it returns it rendered.

2 What makes it different

I think that all template tools suck to some degree, and this has to be because they are bringing together two (or more) languages. The syntax is always creepy. YAJET is no exception from this fundamental rule, but still I think it's better than others. It has a nice lispy syntax for directives, that feels much better than the sheer ugliness that other template toolkits propose. But it also strives to keep simple things simple:

Rather than encouraging you to write pieces of literal JavaScript in your templates2, YAJET defines some high-level directives that allow you to traverse lists, define blocks, conditionals and so on with a cleaner syntax; and it translates them into running JavaScript in a proper way. So you don't write, for example:

<?js for (var i = 0; i < list.length; ++i) { ?>
<?js     var el = list[i] ?>
     <li>#{el}</li>
<?js } ?>

(the example above is from Tenjin). In YAJET you write it like this:

$(MAP (el => list)  <li>$el</li>  $)

Or if you like the Perl style, you can do3: $(FOREACH (list) <li>$_</li>$).

There was an explosion of “jQuery template engines” lately, generated by jQuery's outstanding support for CSS selectors—people4 write <div class="foo"></div> to introduce a DIV containing the variable foo. I don't like this style. YAJET is appropriate for any kind of text templates—it was not designed specifically for HTML, although that's mostly what I use it for.

2.1 Features

Other than the boring variable interpolation, we have the following:

  • simple filters a la Template::Toolkit (i.e. $foo|html will HTML-escape the value of foo).
  • conditionals (IF / WHEN / UNLESS, ELSE, ELSIF).
  • repetitive constructs (REPEAT). They allow you to repeat a part of the template a few times.
  • iterating arrays or hashes (MAP and MAPHASH) – MAP to traverse a list and MAPHASH to iterate an object (hash table).
  • all loops can be controlled by BREAK or CONTINUE.
  • variable definition (LET) and dynamic scope manipulation (WITH).
  • components (BLOCK and EXPORT). Useful to declare blocks that can be called, multiple times, as a function. EXPORT declares functions that can be called from a different template.
  • components can be called as WRAP-pers. They receive some content and are able to output something before and after it. The content is processed as template.
  • various types of comments.
  • the ability to include literal text, or to temporarily modify the reader character.
  • you can easily define custom directives.
  • Correctness.
  • it compiles the template into a runnable JavaScript function which is blazing fast. Compilation speed is quite good too, but of course, you only need to compile a template once.
  • the code is way shorter than the documentation5 and it's completely self-contained (no library requirements, and it should not clash with any library).
  • it's browser agnostic. In fact, it works with standalone JavaScript engines too (such as Google's V8 or Rhino). I started a test suite based on Rhino.

3 Usage

You need to load yajet.js, and to create an instance like this:

var yajet = new YAJET({
    reader_char : "$",
    with_scope  : false,
    filters     : {
        foo: function(val) { /*...*/ },
        bar: function(val) { /*...*/ }
    }
});

All arguments are optional. Then to compile a template, you do this:

var func = yajet.compile("You said: $this.foo $this.bar");

and to execute it:

alert( func({ foo: "hello", bar: "world!" }) );

3.1 Template arguments

The function returned by “yajet.compile” receives one argument, and that argument is available in your templates via the JavaScript this keyword. This is the default behavior, because I think it's the best one in general, but if you don't like to use the this. prefix to access the members you can pass with_scope: true in the constructor. It wraps the generated function in a with(this) { ... } block, so the above would become:

var yajet = new YAJET({ with_scope: true });
var func = yajet.compile("You said: $foo $bar");
// and call it the same way:
alert( func({ foo: "hello", bar: "world!" }) );

It's more convenient, but it's slower. How much slower depends on how big is your template and how many variables there are. If you want a comparison for a large number of iterations with and without the with statement, see this file (Firebug is required for timing the operations; watch the Firebug console; it also works in Chrome with its JavaScript console; Chrome is even slower than Firefox for the with case).

3.2 Reader char

Template syntax is triggered by a single special character called the “reader char”. By default this character is $, but you can use anything else by passing the reader_char constructor argument. I personally would prefer to use some Unicode character, for instance:

var yajet = new YAJET({ reader_char: "•" });
var tmpl = yajet.compile("You said •this.foo •this.bar");

4 Syntax

YAJET parses the template as text, leaving it unchanged, until it encounters the “reader char”. What follows in this document will assume that $ is the reader character (the default). A few types of constructs are recognized:

4.1 Simple interpolation (the $foo construct)

To insert a variable you can say $foo, $foo.bar, etc. This case is quite simple. The parser will stop at a character which isn't a letter, a digit, an underscore, a dot or a pipe. The pipe is for conveniently filtering the value: $foo|html will HTML-escape the value of foo before inserting it into the output.

Note that when the dot or pipe is followed by a non-word character, then they are not considered part of the token and are left as is. Thus you can safely say “Your name is |$user.name| and score is $score.”

Filters are functions that take one argument and should return the modified value. You can easily define your own filters (more on this later). Filters can be combined, for example: $foo|upcase|html will first make foo uppercase, then apply the html filter to the upcased string.

4.1.1 Notes

  • In order to insert literally the reader character in the template output you have to put it twice, i.e. $$.
  • Because the pipe character is used for filtering, you cannot write the following: $foo|bar to get the value of “foo”, followed by a literal pipe, followed by the text "bar". Instead you should write it like this: ${foo}|bar (the next session discusses the ${foo} construct).

4.2 JavaScript expressions (the ${exp} construct)

This is similar to “simple interpolation”, in that the value of the expression gets inserted into the output. For example ${a+b} will insert the sum of a and b. The scanner is smart enough to read arbitrarily complex expressions, provided that they are properly balanced (you need to be careful about literal RegExp-s for now; more on this in Known issues).

So, an example of a perfectly valid call is:

${
   // Comments are ignored, so they can contain the closing bracket: }
   (function(arg){
     // you can use the brackets in your expression too,
     // because the scanner won't stop until it's properly balanced
     return arg.a + arg.b + arg.text;
   })({
     <!-- as a bonus, you can have HTML comments too -->
     a: 5,
     b: 10,
     text: "(foo}" // strings too
   })
}

The expression is evaluated at runtime and its value is inserted into the template output only if it's not null. The above would output "15(foo}".

4.2.1 Filters

As already noted, the $foo construct allows filtering the value through some function using a convenient syntax like $foo|html. At the time of this writing the filters available by default are:

  • html — encodes HTML special characters
  • upcase — converts the string to uppercase
  • downcase — lowercase the string
  • trim — removes leading and trailing whitespace
  • plural — useful for returning "no elements", "one element", "3 elements" depending on a numeric value.

It's easy to define custom filters when you construct the YAJET object:

var yajet = new YAJET({
    filters: {
        md5: function(value) {
            return md5_hex_of(value); // return the modified value
        }
    }
});

… and in your template: $password|md5.

There is also a syntax that allows for filters within the ${exp} construct. But since we parse valid JavaScript code, and since the pipe is a valid JavaScript character (“bitwise or”), we have to use something different. The idea was, thus, that such expressions will be parsed as a list; the first element of the list is the expression itself, and any additional elements are filters. For example:

${ this.getLabel(), upcase, html }

will convert into something like this:

output_string(
  apply_html_filter(
    apply_upcase_filter(
      this.getLabel()
    )
  )
)

Since the comma doesn't look very nice for this particular case, the “list reader” also allows a few aliases. Syntactic sugar, baby! You can also use:

  • “=>”
  • “,”
  • “..”
  • “;”

So the above example can also be written like this:

${ this.getLabel() => upcase => html }
${ this.getLabel() => upcase, html }
${ this.getLabel() .. upcase; html }

These special separators only work for the “list reader”, which is used in the ${exp}-like constructs (and several others). Also, note that filters are only interpreted in the top-level elements of this list, so for instance the following won't apply the "html" filter to “foo”: ${ something(foo, html) }. It will just call, instead, the function something, passing the variables foo and html, which is expected behavior.

When used in the ${exp} construct, filters can receive additional arguments. For example, assuming you have some date formatting library, you can easily define a filter that formats a Date object according to the arguments:

var yajet = new YAJET({
    filters: {
        format_date: function(date, format) {
            // ... now return the *date* formatted according to *format*
        }
    }
});

and in the template:

“Today is: ${ new Date() => format_date("YYYY-MM-DD") }”

The first argument of your filter is always the value from the template (in the above case, the Date object created with new Date()), and the other arguments are passed following the filter name ("YYYY-MM-DD").


You would use “plural” like this:

1. We got ${ count => plural("no items", "one item", "two items", "# items") }
2. We got ${ count => plural([ "no items", "one item", "two items", "# items" ]) }
3. We got ${ count => plural("no items|one item|two items|# items") }

Besides the implicit argument (count) plural accepts multiple arguments (case 1 above), or a single array argument (case 2) or a string (case 3) that specifies the formats separated by a pipe character. In all cases, the arguments specify how to display the numeric value. If it's zero, it selects the first argument; if it's one, it selects the second, and so on. If it's bigger than the number of arguments, it selects the last one. # is replaced with the number. So the above displays "We got no items" when count is zero, "We got one item" when count is 1, "We got two items" when count is 2 and "We got # items" when count is bigger (where # is replaced with the value of count).

4.3 Directives

So far we are able to introduce arbitrary JavaScript variables and expressions in the template. However that's hardly enough. First off, the expressions must be well-formed, so there is no way to start a JavaScript block somewhere and end it some place else. The following is invalid for obvious reasons:

${ if (link != null) { }
  <a href="$link|html">$link</a>
${ } }

I emphasize that the lack of support for partial expressions is a feature, not a limitation. This will never be “fixed”. To support constructs like the above but without encouraging bad style or awful syntax, we have a few special processing directives. Let's call these the $(BAR ... $) construct. To start with, here is how you would write the above code:

$(IF (link != null)
  <a href="$link|html">$link</a>
$)

All blocks end with a closing paren; no need for “{/IF}”, “.IF”, “END”, “<? } ?>” etc. This has an incredible advantage6: if your editor can properly match parens, you can immediately see where a block starts or ends by just moving the caret to the ending/opening paren.

Note that the processing instructions are not case-sensitive. I prefer to use UPPERCASE for them so that they stand out visually.

The $(BAR ... $) construct has the following properties:

  • it starts with $( (so it's a normal paren, not a bracket)
  • it continues with a special instruction (again, I prefer uppercase for this but it's not required)
  • depending on the instruction, certain arguments may follow
  • it usually ends with $)
  • it may contain a block of text between the arguments and the $) terminator

The block of text is parsed normally, so it's interpreted as plain text until $ (the reader char) is encountered, then what follows the reader char is processed by the rules I described in this document.

Following I will describe the directives available at this time. I think the set of them is quite comprehensive and allows you to express any kind of template in a simple and consistent manner.

4.3.1 IF / WHEN / UNLESS, ELSE / ELSIF — conditional execution

IF and WHEN are synonyms, while UNLESS is the antonym. WHEN seems more appropriate for cases where you don't have an ELSE clause. They support one argument which must be a condition enclosed in parens. Examples:

$(WHEN (user_id == null)
  <a href="...">Please login</a> $)

$(UNLESS (user_id != null)
  <a href="...">Please login</a> $)

$(IF (a < b)
  <p>A is smaller</p>
$(ELSIF (a > b))
  <p>B is smaller</p>
$(ELSE)
  <p>A and B are equal</p> $)

Note that you can use ELSE or ELSIF inside UNLESS or WHEN blocks too, although I would not advise to use this style:

$(UNLESS (a == b)
  they are different
$(ELSE)
  they are equal $)

You should also note that ELSE and ELSIF are not actually parsed like other instructions. They don't take a block of text, and thus they don't need to end with $). Whether to do it this way was hard to decide, but since ELSE and ELSIF normally continue an IF block, instead of ending it, it seems to make sense this way. The same applies to $(BREAK) and $(CONTINUE) directives.

4.3.2 AIF / AWHEN — like IF / WHEN, but store the condition in $it

These two come from the anaphoric macro collection from Hell and I find them quite useful for cases where the block inside the IF is not very big. They help with the following case:

$(LET ((foo => this.looongComputation()))
  $(WHEN (foo)
    ... do something with $foo
  $)
$)

The two anaphoric macros (which are synonyms) allow you to avoid the boilerplate:

$(AWHEN (this.looongComputation())
  .. do something with $it
$)

The variable $it is created by the macro and takes the value of the condition, and the text block is executed only if7:

  • $it is not null and not undefined
  • $it is not false 8
  • $it is not an empty array
  • $it is not an empty string

It expands to this code:

(function(it){
  if (it != null && it !== false && !(it instanceof Array && it.length == 0) && !(it === "")) {
    // splice the block of code here
  }
}).call(this, this.looongComputation());

OK, now that you agree that this is useful, but are depressed by the sheer lack of inspiration in picking the name it, let me show you that you can actually name the variable:

$(AWHEN (this.looongComputation() => that)
  <!-- no more $it -->
  .. do something with $that
$)

Also, for cases when you are unhappy with the default falsity rules, you can state the full condition as well:

$(AIF (this.looongComputation() => foo, foo > 5)
  $foo is now this.looongComputation() but this is displayed
  only if it's greater than 5.
$(ELSE)
  And you can still use $foo here.
$)

4.3.3 REPEAT — to repeat stuff

To repeat a part of the template you can use REPEAT. For example, the following outputs “foo” 3 times: $(REPEAT (3) foo $). In various cases you might need to know the current iteration too, so you can pass a variable name for it:

$(REPEAT (5, i)
  Item $i $)

The variable i takes values from 1 to 5 (inclusively) and the output will be “Item 1 Item 2 ” etc. In some cases you might want to specify an interval (so that you start from something else than 1), so the following is allowed:

$(REPEAT (5 .. 10 => i)
  <a href="/page$i">Page $i</a> $)

Note that the arguments are parsed using the “list reader”, so you can use syntactic sugar to separate them (although a simple comma would do).

4.3.4 MAP / FOREACH — iterate an array

Again, MAP and FOREACH are synonyms. You can use them to do something for each element of an array. For example the following outputs links contained in an array:

$(MAP (link => links)
  <a href="$link.address|html"
     title="$link.tooltip|html">$link.text|html</a> $)

That's assuming that links is an array of objects, each containing address, tooltip and text. You could of course use a literal object:

$(MAP (link => [ { address : "http://www.google.com/",
                   tooltip : "Search engine",
                   text    : "Google" },

                 { address : "http://www.ymacs.org/",
                   tooltip : "AJAX code editor",
                   text    : "Ymacs" }
               ])
  <a href="$link.address|html"
     title="$link.tooltip|html">$link.text|html</a> $)

Sometimes you also need to know the current step of the iteration. For example if you want to output some links that are separated with a pipe, you need to know not to output the pipe before the first, or after the last link. We could write it like this:

$(MAP (i, link => links)
  $(WHEN (i > 0) | $)
  <a href="$link.address|html"
     title="$link.tooltip|html">$link.text|html</a> $)

or

$(MAP (i, link => links)
  ${ i > 0 ? "|" : "" }
  <a href="$link.address|html"
     title="$link.tooltip|html">$link.text|html</a> $)

A special case of MAP / FOREACH allows you to pass only the array, and no key or index variables. In this case the special variable $_ (which I will call the Perlism) gets assigned to the current element, and more, the loop body is lexically scoped to each element using a JavaScript with block (I know, your mom told you not to play the with statement, but mine didn't9 :-p).

So using this style the first example would become:

$(MAP (links)
  <a href="$address|html" title="$tooltip|html">$text|html</a> $)

address, tooltip and text access the specific property of each element.

Just a last example showing the Perlism:

$(FOREACH ([ "foo", "bar", "baz" ]) <b>$_</b> $)

will output “<b>foo</b> <b>bar</b> <b>baz</b>”. The $_ variable is bound to each element. Note that because YAJET is doing The Right Thing, the following will work as expected:

$(MAP ([ "foo", "bar", "baz" ])
  $(MAP ([ 1, 2, 3 ])
    inside: $_ $)
  outside: $_ $)

When “inside”, $_ will take the values from 1 to 3; “outside” it will take "foo", "bar" then "baz".

4.3.5 MAPHASH — iterate an object (hash)

MAPHASH is MAP's analogue for hashes. It iterates over all properties of an object, binding a variable for the key and another for the value. You must specify names for these variables. Example, assuming that users is a hash that maps user IDs to some user objects (each of them having a getName() method):

$(MAPHASH (uid, obj => users)
  User <b>$uid</b> has name <b>${ obj.getName() }</b><br /> $)

4.3.6 CONTINUE and BREAK — for loop control

These don't take any arguments, and also don't take a block of text, so the expected syntax is $(CONTINUE) and $(BREAK). They can appear in the text block of some looping construct, be it REPEAT, MAP, FOREACH or MAPHASH, and they do the same as their JavaScript counterparts, that is: CONTINUE will go to the next iteration, skipping any code between it and the end of the loop, and BREAK will immediately end the loop.

I'm giving an example just to illustrate the syntax:

$(REPEAT (10 => i)
  $(WHEN (i > 5) $(BREAK) $)
  $i
$)

The above will print numbers from 1 to 5.

4.3.7 LET — define variables

You can define new variables with LET. It introduces a new lexical scope, so the variables that you define are only available in its block of text. If variables with the same name already exist, they are shadowed while the LET block is in effect. After the LET block ends, previous bindings come back to life.

$(LET ((a => 10) (b => 20))
  $a + $b = ${ a + b }
$)

Since LET takes a block of text, it ends with the normal block terminator $). Here's an example to demonstrate scope:

$( var x = "outside" /* literal JS block, described later */ )
$(LET ((x => 10))
  $x is 10
  $(LET ((x => 20))
    $x is 20
  $)
  $x is back 10
$)
$x is "outside"

LET operates by introducing an anonymous function, so it's compatible with all browsers. JavaScript 1.7 introduced a let statement for declaring block-scoped variables, and it's supported by Firefox, but unfortunately no other browser has it at the moment10.

4.3.8 WITH — modify the scope chain

When you have an object that has properties you need to access, you can use a WITH block to make for a more convenient syntax, so instead of saying $object.foo you would be able to say only $foo. Assuming that link contains address, tooltip and text, the following two are equivalent:

<a href="$link.address|html" title="$link.tooltip|html">$link.text|html</a>

$(WITH (link)
  <a href="$address|html" title="$tooltip|html">$text|html</a> $)

WITH can be used with literal objects as well:

$(WITH ({ foo: 10, bar: 20 })
  $foo + $bar = ${ foo + bar }
$)

thus emulating a LET block, but it's less efficient because it uses the JavaScript with statement.

4.3.9 BLOCK — define reusable template blocks

A BLOCK doesn't immediately print anything into the template output; instead it defines a function that returns its processed block of text.

The syntax is straightforward. It expects a name for the function, followed by a list of arguments in parens (if there are no arguments, put () like you do for a plain JavaScript function). Then continue with the block of text that the function should expand into:

$(BLOCK display_link(link)
  <a href="$link.address|html" title="$link.title|html">$link.text|html</a>
$)

<!-- call it literally -->
${ display_link({ address: "/", title: "Home page", text: "Home" }) }

<!-- or call it for an object -->
$(FOREACH (i => links)
  ${ display_link(i) }
$)

Note that the call to display_link is inside a ${...} block, so that the returned value gets inserted into the output.

Combining BLOCK and LET or WITH we can define closures:

$(WITH ({ value: 0 })
  $(BLOCK counter()
    <p>Counter is ${ ++value }</p> $) $)

${ counter() } -- now it's 1
${ counter() } -- now it's 2
${ counter() } -- now it's 3

Doing the above with LET is a bit more tricky because LET creates its own environment, so the BLOCK that you define within it is actually local to the LET block. The following won't work:

$(LET ((value => 0))
  $(BLOCK counter()
    <p>Counter is ${ ++value }</p> $) $)

${ counter() } -- error, counter is not defined!

It's easy to see why if you see the code that gets generated for the above. It looks like the following:

(function(){
    var value = 0;
    function counter() {
        output("Counter is " + (++value));
    };
})();

output( counter() ); // but there's no free lunch

To do this with a LET block we would have to export the function; we can use an outside variable for that:

$( var counter )
$(LET ((value => 0))
  $( counter = _counter <!-- export it --> )
  $(BLOCK _counter()
    <p>Counter is ${ ++value }</p> $) $)

${ counter() } -- now it works.

4.3.10 WRAP, CONTENT — call a wrapper with an additional block of text

BLOCK-s can be used as wrappers. A wrapper is a function that receives a bit of text and puts something before and after it. For example, to define a wrapper that creates a table we can say:

<!-- define our wrapper -->
$(BLOCK table(cols)
  <table>
    <thead>
      <tr>
        $(MAP (label => cols) <td>$label</td> $)
      </tr>
    </thead>
    <tbody>
      $(CONTENT)
    </tbody>
  </table> $)

<!-- and here's how we use it -->
$(WRAP table([ "Name", "Phone", "Email" ])
  <tr> <td>Foo</td> <td>123-1234</td> <td>foo@foo.com</td> </tr>
  <tr> <td>Bar</td> <td>1234-123</td> <td>bar@bar.com</td> </tr>
$)

You can note that the wrapper is a normal function (BLOCK) and it can take arguments. To send the arguments with a WRAP block, just make it look like a normal function call. If there are no arguments, you still need to insert the parens (). When it's calling your block, WRAP sends an additional hidden argument that contains the text which is expanded by $(CONTENT). For now this argument is a function that renders the text, and $(CONTENT) simply calls this function.

4.3.11 EXPORT — define a BLOCK that can be used in another template

EXPORT is like BLOCK, but the function that it creates is “exported” and can be called from different templates. The assumption for this to work is that all templates are compiled with the same YAJET object instance (since it will maintain some runtime environment for this case).

Here's a quick example:

var yajet = new YAJET();
yajet.compile("$(EXPORT foo(arg) foo got $arg $)");

var t1 = yajet.compile("$(IMPORT (foo)) ${ foo('bar') }");
alert(t1()); // displays "foo got bar"

var t2 = yajet.compile("$(PROCESS foo('baz'))");
alert(t2()); // displays "foo got baz"

// call the exported function directly
alert(yajet.process("foo", null, [ "something" ])); // displays "foo got something"

Above you can see a few ways to call an exported block. One is by calling $(IMPORT (block_name)) first, which will actually make it available as a local function, which you can then use as if it were defined with BLOCK. The second way is using $(PROCESS block_name()). PROCESS expects that the name of the block that you type there is a function created with EXPORT and compiled before the call to PROCESS.

It might be important to understand that compile() actually runs your template once when it contains EXPORT-ed functions, so that they get into the YAJET instance. This shouldn't be a problem—in practice, you will have templates that contain only export blocks, where you will put utilities. For example, above we don't store the result of yajet.compile for the first template, since all it does is just export the function. The exported function gets into the YAJET object instance.

Some notes:

  • the order in which you compile the templates is not important. However, when you execute a template you must make sure that any dependencies were already compiled.
  • there is no namespace support, so make sure that you don't export a block with the same name in two templates. Typically, the second will overwrite the first (depending on the compilation order), but you get no warning.
  • within the template where they are defined, you can use exported blocks as if they were local. They can call each other if needed, they can $(WRAP) each other, etc.
  • You can use WRAP in a different template to call an exported function as wrapper, without having to IMPORT it first. Note however that if some BLOCK or other function with the same name exists in the current template, it will take precedence.
  • as you can see above, it is possible to call an exported block directly without using an intermediate template. Use yajet.process(exported_name, self, [ more, args ]). The self argument is accessible as this within the block, and the last is an array of more arguments that are passed to the function.

4.3.12 LITERAL — include literal text

With this directive you can include a literal block of text. No constructs within it are expanded. Example:

$this.foo -- here it's replaced with the value of the variable
$(LITERAL "STOP"
  $this.foo -- here it's left untouched
STOP)

The LITERAL directive takes one string as an argument. That string immediately followed by a closing paren is expected to end the literal block of text. In the above example we state that “STOP)” should end the text. Note that the closing paren is implied, and required:

$(LITERAL "FOO"
  The following doesn't end the block:
  FOO
  but the following does:
  FOO)

4.3.13 SYNTAX — temporarily change the reader character

When you need to display the current reader char literally, many times in a block of text, rather than typing it twice each time it's sometimes more convenient to temporarily change it to something different. Example:

Price: $$ $this.price
TAX:   $$ $this.tax
This is a dollar sign: $$

<!-- here's another way: -->
$(SYNTAX #
  Price: $ #this.price
  TAX:   $ #this.tax
  This is a dollar sign: $
#)

$this.tax -- back to previous reader char

SYNTAX takes a single char argument (the first non-white-space character that follows the directive) and that char becomes the reader_char while its block of text is in effect. Note that to end the block of text you need to include the normal block terminator, but using the new reader char—so we need #) instead of $) in the above sample.

4.3.14 Literal JavaScript with $( ... )

Finally, you can include literal JavaScript code, if needed, by placing a space after the open bracket. The code inside $( ... ) must be valid JavaScript and by this I mean properly balanced (you cannot open a paren in such a block and close it in another).

For example, if you need to change the value of some variable which is already defined, you can do this:

$( myVar = doSomething() )
  ^-- note this space.

Unlike a ${ ... } block, which would allow the above code as well, this one won't place the result into the template output. Also, unlike a ${ ... } block, this one allows multiple statements separated with a semicolon:

$( foo = "bar";
   moreSideEffects();
   i = 10 )

5 Correctness

YAJET aims to do The Right Thing. If you've ever written Lisp or C macros, then you know that it's dangerous to invent variable names, or to use a macro argument more than once. YAJET is essentially a macro expander and it's built around these good principles.

For example, a dumb implementation would translate $(FOREACH (link => links) ...STUFF... $) into this:

for (var i = 0; i < links.length; ++i) {
    var link = links[i];
    // ... do STUFF
}

However the above code has two problems:

  1. if the text in STUFF defines a variable named i, then it will collide with the loop variable.
  2. if links is not a real array, but say, a (possibly expensive, and perhaps with weird side effects) function call that returns an array, then it will be called for each iteration… twice.

If FOREACH would really expand into the above code, then the following sample would suffer from both problems:

$(FOREACH (link => this.getLinksFromServer())
  $( var i = link.text.length )
  $(WHEN (i > 30)
    ... truncate text
  $)
  ...
$)

The resulted code would be:

for (var i = 0; i < this.getLinksFromServer().length; ++i) {
  var link = this.getLinksFromServer()[i];
  var i = link.text.length;
  if (i > 30) {
    ... truncate text
  }
  ...
}

… which means that this.getLinksFromServer() will be called twice for each step, and also that the loop would be stopped arbitrarily when we encounter a link whose text has more characters than the number of links. That would break in unexpected and hard to debug ways.

What YAJET actually generates for the above case looks like this:

(function(__GSY12){
  for (var __GSY13 = __GSY12.length, __GSY14 = 0; __GSY14 < __GSY13; ++__GSY14) {
    var link = __GSY12[__GSY14];
    var i = link.length;
    if (i > 30) {
      ... truncate text
    }
    ...
  }
}).call(this, this.getLinksFromServer());

The variables that aren't explicitly named in the template get unique names with the prefix __GSY, so you should be safe as long as you don't use the __GSY prefix yourself. Also, there are a few other internal variables that YAJET has to use:

I prefixed those that I assumed won't be generally useful with two underscores. OUT and VUT are not prefixed because they are needed for custom directives. Both of them are functions that currently do the same thing11: they take one argument and if it's not null they insert it into the output.


You can note in the code above that the loop block is embedded in a function, so that it doesn't affect outside variables. Creating lambdas everywhere has other implications that we need to be careful about:

5.1 Loop control

Imagine this loop:

$(MAP (a => [1, 2, 3, 4, 5])
  $(LET ((b => a))
    $(WHEN (b > 3) $(BREAK) $)
    $b
  $)
$)

If $(BREAK) would translate into the plain JavaScript break statement, it would be a syntax error because LET introduces an anonymous function (in order not to mess with outer variables). The above block expands into something like the following, which is the right thing:

(function (__GSY31) {
    for (var __GSY32 = __GSY31.length, __GSY33 = 0; __GSY33 < __GSY32; ++__GSY33) {
        try {
            var a = __GSY31[__GSY33];
            (function () {
                var b = a;
                if (b > 3) {
                    throw __YAJET.X_BREK; // this is BREAK
                }
                VUT(b);
            }).call(this);
        }
        catch (ex) {
            if (ex === __YAJET.X_CONT) { // here we handle CONTINUE
                continue;
            }
            if (ex === __YAJET.X_BREK) { // and here we handle BREAK
                break;
            }
            throw ex;
        }
    }
}).call(this, [1, 2, 3, 4, 5]);

So BREAK and CONTINUE are handled with exceptions, which has an interesting implication: if you know that some function will only be called from loops, then you can safely use BREAK and CONTINUE within it. But only if you know that. ;-)

5.2 The value of this

Put simply, YAJET will “copy” this to all functions that it creates or calls itself, so that this will still refer to your template argument even if you're inside some loop or other automatically generated function.

6 Custom directives

YAJET allows you to add custom directives fairly easily, though you'll have to dig somewhat uncharted territory. You need to pass a directives hash to the constructor, in which you map directive name to a parser function. Your function is responsible for parsing any arguments that you want your directive to support, and to generate any code that your directive should expand into.

Let's start with an easy one:

var yajet = new YAJET({
    directives: {
        author: function(c) {
            c.out("OUT('Mihai Bazon <mihai.bazon@gmail.com>');");
            c.assert_skip(")");
        }
    }
});

The above defines a directive that doesn't take any arguments. You can notice that your parser function receives one argument—it's an object that stores the current context and provides some helper API for you to do your stuff. Above I used the out method, to output code that should be part of the compiled template, and assert_skip to force an error unless the template continues with a closing paren.

In a template compiled with the above object instance, we can now type $(AUTHOR), and it will expand into this:

OUT('Mihai Bazon <mihai.bazon@gmail.com>');

In turn, when the template is executed, OUT will put its argument into the output stream.

Your directive handler is free to parse any syntax that you desire, but after it finishes the “normal” parser resumes execution for the remainder of the code. The “normal” is: assume plain text until we meet the reader char, then parse according to the rules described in this document.

Again, this isn't for everyone so I won't get into much detail—please feel free to read the source to figure out more. I'll just summarize the API that the context object exposes:

You should understand that your custom directives run at compile time. So “c.out” does not produce the final template result; instead, it should produce JavaScript code that generates the final result when executed. This is why our sample above doesn't simply say c.out("author..."), instead it has to say c.out("OUT('author...');"). The OUT function is available at run-time and inserts text as part of the final result.

6.1 A more involved example — SWITCH

For a non-trivial example, here's how to implement a SWITCH directive. It has the same semantics as the standard JavaScript SWITCH — that is, depending on the value of some expression, it selects and executes a CASE. The DEFAULT case is executed when no other case matches the expression. We will in fact make use of the standard JavaScript switch for this.

var directives = {
    "switch": function(c) {
        // SWITCH expects one expression in parens:
        var args = c.read_balanced(true);
        var expr = args[0]; // here is the argument

        // save old meaning of CASE and DEFAULT, if any
        var old_case = c.directives["case"];
        var old_defa = c.directives["default"];

        // inject the CASE directive
        c.directives["case"] = function(c) {
            var args = c.read_balanced(true);
            var expr = args[0];
            c.set_output(save);
            c.block_open(
                "case " + expr + ":",
                function() {
                    c.out("break;");
                    c.set_output([]);
                }
            );
        };

        // and the DEFAULT directive
        c.directives["default"] = function(c) {
            c.set_output(save);
            c.block_open(
                "default:",
                function() {
                    c.out("break;");
                    c.set_output([]);
                }
            );
        };

        // finally, open the switch block and prepare to close
        // and restore everything when the block ends.
        c.block_open(
            // open
            "switch (" + expr + ") {",
            // close
            function() {
                c.set_output(save);
                c.directives["case"] = old_case;
                c.directives["default"] = old_defa;
                c.out("}");
            }
        );

        // any text between these directives is not interesting.
        var save = c.set_output([]);
    }
};
var yajet = new YAJET({
    directives: directives
});

And now, you can use SWITCH in your templates:

$(SWITCH ("foo")
   $(CASE ("bar") This won't be written. $)
   $(CASE ("foo") But this will. $)
   $(DEFAULT And this not. $)
$)

The implementation of SWITCH needs to be a bit complex. We insert the CASE and DEFAULT directives when our SWITCH directive runs (that is, when the template is compiled), but remove them once the SWITCH block is ended (since they don't make sense outside SWITCH). We need to use set_output to change the output array to some temporary one which we will discard, because otherwise the whitespace between $(SWITCH (...) and the first $(CASE) will be transformed into code, and it won't be valid JavaScript syntax. And we need to be careful to set the output back to the saved value when needed12.

This topic is advanced so I will stop here. If you want to write your own directives, you are assumed to have some good JavaScript knowledge and dig through the code for more (and/or ask on the YAJET group, but preferably after you have something to show us).

7 Known issues

7.1 Literal RegExp-s in JavaScript expressions

The JavaScript scanner is not “complete”, although it's smart enough to skip comments and strings while looking for a closing paren. Literal regexps are tricky to figure out, so I left this out for now. What this means is that you should be careful about parens in literal RegExp-s. Since the parser does not allow for unbalanced parens, the following should not be a problem:

$( if (/(a|b)/.test("bar")) {
     matches();
   } else {
     no_match();
   }
 )

All parens are properly closed, so there's no reason why our parser should miss the closing paren. However, the following will break stuff:

$( if (/\)/.test(")")) { ... } )

Although it is valid JavaScript inside, having the closing paren in the RegExp will confuse YAJET. It looks quite ugly, too—for such cases, encode the paren as \x29. Note that you have to escape open parens as well (\x28), and same goes for all the other types of brackets such as [, ], { and }.

7.2 Error reporting is less than ideal

While YAJET is smart enough to scan complicated constructs, it will not do any syntax checking on its own. It just scans your template and generates JS code. Then it compiles a function (using the Function constructor). At this point the browser (its JavaScript engine) does the proper syntax checking and error reporting. If anything goes wrong, you do get an error, but it's less informative than it could be. If you're using Firebug or Google Chrome, the generated code will show up in the console so you get a chance to see what's wrong, but don't trust the line number or file that its displayed there.

I don't see me writing a full JavaScript parser anytime soon, so for the time being we will have to live with this. It's still pretty good. ;-)

7.3 Whitespace handling

Currently YAJET keeps all whitespace in the generated source13. There is a directive that allows you to say “kill following whitespace” ($-, that is, the reader char followed by a minus sign) but it's not very convenient. For example:

<p>
$(IF (true)
  foo
$(ELSE)
  bar
$)
</p>

results in this output:

<p>

  foo

</p>

Generally, it's not what one would expect. We can make it look better with the “kill whitespace” directive, but it's totally unintuitive:

<p>$-
$(IF (true)
  foo$-
$(ELSE)
  bar$-
$)
</p>

This outputs better:

<p>
  foo
</p>

So the default behavior should probably be:

  • if a line starts with whitespace followed by a directive, the whitespace should be eaten.
  • if a line ends with a block close paren ($)) followed only by whitespace, then that whitespace + the newline will be eaten.

Need to think about it a bit more. However, fortunately in HTML whitespace is not too important.

8 How to get help

If you have any questions please post them on the YAJET Google Group.

9 License

Copyright (c) 2010, Mihai Bazon, Dynarch.com. All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Footnotes:

1 The misspelling is intentional. Various combinations of the letters Y, A, J, T, E from “Yet Another JavaScript Template Engine” led to the name YAJET. YAJET stands for “Yet Another JavaScript Emplate Tengine”. Sounds buzzy, isn't it? Also, JET-s are fast, and so is YAJET.

2 You can still put literal JavaScript inside using $( ... ), but it has to be properly balanced.

3 I added this because it was easy, and it can be useful for one-liners, but I vote against it for blocks bigger than a few lines.

4 Pure comes first on Google when we search “JavaScript template engine”. Have you notice how exaggeratedly creepy is the syntax for rendering with directives? I guess we truly live in a “worse is better” world, but I'm still trying to do The Right Thing.

5 This is a double-feature: we have good documentation and lots of features for small code. ;-)

6 Sarcasm intended. I still can't figure out why all other template solutions insist on not using plain parens.

7 Note that the JavaScript rules for falsity are different, and I think less useful: an empty array will stand true, while the number 0 (zero) is false.

8 BTW, did you know that in JavaScript the expression (0 == false) evaluates to true in conditionals?

9 Seriously though, everything under an with block is s…l…o…w… – so, while this makes for a nice syntax, you should not use it where speed is critical.

10 Since I'm not sure what are the benefits of the let keyword from JavaScript 1.7 compared to using an anonymous function, I decided not to add a browser check for this. When more browsers will support it I'll change my mind. But the template syntax will remain the same.

11 VUT however might change in the future. OUT is intended to output plain text, while VUT is meant to output the value of an expression.

12 The fact that JavaScript properly supports closures plays a key role into all this, but I'm tired of saying this all over. :-)

13 Speaking of it, most (all?) template engines do the same.

Author: Mihai Bazon <mihai.bazon@gmail.com>

Date: 2010-05-30 12:46:37 CEST

HTML generated by org-mode 6.21b in emacs 23