Difference between revisions of "Insertion point proposal"

Latest revision as of 08:55, 31 July 2009

Xpand feature request (Jos Warmer, March 18, 2008)

I have used both model-to-model and model-to-text transformation languages. There is one characteristic which makes them very different in their possibilities. Well, there are more, but there is one which I want to focus on. This document describes a proposal to bring some of the strengths and flexibility from M2M to M2T transformations.

Model-to-model transformations

One of the advantages of model-to-model transformations in many cases is the flexibility that it gives. You can build the target model incrementally because you have access to the whole structure. After building the initial target model it is easy to insert other things later on. This allows for more and often better composition of transformations. An example :

• First Java classes are created
• Secondly, we have a transformation that adds stuff to this Java model. E.g. add code for implementing the visitor pattern to all classes. This means that an ‘accept’ operation needs to be added to all classes, etc.

Using model-to-model transformations we can easily separate these two.

Model-to-text transformations

Model-to-text transformations are often easier to write, one of the reasons that this approach is popular. However, your flexibility is more limited. Given the example above, once you have created the java class files it is not possible to add a method (or anything else) later on to the file. oAW has some support, because it can add to the end of an already existing file.

The insertion points proposal

The idea is to make the use of templates more flexible. For this we need to be able to insert text at different places within already generated files. I would like to explain my idea of how this can be achieved. Since we are still doing M2T, the result is text, without a direct way of finding out where to insert additional text. The first thing we need is a way to define what I call insertion points in the generated text. These point need to be marked in the generation template. For example, the Xpand template below has three such points marked with «INSERTIONPOINT».

«IMPORT statem»
 
«DEFINE root FOR StateMachine»
    «FILE implBaseClassFileName()»
        package «implPackage()»;
 
        import java.util.List;
        «INSERTIONPOINT ImportList»
 
        public abstract class «implBaseClassName()»{
        «INSERTIONPOINT FieldDefinitions»
 
        «FOREACH relevantStates() AS s ITERATOR i»	
        public static final int «s.constantName()» = «i.counter0»;
        «ENDFOREACH»
 
        public void trigger( int event ) {
            if ( dead ) throw new RuntimeException("this SM is already dead");
            switch ( currentState ) {
                «INSERTIONPOINT TriggerCaseBlock»
                «EXPAND caseBlock(this) FOREACH relevantStates()»
            }			
        }
        }
    «ENDFILE»	
«ENDDEFINE»

When new code needs to be inserted in the generated file above the template needs to not only refer to the file, but also to the insertion point in the file. This could look like:

«IMPORT statem»
 
«DEFINE extra FOR StateMachine»
    «FILE implBaseClassFileName() AT FieldDefinitions»
 
       «FOREACH events AS e ITERATOR i»
        public static final int «e.constantName()» = 1000+«i.counter0»;
        «ENDFOREACH»
 
    «ENDFILE»	
«ENDDEFINE»

This approach works out slightly different from M2M transformations. In an M2M transformation you have more freedom because you can always insert anywhere you like (the AST is fully available). The author of the Xpand template needs to explicitly mark insertion points, which can then be used. On the other hand, for some insertion points the AST information isnot enough and the author of the generator need to matk it anywaty. E.g. in the example:

• The ImportList and FieldDefinitions insertion points can be found in the AST
• The TriggerCaseBlock insertion point cannot be found in the AST. There might be multiple switch statements and it isn’t clear from the AST which one is handling the triggers.

Implementing Insertion Points

The changes to the Xpand language seem relatively small. The implementation will be slightly more elaborative. You need to be able to lookup insertion points in generated files and write new text at that location in a file. There are several ways of keeping track of the insertion points.

• Write the insertion point explicitly in the generated file? When needed, the insertion point can be easily found by a string search. This isn’t foolproof, because the same string may occur in the generated text as well. Also, the insertion points need to be commented out to not affect the generated result. However, commenting out depends on the target language and might not even be possible for some targets types. Alternatively the insertion points can be cleaned up from the generated files after the whole generation process has ended. We then need to be sure when the end is reached (nested workflows might be tricky) and we need to keep track of all generated files with insertion points.
• Keeping the index of the insertion points in the generated files in administration. When an insertion point is needed, it can be directly found in the administration. Need to take care of updating these locations when text is being inserted. Advantages: not dependent on target language, much faster finding of insertion points and no leftovers after processing.
• Write all inserted text to special files and resolve all insertions at the end of the workflow
• ...

Open questions

• nested insertion points
• where to add when adding multiple pieces of text at an insertion point (e.g before or after the previous one)
• Should e reference to an insertion point use filename + insertion point name or just insertion point name.