The Magic of Scripting!

Scripting is one of those things that always tend to get pushed aside by smaller developers. especially considering its takes something like 15 times longer for a script to compile a certain piece of logic than any functional programming language, developers usually just opt for the "program it yourself" option. but I think the usability and simplicity that scripting can bring to your engine will open it up to much more potential. Even though scripting does take a certain amount of logic, it is important to realize that an "OnCollision" function,(thanks unity) will make it far more accessible to your artists and designers than a huge block of code that does one type of collision detection.today I want to look at how a simple scripiting language could be implemented into a game engine, and what other advantages using such methods can bring.Lexical Analysisthis is the first step to implementing the scripting software in your engine. There are certain conventions that most programming languages are known to follow. ASCII letters and numbers white spaces, equality operators, commenting  etc. they all need to be functional in your scripting language which means you need to tell your engine what every piece of syntax is going to do.that sounds like hell right there doesn't it? typing out every known ASCII digit and telling your engine how to read it. but it turns out there are actual programs that most companies use to do it for them. one very popular example is the program LEX. which would read in a piece of code like this:

"if"     {return IF;}"="      {return ASSIGN;}";"      {return END_STMT;}{IDENT}  {Identifier ();             /* identifier: copy name */return ID;}{STR}    {StringConstant ();         /* string constant: copy contents */return STRING;}"//"     {EatComment();}             /* comment:    skip */\n       {lineno++;}                 /* newline:    count lines */{WSPACE} {}                          /* whitespace: (do nothing) */.        {return ERROR_TOKEN;}       /* other char: error, illegal token */

as you can see you have most of the expected tokens in most programming languages, but thats only the first step. now that I have the lexicon how am I supposed to use it?!The Parserthankfully there is yet another program to feed in all our potential "statements" and compare them with what they are supposed to do, such as an if statement or an expression. Following the unix train there is a program called Yacc that works similar to the Lex software I described.except this time it reads in statementsIF_STATEMENT is an if statementI know it seems redundant but imagine if you had a piece of code that said something likeif( a==b+c)your compiler wouldn't know how to read this because it isn't sure when to evaluate the boolean operator (which we know it wouldn't make sense to begin evaluating the boolean and then do the addition, but hey, computers are stupid and thats why you need a parser)up next?Symbols and Syntaxwhen we are using script the main idea is that it is easier. the code genereated from the parser is fine and dandy, but we want something to act as an intermediate to make the code more readable and more importantly fast to think about and type out. you could use a Syntax tree which breaks down the structure of say an "if" statement so instead of having the very computer generic COMPARISON_OPERATOR; INPUT_INTEGER_("A") BRANCH INPUT_INTEGER("B")we can juts say if A(B);heres an example of what such a tree could look like.so while this step isn't fully necessary it is definitely something that will help, especially if you are trying to make a tool for the use of non programmers. after you have generated this tree you will want to actually convert this into intermediate code. such that each Instruction will have a pointer to the next instruction(at least this is probably the best way to go about it)Semantic Checkingthis can get really tricky because you have to know all the different types of errors that may occur. while the compiler may be able to spot some of these for you, there will still be many things you are going to need; some sort of "error checking" function would be a good start. it is also common to see coercion from boolean to equality operators, but I think this is a bad idea, as it may generate confusion, and it is important to understand why what you are typing is wrong in the scheme of the system.Intermediate Code and The Virtual MachineThis is where the fun starts. I can't help but compare this to coding in assembly but its actually not all that difficult. (so they say), well you aren't really coding in assembly, you are actually creating the code that creates assembly(I know I know, this is NOT what you signed up for! well.. deal)here look at some code its actually not all that scary, this is what it would look like to genereate an IF statement to be processed into op code

case IFTHEN_STMT:// First, create the necessary code parts cond = GenIntCode (root->child[0]);jump2end = new IntInstr (OP_JMPF); // set target below thenpart = GenIntCode (root->child[1]);endif = new IntInstr (JUMPTARGET, jump2end);jump2end->target = endif;// Now, concatenate them all Concatenate (cond, jump2end);Concatenate (jump2end, thenpart);Concatenate (thenpart, endif);return cond;

as you can see it is just a block that first evaluates the node that sets off the if statement then we branch to the end (or JUMP if that is more exciting for you) if the condition was not met, or if the condition is satisfied we go to the -then- (like if you do your homework THEN you can watch TV)so a program like this will generate the intermediate code we read from our tree that was processed by our parser and push it onto the stack as OP code.OK HARD PARTS OVERnow you actually have to get your virtual machine to read the code you have generated, but thats not all too difficult. you just want something now to read the values on the stack.two functions- Read(); Execute();. that should be it.  you want to have some sort of data structure to store the value of the stack but other than that you are pretty much doneyou can also add a "reset" which will re-initialize the VM.now I will admit I have left out some important features such as optimization but I won't leave you up-stream without a paddle. many thanks go out to Jan Niestadt over at flipcode.com for providing me with invaluable information on how to do this sort of thing. please click the link and read some of the articles because it is an awesome resource with lots of downloadable coding goodies. I also learned alot from reading "Game Engine Architecture" by Jason Gregory. where he spends a whole section talking about scripting and the different types of scripts you can generate.