RPGDXThe center of Indie-RPG gaming
Not logged in. [log in] [register]
 
 
Post new topic Reply to topic Goto page Previous  1, 2, 3, 4  Next 
View previous topic - View next topic  
Author Message
Hajo
Demon Hunter


Joined: 30 Sep 2003
Posts: 779
Location: Between chair and keyboard.

PostPosted: Mon Mar 15, 2010 11:17 am    Post subject: [quote]

Lex/Yacc (or Flex/Bison) really are incredible tools for helping in compiler and interpreter creation. You can focus on the syntax, and the code production/interpretation, and all the tokenizing and parsing is done by these for you.

A physicist once said "We stood on the shoulders of giants", when asked about a recent success. Don't start at the bottom. Use what the people already created, and then build something more on top of that.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Tue Mar 16, 2010 3:11 am    Post subject: [quote]

OK I've definitely come to the conclusion that this system is WAY non-portable to any language which does not permit variable length arrays of arrays. The reason for this is that it does not place an arbitrary limit on the size of an NPC's script, therefore using 2D arrays of determinant length is stands to be wasteful. As such, either variable-length arrays or member arrays of object arrays are necessary features for languages which host this system. Although workarounds are possible, they are WAY mind bending and NOT recommended.

Variable length arrays are very important to the creation of a flexible GCS. In a GCS, you have a collection of NPCs, each of which has its own script. To process these scripts, they must first be parsed. It is preferable to parse scripts before the game loads, or else the game may bog down under the burden of processing the script. Instead you want to have all these scripts tokenized, even compiled, before the NPCs are loaded. (or, in the case of a real-time script, before the updated NPC is loaded). Actually, there is no real issue with loading NPCs that have a limited number, because then you can just loop through a set of predefined arrays, one for each NPC pre-provided for. ZZT did this, I think, as has most every other NPC scripting system ever made. But as this GCS places no explicit limits on its NPC counts, we don't have the luxury of predetermination. So you might want to think twice before trying to implement this system in a procedural language.

In the interest of portability to procedural languages, I had been doing workarounds. For example, I had been stocking all the tokens for all the NPCs in one array, and the line beginning/end points in another array, and was using tracking variables to keep track of both. I was a serious cerebral exercise about on par with BrainFuck programming, if maybe a little more productive. But I don't recommend it. I have determined this level of portability to be unnecessary, and am redoing the scripting system using variable length arrays.
Back to top  
toturi
Pretty, Pretty Fairy Princess


Joined: 09 Feb 2008
Posts: 9

PostPosted: Tue Mar 16, 2010 8:17 am    Post subject: [quote]

LordGalbalan wrote:
Variable length arrays are very important to the creation of a flexible GCS. [...] So you might want to think twice before trying to implement this system in a procedural language.


Variable length arrays may not be native data types in procedural languages (assuming you are talking about "C/C++"-like languages), but that's what STL and numerous other 3rd party libraries are for. If you're not a fan of STL, you can always write your own set of list functions or vector class.

LordGalbalan wrote:
In the interest of portability to procedural languages, I had been doing workarounds. [...] I have determined this level of portability to be unnecessary, and am redoing the scripting system using variable length arrays.


So you want to make the code portable to other languages, but you don't want to make it "too" portable? Sense. It makes none.

I tend to see most programs as things that manipulate data. If your data structures are defined in such a way that it's easy to see how they interact with each other and how the application makes use of that data, it's usually not too difficult to port the logic to other programming languages.
Back to top  
Hajo
Demon Hunter


Joined: 30 Sep 2003
Posts: 779
Location: Between chair and keyboard.

PostPosted: Tue Mar 16, 2010 9:39 am    Post subject: [quote]

LordGalbalan wrote:
In a GCS, you have a collection of NPCs, each of which has its own script. To process these scripts, they must first be parsed. It is preferable to parse scripts before the game loads, or else the game may bog down under the burden of processing the script.


My old H-World engine did not load everything at he beginning. There was no problem loading it later - but! - it lacked integrity checks and sometimes something failed halfways through the game because some scripts had a problem, and it was not discovered until the script actually was run.

So yes, I agree that scripts should be loaded at start, but not for efficiency reasons, but for syntax and integrity checks. These are really game breaking problems in a deferred loading system.

Also, besides linked lists, queue and vector types, you might want to look into hashmaps/hashtables, because they are a very flexible sort of data container that I found very useful in generic game engines. Sometimes those are also called "associative array". I'm not quite clear about why you think such are not usable in a procedural language - admitted you should use a modular approach with factory module for the types, but I saw very nice procedural implementations of those.

Edit: I googled a bit and found this fairly soon:

http://codingrecipes.com/implementation-of-a-vector-data-structure-in-c

One could make this more secure, but it actually is a fairly fine vector type implementation in plain C.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Tue Mar 16, 2010 4:54 pm    Post subject: [quote]

Using Javascript-specific features, I turned this:
Code:

function buildCodeLinesDifficult () {

// This function builds code lines from parsed tokens. The lines are recorded by reference to the token indices whereat they end.
// This is a very complex, abstract system, especially in that it stores an array index in an array index. The complexity occurs when moving
// the tokens, which in return requires changing the end indices of the lines to reflect the new positions of the tokens in the token array.
// However, this puzzle is easier to solve then it appears: all that is required is to displace the end points by the scale of the displacement
// of the end token in the array. For example, if we remove all the tokens respective to a given actor from the token array, then all the tokens
// of the actors after that actor will be displaced by the sum of all tokens in the actor's code. This means that the line end points respective
// to each of those actors will also need to be collapsed by that same margin.

   codeLines = new Array(0);
   lineEndsListIndex = 0;

   for (index = 0; index < tokensLimit; index++) {
   
      if (tokens[index] == lineTerminator) {
   
         codelineEndsList[lineEndsListIndex] = index;
         lineEndsListIndex++;
      }
   }
   
   lineEndsListLimit = linesEndsIndex;
}



function addCodeLines (actor) {

// This function attributes a series of code lines to an actor. It keeps a record of the point at which these lines are entered into the lines array.

   actor.lineEndsListLimit = lineEndsListLimit;
   actor.lineEndsListStart = actors.codeLineEndsList.length;
   actor.lineEndsListEnd = actor.lineEndsListStart + actor.lineEndsListLimit;
   
   index = 0;
   
   actors.codeLineEndsList.concat(codeLineEndsList);
   
   /* LONG
   for (listIndex = actor.lineEndsListStart; listIndex < actor.lineEndsListEnd; listIndex++) {
   
      actors.codeLineEndsList[listIndex] = codeLineEndsList[index];
      index++;
   }

}


function removeCodeLines (actorIndex) {

// Procedure: subtract the line count for the removed actor from each line start after the actor's last line on the actorCodeLines table.

   actor = actors.data[actorIndex];

   for (index = actorIndex + 1; index < actors.limit; index++) {
   
      lineEndsListStart = actors.data[index].linesEndsStart;
      lineEndsListEnd = actors.data[index].linesEndsEnd;
      
      for (lineEndsListIndex = linesEndsStart; lineIndex < linesEndsEnd; lineEndsListIndex++) {
      
         actors.codeLineEndsList[lineEndsListIndex] -= actor.lineEndsListLimit;
      }
   }
   
   actors.codeLineEndsList.splice(actor.lineEndsListStart, actor.linesEndsLimit);
}


function removeActorProgram (actorIndex) {

   actorTokens.splice(actor.tokenStart, actor.tokensLimit);
}


function addActorProgram (actorIndex) {

   actor = actors.data[actorIndex];
   actor.tokenStart = actorTokens.length;
   actorTokens = actorTokens.concat(tokens);
   actor.tokenEnd = actorTokens.length;
}*/


into this:
Code:

function buildCodeLines (actorIndex) {

   actor = actors.data[actorIndex];
   tokensLimit = actor.tokensLimit;

   codeLines = new Array(0);
   linesIndex = 0;
   
   codeLines[0].start = 0;

   for (index = 0; index < tokensLimit; index++) {
   
      if (tokens[index] == lineTerminator) {
   
         // Don't include the line terminator.
         codeLines[linesIndex].end = index - 1;
         linesIndex++;
         // Get the beginning of the next line
         codeLines[linesIndex].start = index + 1;
      }
   }
   
   linesLimit = linesIndex;
   
   actor.codeLines = codeLines.slice(0);
   actor.linesLimit = linesLimit;
}



No real comparison. Having to put all the arrays into one data structure involved substantial overhead and made, for me, the the design of those few algorithms day-long affairs. Not to mention debugging would be impossible for me to fathom. Adding the code lines dimension in particular made the situation profoundly complicated. The sheer scale of indirection overwhelmed me.

I think it would be better to focus less on code portability and more on actually providing modern language features to different platforms. Then the portability matter fixes itself.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Sat May 22, 2010 4:48 pm    Post subject: [quote]

OK I've got the inventory system done... now for the actual integration of the script... (this is gonna be a bitch... always is).
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Mon May 31, 2010 5:03 pm    Post subject: [quote]

The script system is proving quite a complicated beast. The main complication is the event system. The approach I'm taking involves processing the declaration of an event and immediately after the declaration, looking up its subroutine for later reference. It's just too slow to search for the subroutine every time the event is called. (though for completeness' sake I'll probably do a slow version as well).

Events use the BASIC "ON" keyword:
Code:

ON CONTACT WITH King GOTO GetSwordFromKing


Assuming GetSwordFromKing begins at line 20, on every contact with the actor named "King", code begins executing at line 20.

The idea is to register every event declaration in an array of its own, with an associated jump point, and cycle through these whenever an event of a given type is flagged. The question is how to organize these elements for maximum efficiency.

Code:


contactEvents[0].actor = "King";
contactEvents[0].start = 20;

if (contactEventFlag == true) {

    for each event in contactEvents {

        // do something
    }
}



The question is how to record the events. I believe the norm is only to process a few events at a time, on a first-come first serve basis. That's the fastest method. The slower method involves keeping an ever expanding record of the events, using either strings or dynamic arrays. SNES games in particular seem to have used the former. (for example, if the player gets hit, they become invincible for a short time, thus eliminating the possibility of additional hits.). What I'd like to do is record the events by the index of their actors in the set of all actors. Each actor would have a property (or property array) which kept track of the actors they interacted with during a given tick of the game clock. This record would be expunged at the end of every game tick.

Most early games used contact as the sole mechanism of event. Zelda, for example, used contact as its primary game mechanism, using context to determine what to make of it. Each NPC represented a different context. Zelda II allowed the player to create their own events through talking. Zelda III used state machines to modify the expression of events (for example, the clanging-sword effect when Link's charged-up sword makes contact with an enemy's hand weapon). I'm aiming for something about on par with Zelda III's approach, with event subroutines for every NPC. I think I'm going to add a type attribute to NPCs to simplify this process.
Back to top  
Hajo
Demon Hunter


Joined: 30 Sep 2003
Posts: 779
Location: Between chair and keyboard.

PostPosted: Tue Jun 01, 2010 8:01 am    Post subject: [quote]

tcaudilllg wrote:

The question is how to record the events. I believe the norm is only to process a few events at a time, on a first-come first serve basis. That's the fastest method. The slower method involves keeping an ever expanding record of the events, using either strings or dynamic arrays.


Usually events are kept in a queue. You don't want events to be lost. The event handler (the code that reads from the queue and takes action based on the events) may decide not to act on certain events, but the queue should keep them all.

A queue can be implemented as dynamic array, but also as a linked list.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Tue Jun 01, 2010 9:43 am    Post subject: [quote]

Yeah a linked list is what I'm going to have to use.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Thu Jun 03, 2010 7:41 pm    Post subject: [quote]

OK here's what I've got so far:

  • at runtime, all the events that an actor is capable of are read into the actor's data structure.
  • as events occur which involve an actor, they are loaded into its queues. The actor has a queue for each type of event.
  • At the beginning of every cycle, the queue for an actor is read. The queue contains a reference to a pointer table which contains the index of the involved actor and the index of the event as it lies in the table.


The disadvantages of this method are the overhead of managing the actors. I'm thinking of just getting rid of labels and doing it the VB way. That way it would just be a matter of queuing actors and running through the list every time, jumping to the subroutines registered to each actor in the queue. But I still need the indirection table regardless, otherwise it becomes impossible to delete actors without collapsing the system. Although, you could argue that actor deletion in unnecessary as well, that the designer should have a plan for the entire scene.

Maybe what's needed are actor identity libraries, which load actors into pre-declared slots.

Change vs permanence... which to use...?
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Sat Jun 05, 2010 11:53 pm    Post subject: [quote]

After contemplating the situation for a week, it would seem that either approach has definite downsides. The "slot" system has the downside of being very unwieldy: no matter what, you need an array index for every actor on the board PER actor. So if I've got a thousand actors on the board, that a million array indices just for the actors. Talk about a memory hog.

The other route spares memory, but is very CPU intensive: it requires looking for a match in the event tables for each actor in the queue. Actually, this is probably the best technique to use all things considered, because it allows the user to more easily manage the complexity of the interaction system: to support more actors, use fewer events; to use more events, use fewer actors.

I'm kinda surprised though: despite our advances the potential for a truly vibrant world with thousands of independent actors remains out of the reach of consumers. (not counting MMORPGs, of course) It would seem that the PS3 generation has mostly obscured this fact by putting emphasis on graphics and realistic physics involving only a few elements, which combined with focuses for the player's attention distracts from the essential simplicity of the situation in-game.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Tue Jun 08, 2010 7:50 am    Post subject: [quote]

I've wrote a parser for BASIC code (in Javascript). I'd like you to check it out and tell me if there is anything wrong with it. Right now it does commas, string quotes, line extension characters, and whitespace trimming.

Code:

function parseScript (actor) {

   script = actor.script.body;
   
   for (index = 0; index < script.length; index++) {
   
      currentChar = script.charAt(index);
      
      // remove quotes
      if (currentChar == "\"") {
      
         if (quoteFlag == 0) { quoteFlag = 1; }
         else
         if (quoteFlag == 1) {
         
            quoteBody = quoteBody.split(" ");
            
            // change all the space characters in the quote to an unused code. Later, we will change them back.
            quoteBody = quoteBody.join(fromCharCode(1));
            quoteFlag = 0;
         }   // alternate method: read quotes and their locations into array elements.
      }
      
      if (quoteFlag == 1) {
      
         quoteBody += currentChar;
      }
      else { // do all work related to character spacing in here.
      
         // Everything in BASIC has a space before and after it, even if it doesn't. (later, expand this to include more characters).
         if (currentChar == ",") {
      
            if ((script.charAt(index - 1) != " ") || (script.charAt(index + 1) != " ")) {
                     
               middle = ",";
               if (script.charAt(index - 1) != " ") middle = " " + middle;
               if (script.charAt(index + 1) != " ") middle = middle + " ";
               script = replaceChar(script, index, middle);
               index = index + middle.length - 1;
            }
         
         }
         else // remove line extender characters
         if (currentChar == "_") {
            script = replaceChar(script, index, "");
            index--;
         }
         else // trim whitespace
         if (currentChar == " ") {
         
            if (index > 0) {
               if (script.charAt(index - 1) == " " || script.charAt(index - 1) == "") {
                  script = replaceChar(script, index, "");
                  index--;
               }
            }
            else // if we're at the first character, there shouldn't be any whitespace.... Trim until it's all gone.
            if (index == 0) {
               script = replaceChar(script, index, "");
            }
         }
      }
      
   }
   
   // parse lines

   actor.script.lines = script.split("\n");
   
   for (index in actor.script.lines) {
   
      line = actor.script.lines[index];
      actor.script.lines[index] = { };
      actor.script.lines[index].line = line;
   }
 
   // parse terms
   lines = actor.script.lines;
 
   specialChar = fromCharCode(1);
 
   for (scriptIndex = 0; scriptIndex < linesLimit; scriptIndex++) {
   
      line = lines[lineIndex].line;
      lines[lineIndex].terms = line.split(" ");
      lines[lineIndex].termsLimit = lines[lineIndex].terms.length;
      
      // Now for those quotes with space characters in them
      
      terms = lines[scriptIndex].terms;
      
      for (index in terms) {
      
         term = terms[index];
         
         for (charIndex in term) {
         
            currentChar = term.charAt(charIndex);

            if (currentChar == specialChar) term = replaceChar(script, charIndex, " ");
         }
      }
   }
 
}


function replaceChar (target, position, characters) {

   part1 = target.substring(0, position);
   part2 = target.substring(position + 1);
   return part1 + characters + part2;
}



Does anything seem left out?
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Tue Jun 08, 2010 10:39 am    Post subject: [quote]

Did a little more polishing...

Code:


function replaceChar (target, position, characters) {

   part1 = target.substring(0, position);
   part2 = target.substring(position + 1);
   return part1 + characters + part2;
}


function parseScript (programScript) {

   script = programScript.body;
   
   for (index = 0; index < script.length; index++) {
   
      currentChar = script.charAt(index);
      
      // avoid quotes
      if (currentChar == "\"") {
      
         if (quoteFlag == 0) {
         
            quoteFlag = 1;
         }
         else
         if (quoteFlag == 1) {
         
            quoteFlag = 0;
         }
         
         tokenAppendFlag = 1;
      }
      else
      if (quoteFlag == 1) {
      }
      else { // do all work related to character spacing in here.
      
         // Everything in BASIC has a space before and after it, even if it doesn't. (later, expand this segment to include more characters).
         if (currentChar == ",") {
      
            if ((script.charAt(index - 1) != " ") || (script.charAt(index + 1) != " ")) {
                     
               middle = ",";
               if (script.charAt(index - 1) != " ") middle = " " + middle;
               if (script.charAt(index + 1) != " ") middle = middle + " ";
               script = replaceChar(script, index, middle);
               index = index + middle.length - 1;
            }
            
            tokenAppendFlag = 1;
         }
         else // remove line extender characters
         if (currentChar == "_") {
            script = replaceChar(script, index, "");
            index--;
         }
         else // trim whitespace
         if (currentChar == " ") {
         
            if (index > 0) {
            
               if (script.charAt(index - 1) == " ") {
                  script = replaceChar(script, index, "");
                  index--;
               }
               else // guess this is the end of the current token, then!
               {
                  tokens[tokenLimit] = token;
                  token = "";
                  lineLength++;
               {
            }
            else // if we're at the first character, there shouldn't be any whitespace.... Trim until it's all gone.
            if (index == 0) {
               script = replaceChar(script, index, "");
            }
         }
         else
         if (currentChar == "\n") {
         
            scriptLines[lineLimit] = terms;
         
            for (index = 0; index < lineLength; index++) {
            
               scriptLines[lineLimit].terms[index] = tokens[lineStart + index];
            }
            lineLimit++;
            lineStart = lineStart + index;
            lineLength = 0;
         }
         else
         {
            tokenAppendFlag = 1;
         }
         
      }
      
      
   if (tokenAppendFlag == 1) {
      token += currentChar;
      tokenAppendFlag = 0;
   }
 
}




Much better!
Back to top  
Verious
Mage


Joined: 06 Jan 2004
Posts: 409
Location: Online

PostPosted: Tue Jun 08, 2010 3:11 pm    Post subject: [quote]

I would recommend using Regular Expressions to trim/normalize whitespace. It will greatly simplify your code and improve performance.
Back to top  
tcaudilllg
Dragonmaster


Joined: 20 Jun 2002
Posts: 1731
Location: Cedar Bluff, VA

PostPosted: Sat Jul 03, 2010 8:48 am    Post subject: [quote]

I implemented the parser (it's not the fastest thing ever (if you use lots of whitespace), but for now functional is enough) and tested it. It works fine.

The actor system provided a unique challenge: I had to first parse the code, then rearrange it. First I used the parser to sort the code into lines, which were defined by pre-parsed tokens. I then divided the actor scripting language into two parts with distinct semantics. The first part modularized the code by looking for commands that indicated the beginning and ending of modules. There are two kinds of modules, actor modules and event modules. Event modules are always contained within actor modules and define the relations between actors. At the moment, there are two events defined, "talk" and "contact". However, I intend to implement a "line of sight" event eventually.

When events occur, the 2nd part of the language is invoked to actually perform the tasks which carry out the effect of the event. I was very surprised that I needed to create two semantic processor functions; however, I needed different command sets for completely different purposes: it was necessary to modularize the code before working with it, which required the use of two completely independent systems. In essence, the language I have defined is two languages, not one.
Back to top  
Post new topic Reply to topic Page 3 of 4 All times are GMT
Goto page Previous  1, 2, 3, 4  Next 



Display posts from previous:   
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum