Data Manipulation Basics

[Context: HL Kit … Basic Concepts and Terminology]

Overview

In order to handle the complexities of every game system, the HL engine is a very sophisticated tool. However, we've invested significant time and effort to keep things as simple and streamlined as possible. The net result is that the Kit makes extensive use of a few powerful mechanisms that give you complete control over both how data is manipulated and when it is manipulated.

Given the volume of information involved in many game systems, data manipulation entails the proper sequencing of tasks. The first key mechanism is the evaluation cycle that governs when the data is manipulated. The second key mechanism is a fast and flexible classification system, called tag expressions, to identify which objects are to be manipulated and which manipulations should be applied to them. The final key mechanism is the scripting language that allows the actual manipulation of the data.

Each of these mechanisms is introduced in the sections below. An in-depth discussion of these mechanisms is provided in separate sections of the documentation.

Evaluation Cycle

In order to ensure that all data manipulation operations are applied in a correct and consistent order, the HL engine enforces a strict sequence of evaluation that you completely control as data file author. This sequence is triggered repeatedly as the user makes changes to characters, and it is referred to as the "evaluation cycle".

Whenever the user takes any action, HL automatically re-evaluates all facets of the portfolio that are impacted by the change. This updates all dependencies on the user's changes. For example, if the user modifies an ability score in the d20 System, all linked skills and dependent weapons are updated to reflect the impact of the change. To safeguard against lag time in the user-interface when the user takes multiple actions in rapid succession (e.g. clicking the '+' button to increment a skill rating a dozen times), HL waits until the user pauses for a moment before it initiates a new evaluation cycle on the portfolio.

Everything that is performed by the Hero Lab engine is done in a specific sequence, and the majority of actions occur during the evaluation cycle. Each of these individual actions is referred to as a "task", and the complete set of tasks comprising the evaluation cycle is referred to as the "task list". For virtually every task, the data file author controls the evaluation sequence by designating when each task should be processed. There are two criteria used to determine the scheduling of a task: phase and priority.

For each game system, the data file author defines a set of phases that dictate the general sequence in which evaluation is performed. Each phase typically corresponds to a logical step in the overall evaluation cycle, such as "initialization", "before level-based calculations", "after attribute modifiers", etc. All phases are ordered, thereby dictating the sequence in which the phases are processed during the evaluation cycle.

Every task is assigned a phase during which it will be evaluated. All tasks is also assigned a priority, which controls the order in which tasks within the same phase are processed. If two or more tasks have the exact same phase and priority, then the engine uses a number of rules to order them. If the two tasks are still scheduled for the same time, the engine is free to schedule them in whatever sequence it finds convenient, and this order may change from one evaluation pass to the next. Consequently, assigning the correct phase and priority is often critical to ensure that modifications are applied before or after subsequent tests are made that rely on those modifications.

The Kit provides an assortment of pre-defined phases that should serve as an excellent starting point for just about any game system. You are free to change or delete these phases, as well as add new phases to suit your needs. However, the set provided will typically work well for most game systems we've encountered.

NOTE! When the evaluation cycle begins, it continues until completion. This is usually transparent to the user, but it can become noticeable on older (i.e. slow) computers when the data files are highly complex. Therefore, it's best to utilize tag expressions whenever possible to limit the number of objects that must be processed during evaluation. Similarly, its typically faster to use tags instead of scripts when possible, because they are significantly faster to execute.

Tag Expressions

Tags form a fundamental building block upon which much of HL is constructed, and tag expressions are where they become of critical importance. Since the vast majority of objects you'll be managing are things and picks, there must be a way to identify the proper subset of these objects that apply to a particular situation. For example, attributes, skills, and weapons are used in completely different ways, so you want to keep them separate from each other – yet they are all things (or picks). The solution is to assign tags to each object and then use a tag expression (or tagexpr for short) to identify the subset of objects that apply in a given situation. A major (separate) section of the documentation is dedicated to the subject of tag expressions, but a brief overview is valuable at this point.

A tag expression is essentially a filter that gets applied to all objects of particular type (e.g. things or picks) and selects only the ones that meet the specified criteria. Tag expressions are Boolean expressions, which means they evaluate to a simple "true" or "false" result. They examine all of the assigned tags and determine whether those tags satisfy the expression or not. Separate criteria can be combined, allowing you to require that multiple criteria all be met, one of a set of criteria be met, certain criteria be excluded, or some combination thereof. For example, a tag expression could test whether a thing has the tag "Elven" from the "Language" group. Or a more complex tag expression could test whether a thing has the "Language.Elven" tag and also has either the "Race.Elf" or "Race.HalfElf" tag.

Since tag expressions can utilize full Boolean logic (i.e. "and", "or", "xor", "not") and can even extract and test numeric values from tags, tag expressions can model extremely complex conditions without difficulty. The bottom line with tag expressions is that they provide a powerful and flexible method for quickly determining whether to include or exclude an object, and they are based exclusively on the set of tags assigned to that object. As such, they are used extensively throughout HL.

Scripts

HL makes extensive use of scripts to allow the data file author substantial freedom and flexibility. In fact, scripting is such a fundamental and diverse topic that huge sections of the documentation are dedicated to various facets of writing scripts. This section merely provides a brief overview.

The scripting language syntax within the Kit is relatively simple. You can declare variables, assign values, perform simple conditional tests, and utilize a number of built-in intrinsic functions for various purposes. There is also a syntax that allows access to all of the objects within a given actors, such as the various picks and containers, plus the field values and tags that may be assigned to them. The language syntax itself is somewhat similar to the age-old Basic language. Using scripts, an author can pretty much do whatever is necessary to properly model the requirements of a given game system.

To make the writing of scripts easier, the Kit supports re-usable procedures. A procedure is nothing more than a mini-script that can be called from multiple places. The data files for each game system make extensive use of procedures so that many scripts can be reduced to simply calling one or two procedures to do all the work.

The starting data files provided by the Kit include numerous scripts and procedures that you can use, adapt, and extend according to the needs of the game system you're implementing.

Thing-Based Scripts

The most common type of script that you will find yourself writing the "eval script". Of all the different types of scripts, most are triggered in response to a specific action. However, an eval script is scheduled to occur at a specific phase and priority during the evaluation cycle, hence the name.

Every eval script is associated with a thing. This means that every pick derived from a given thing inherits all of the eval scripts for that thing.

Since an eval script is performed during evaluation processing, a separate task is always created for each eval script, which is then scheduled within the overall task list. If a thing is added to a container multiple times, each eval script must be processed separately for every pick. Consequently, separate eval script tasks are created and scheduled for every pick.

Eval scripts are the most commonly used script due to the ability to schedule them. So many facets of a complex game system are inter-dependent. Dependent calculations must be performed in a carefully ordered sequence to ensure that all of the game mechanics are accurately implemented within the data files. Eval scripts provide the means for this scheduling.

Thing-Based Rules

Sometimes, you don't need to apply a modification to anything and instead need to verify whether a required condition is satisfied. For example, the "Knowledge" skill might require that the user specify the domain to which the skill applies. This entails checking that the user has entered a suitable value.

To handle situations like this, the Kit provides "eval rules" as a companion to eval scripts, and you will likely find yourself writing a number of these as well. Just like eval scripts, eval rules are scheduled as tasks and performed with specific timing during the evaluation process. Unlike eval scripts, the purpose of an eval rule is to verify a condition, so an eval rule must always determine whether its condition is satisfied or not. If the rule is not satisfied, the corresponding pick is flagged as being invalid and an appropriate message is displayed to the user within the validation report.

Component-Based Scripts and Rules

There are many situations where you'll want the same script or rule to be applied to all things that derive from a particular component. For example, every piece of gear the character is wielding should be verified to not be stored in a backpack or some other container. The Kit makes it easy to handle situations like this by allowing you to actually define the script or rule directly on the component.

Whenever a script or rule is defined on a component, that script/rule is treated as if it were individually assigned to every thing that derives from the component. This means that a new task is automatically created and scheduled for the script/rule on every pick that is added to the character.

It is reasonably common to have both component scripts/rules and individual script/rules associated with a particular thing.