Ink for show control

In some of these other essays, I’ve referred often to the Ink scripting language. I’m fascinated by Ink because it is an executable markup language - something that doesn’t require an understanding of most programming concepts in order to read. Typing is faster than drag and drop, easier to reason about, and easier to version. This makes it easier to produce the myriad amount of content necessary in order for a world to feel lived-in and alive.

The key insight here is that any immersive storytelling that lives inside a computer simulation requires a human to write all the content.1 The more content we can write in a coherent way, the more immersive the experience will be. Thus, the most important part of automated storytelling is having a suite of tools at concepts that make it easy to add more content. The goal is that each unit of content improves the overall story experience in a compounding way: One unit of content may make the story one unit better, but if you add another unit of content the story is now three or more units better.

Each bit of content we add to the overall story world needs to make sense in context-sensitive ways, without being burdensome on storytellers to keep all that context in their own heads. It should be enough to say “Here’s a scene that can happen if X, Y, and Z are true, but only if A has just happened and B has yet to happen.”

Storytellers will need to learn some technical syntax, but the goal is for this syntax to “fail safe”: bad syntax should never crash a flight (assuming a human flight director). Fully automated flights become possible after enough crews have run through the content to show that it can do the job.

Conceptually, this is like self-driving cars. Robotaxis today can only operate in spaces that have been fully mapped and virtually driven millions of times. Once you’ve put in that level of training, however, fully-automated driving is possible. Expand the amount of training, and you can theoretically achieve full autonomy. This is even more true for a flight, since you have a much more constrained environment than the real world.

I had previously advocated for a “deck of cards” metaphor for modeling all this story state. I still think that, conceptually, the model is robust. The trouble is that metaphors like decks of cards require a lot of information to be put into a single card. Defining that card, what happens on it, and everything else becomes rather opaque. It’s technical enough and “data structure-y” enough that a lot more thought would need to go into making script authoring fluid and intuitive, flexible and forgiving. YAML is none of that.

Having explored a YAML variant of the deck-of-cards metaphor in “Cards: A technical breakdown” and “Cards are for storytelling”, I’ve come to the conclusion that Ink is a better fit for show control than decks of cards.

Why the change of heart?

Ink can do everything our deck of cards metaphor can do with a few notable exceptions:

1. Ink is not strongly typed, meaning it’s easy to make mistakes. Fortunately, if we run Ink in an editor that is constantly parsing where we might be at in the code (like a souped up version of the Inky editor), we can catch these mistakes early on.

• Also, if we design the way that we run our Ink script in a way that is forgiving, it’s possible to tolerate a lot of mistakes when running a flight, assuming a human is in the loop to paper over those mistakes.

2. Ink does not allow for remixing content at runtime, at least not natively. So, the “expansion pack” concept takes more work to get right. I think that this constraint is a worthy tradeoff for now. Fortunately, I think there are still ways to solve for this limitation involving some text preprocessing before compiling our Ink script. This is discussed below in the section on hooks.

What we need out of Ink

1. A script that is valid, executable ink in the normal Inky editor
2. It should generally make sense to a non-technical reader, so that they can read it and understand the different ways that a scene might play out
3. It is structured enough to be machine readable, so that we can perform any show control task in an automated way, but falls back to human intervention when things go off the rails. It should be a helper, and not get in the way of telling a good story.

In order to do all of this, we need to extend Ink’s syntax with some extra conventions. Everything is built to be right at home with Ink’s existing syntax. This means you can perform any number of branching decisions and advanced scripting, or you can just write a simple script that flows top to bottom. Any script should be playable in default Ink tools, such as the Inky editor. When dropped into a simulator, however, it should be able to assist in running the flight.

Building from that foundation, we have a few new concepts that extend Ink for show control.

Cues

A cue is just what it sounds like. Every thing that happens in a scene is a cue. As far as Ink is concerned, it’s just a line of text that looks like this:

Tex: Engine room to the bridge! Engine room to the bridge!
Tex: This is Tex, can you hear me up there captain?

Those are valid cues. It should be pretty self-explanatory, too. Here’s something a bit more complex:

Short range comms: Hail #freq:General Use #from:Panau Warship

Reads like a social media post, doesn’t it? It should be obvious what this is for, too. It is both human and machine readable.

In an actual flight, this line of text could be presented to the flight director as a cue for them to follow. However, this line is structured enough for us to teach the computer to parse, understand, and execute.

The default action to take with each cue would be to present it to the flight director in a sort of “teleprompter” box on the core computer. Ink choices would be presented there as well, as buttons they could click on to advance to the next step. It’s not inconceivable that this teleprompter could watch for this “hail” to be triggered by the flight director, and that cue could then be marked as complete. It’s also not inconceivable that the computer could execute this cue automatically. Whether we run in a manual, semi-automatic, or fully automatic mode, the cue should just make sense.

More exploring on the idea of a flight director teleprompter in the essay “All you need is a teleprompter”.

Multi-line cues

“That’s great,” you might be thinking, “for simple stuff. But what about when things get more complex?”

Some cues might require more input than we can squeeze on a single line, or it might require so much detail that things start to get really difficult. And, you’re right! If a cue is too complex for the computer to work with, it can just print what it sees on our teleprompter and call it a day. Programmers or story writers could then explore how to expand this system to handle more complex cues.

Here, for example, is a multi-line cue:

Long-range message: $$$ #from:Starbase 12 #encrypted

Voyager, please return to Starbase 12 immediately.
The Admiral needs to share some intelligence with you in-person.

Starbase 12 out

>>> $$$

A multi-line cue starts out the same as any other cue. The key difference is that $$$ symbol. We use that as a hint to the computer that this cue needs content that will come on a later line of text. The computer will then read until it finds that >>> $$$ symbol. Everything it sees from the cue to the chunk hint is used for that cue content. We call these “chunk hints”. The end of the chunk hint has the >>> (which could be read as “Into”) and $$$ (which is the name of the chunk hint). Altogether, it reads as “Put (Whatever came previously) into (This chunk)”.

If a cue only needs one chunk, we can actually omit the $$$ from the cue line itself if the cue line is missing a chunk of content:

Long-range message: #from:Starbase 12 #encrypted

< Message here >
>>> $$$

// This next one is malformed because we have "Some message here"
// And a chunk, too.
//
// The "content" parameter for our cue is basically filled in twice.
//
// The system would tolerate mistakes like this,
// printing each line to the teleprompter, and
// logging a warning when it sees the unexpected `>>> $$$`

Long-range message: Some message here #from:Starbase 12

< This chunk will not be read into that cue >
< but instead each of these lines will get printed by the teleprompter >
>>> $$$ // This line will emit a warning

If a cue needs more than one chunk, we use a slightly different syntax:

Long-range message: $Content$ #from:Starbase 12 #cipher:$Cipher$

This is a message
It can span multiple lines.
Such a great haiku

>>> $Content$

// This is the start of another chunk, where we describe
// how to substitute the letters in our message
A=1
B=2
C=3
D=...

// You get the idea
>>> $Cipher$

These are “Named chunk hints”. When we run this script, we’ll see the cue, and we’ll also see the chunk hints. We will then read each line of text until we fill up those chunk hints, and then we know what content belongs in which parts of our cue.

Cue tags

Ink supports tagging lines, and we’ve seen that already in a few places. They work just like hashtags, though they can also include spaces. A tag can work like a single label, or we can use a colon to provide the name and value for a tag. More complex syntax can be built from there, including predicates like #where:key=value. So long as you don’t put another # in the tag, you can make these tags as long as you like.

As shown previously, named chunk hints can be used to create multi-line tags:

// Note that this query syntax is pseudocode, but I suspect something
// like this will be necessary in a working system.
Get: #component:thing #where:$Where$
  (
  key=value and
  another_key > value
  ) or foo=bar
>>> $Where$

Cue labels

Sometimes, it’s helpful to label a cue so that we can refer to it later in the script. This is particularly useful for hints, so that choices can refer back to longer chunks of text to suggest why we might pick one choice versus another.

Trader: So, Captain, do we have a deal?

Hint (Accept):
  The crew accepts the offer.
>>> $$$

Hint (Reject):
  Crew: Nah, we want more money.
  Weapons: Acquire target
>>> $$$

Hint (Stall):
  Wait: 10 seconds
>>> $$$

* [Accept]
  Trader: Very good, captain. I await your payment.
* [Reject]
  Trader: You can't be serious. I suppose I should find someone with better taste than you to do business with.
* [Stall]
  Trader: I don't have all day captain. What will it be?

Those cue labels allow us to track a list of hints, and then refer to those hints later on in our choice script.

How might we execute cues?

Up until this point, the assumption has been that each cue is simply displayed to a flight director. By default, that’s actually all that this system does! This is a subtle but important feature: if the system doesn’t know what to do with a line of dialog, it should just print it out to a staff member and move on. This allows humans to get a nudge in the right direction at best, and if it’s really failing it might elicit an eyeroll before we move on to whatever comes next. The goal would be to correct any issues that crop up after a flight so that the next time through has fewer eyeroll moments.

When the game engine runs the Ink script, it will read out each line at a time. It will apply the previously described rules for things like building up multi-line chunks and collecting tags. It will then look at the part of a cue line that comes before the first colon, optionally parsing out the cue label (if one exists). At this point, with Ink that looks like this…

Long range message: #from:Starbase 12 #encrypted

Captain, what do you see in orbit around that moon?
Report back immediately.

>>> $$$

We get a data structure that looks like this:

{
  "cue": "Long range message",
  "content": "Captain, what do you see in orbit around that moon?\nReport back immediately.",
  "tags": %{
    "from": "Starbase 12",
    "encrypted": true
  }
}

We can now attempt to execute it by looking up the “cue” in a dictionary of cues we know how to execute. If we find a matching “cue executor”, we can call it with that blob of cue data and let it play out. If it fails, we can fallback to printing the cue out on the teleprompter and log the error. We may fly with a “verbose” teleprompter that prints each cue out. If the cue gets executed, we can note that on the teleprompter along with what was done.

Cues that are not found in our lookup table will just be printed out on the teleprompter. We might do a brief check to see if the name of the cue is close to other cues we know how to execute and, if it is, we might note that as a warning, like “Didn’t find cue “Long range message”, did you mean ‘Send LRM’?”.

Queries

Queries are things we can use to retrieve information from other game state. These queries are actually just cues! The output of the code that executes the cue will be set to a special variable in the Ink runtime called it. The Ink script can then refer to that value in whatever way makes sense. Other cues can read from previous cues if those cues have a label.

If a cue doesn’t end up getting executed, then a default value will be used instead:

// This will go look in our ECS simulation for an entity
// with a shield component
// and a label component with the value "Panau Warship"
// and it will return the value of the shield component
Get (Panau warship shield strength): #component:shield #where:label=Panau Warship

Tex: It looks like the Panau Warship's shield strength is {it(10)}%.

// That line, when evaluated in Inky, would read like this:
// Tex: It looks like the Panau Warship's shield strength is 10%.
// We could then present different choices,
// based on how strong the shields are.

// However, because we ran that cue for Tex's dialog,
// we need to use the full name when we refer to it later.
// Every cue overwrites the `it` variable, even if the output is empty.
// We are also going to read our own shield strength,
// so we need to be able to tell the difference between the two

Get (Our shield strength): #component:shield #where:label=Voyager

Tex: Our shield strength, on the other hand, is {it(100)}%.

{get("Panau warship shield strength", 10) < get("Our shield strength", 100)}
  -> decide_if_we_should_spare_them_or_blow_them_up
{get("Panau warship shield strength", 10) > get("Our shield strength", 100)}
  -> we_are_in_trouble

// Of course, we can also use Ink variables to stash stuff when we need it:

Get (Our torpedo count): #component:torpedo_count #where:label=Voyager
~ temp our_torpedo_count = it(5) // We can now use this variable in our Ink script

More complex queries may require additional thought than what we can offer here. The primary goal we’re optimizing for is ease of authoring the main script, and enough support to be able to augment that script with cue executors as needed. Again, the fallback is to print the cue out on the teleprompter and allow a human to interpret and intervene as needed. None of this should ever block the story, we should always be able to keep moving forward.

Bringing it all together

Here is an example script that weaves all of this stuff together.

// This is some preamble that I'm including
// so that this script is fully executable in Inky.
// You can ignore it.
CONST use_it_default = true
VAR it_value = ""

-> encounter_with_panau

// This is where our story begins
= encounter_with_panau

// This next line could trigger a number of things, depending
// on how the game engine is configured. At a basic level,
// it could just add a hail to short range comms.
// That could trigger a sound effect,
// or a line of dialog from the main computer that "just happens"
// every time we open a hail.

Short range comms (Panau warship call): Hail #freq:General Use #from:Panau Warship

- (opts)

// Choices can either have a single hint, like this...
* [Short range comms: Connected?] -> demand_rool_back
* [Wait: 20 seconds]
  Tex: Captain, don't you think we should answer the call?

  // A bit of a cop-out line, but whatever the crew says here
  // would trigger the next line of dialog.
  // In reality, crews might say any number of things,
  // and we could use the "hints" system described later to
  // try and account for them all.
  * * [Crew: *]
    Tex: Well, if you say so. You are the captain, after all.

* [Wait: 1 minute]
  // This get_or_blank function is the same as `get` but, as it says,
  // we have no default value.
  // We return an empty string if we can find what we're trying to get.
  Short range comms: Cancel Hail #hail:{get_or_blank("Panau warship call")}

-> DONE

= demand_rool_back

Bounty hunter:
I am Panau! I insist you return that escaped
slave to me this instant! He belongs to me!
>>> $$$

- (options)

Hint (Protect Rool):
    // Each of these is a cue. Any that evaluate to true will be used.
    Weapons: Fired?
    Short range comms: Disconnected?

    /*
    These are fun: if we are performing speech-to-text
    on what the crew is saying on the bridge,
    then we can try and match what they say to any of these "Crew" lines.
    The #confidence tag lets us set a confidence threshold
    for speech recognition. If recognition is at least 90%
    confident in a match on this text, then it's valid.

    Note that we are nesting the `Crew` hint with its own multi-line block here.
    */
    Crew: #confidence:90
      Where we come from, we don't believe in slavery. You can't have him!
    >>> $$$

    Crew: No way! You can't have rool!
>>> $$$

Hint (Betray Rool):

    /*
    This line opens up a potential narrative branch,
    where we set up a transporter lock and insist
    on them lowering their shields. But,
    we could then use that to launch a surprise attack
    on the Panau warship.

    Tex or other NPCs could also stick their nose in to
    this branch to try and steer us towards that main thread.
    */
    Transporter: Targeted? #destination:Panau Warship
    Crew: The safety of my crew matters more than this stranger, take him back!
>>> $$$

Hint (Stall):

    Wait: 10 seconds
>>> $$$

* [Protect Rool]
  -> engage_in_combat

* [Betray Rool]
  -> abandon_asylum_seeker

* [Stall]

  Bounty hunter: What will it be captain? Stay, or go?
  -> options

== engage_in_combat
Bounty hunter: Very well.

Short range comms: disconnect

Wait: 10 seconds

SFX: red_alert.wav #loop

Get (Our shield status): #component:ShieldStatus #where:label=Voyager

Computer: Warning: weapons are locked on this vessel. {it(false) == true:
    Shields are up.
- else:
    Shields are done.
}

// This is a hook. It's how we can remix user-generated content into an Ink file.
// More on this later.
>>> bounty_hunter_attacks <<<

-> DONE

== abandon_asylum_seeker

Bounty hunter:
*Chuckles* Very good, captain. Transport the prisoner immediately.

You have 5 minutes to comply before I destroy your ship
and take you and your would-be "asylum seeker" with me back to the markets.
>>> $$$

Short range comms: disconnect

-> DONE

=== function it(default)
{use_it_default:
    ~ return default
- else:
    ~ return default
}

Frontmatter & Content packs

Admittedly, we’re diving into more technical details here. I hope that tools for authoring Ink scripts like this would help assist with some of these more technical details for a non-technical audience. Even so, there’s some room in here for some of the “meta” tasks that we need to perform over chunks of story content.

We can put frontmatter in a file to help describe what a piece of content is, and how it fits into a project. Frontmatter is YAML and can store whatever data you want about the file or a chunk of content. This could be used to prevent certain content from being included in a final story file.

For example, suppose we have a story that needs to adapt to different audiences. We can have a basic story structure that will take us through a 2 hour version of the story for a field trip. Some of that content may only make sense in a field trip. Other content may make sense for an after school flight, a five-hour flight, a summer camp, an audience that’s done the story before, and more.

When we go to compile the story for a given flight, we can indicate which content ought to be enabled/disabled from the material we have available. Let’s say we have tags for “Field trip” or “Birthday party” or “Summer camp”. When we go to compile our flight, we should be able to mark tags as required/disallowed/ignored, and use that to select the content we want to include in our build process. From there, we can compile the story to verify it doesn’t have any glaring errors. If it passes, it could then be used to start a flight.

---
id: this-file-is-tagged-for-summer-stories
tags:
- summer-story
---

Tex: Captain, what's going on?

Each file with frontmatter could then be added to a “content pack”.

Content packs

Content packs are bundles of Ink scripts, images, videos, sounds, and cues. Other systems might call them “plugins”. The goal at the end of it all is to be able to bundle a unit of storytelling in a way that we can inject into a story without too much trouble. Multiple content packs should be able to be combined and remixed together at runtime.

Content packs should be able to include:

• Ink files
• Arbitrary assets like images, videos, or sounds
• Cue executors

The content pack is just a folder with one or more resources in certain folders, and an optional Manifest file for metadata about the content pack. Files are automatically detected based on the folder structure.

For example, an Ink script in a content pack may want to play a sound effect.

• ink/some_file.ink
• assets/arbitrary/path/some_audio.mp3
• cues/sfx.js

// ink/some_file.ink

SFX: #file:@assets/arbitrary/path/some_audio.mp3

That @ symbol is a hint that our compiler would look for and replace with fully qualified URLs that cues can resolve to locate a file included in the same content pack. So, that same script once compiled might look like this:

// ink/some_file.ink after being compiled

SFX: #file:content://content_pack_id/assets/arbitrary/path/some_audio.mp3

Replacements would only happen if the @ symbol is followed by a valid file path for the current content pack. Fully qualified URLs could be inlined when accessing a file in a different content pack. Cues get to decide what to do if a URL doesn’t resolve to something.

Cues in a content pack would be loaded and registered by the runtime by convention.3 In this case, the SFX.js file would register an SFX cue. When running the game, we would choose the cues we want to run with. If we don’t know what a cue is, we default to the teleprompter cue.

Injecting/Remixing into Ink

Ink isn’t designed for user-generated content extending a story. It’s designed to be compiled ahead of time. In order to extend Ink (including remixing content from or accross content packs), we need to compile it after pre-processing.

Part of this processing is some special notation I’d call “Hooks”. Hooks allow one piece of content to “invite” zero or more other pieces of content. When compiling a complete story to execute for a flight, we would scan for invitations and hooks and glue them together. Invitations that don’t have corresponding hooks would be replaced with a blank line when compiled.

---
id: main_story
---

= some_knot

Here's all this stuff.

* Some choice here
    -> go_there
* Another choice
    -> another_knot

/*
This next line is a "hook invitation" where we can inject
multiple lines of code from another file.

It gets replaced with zero or more lines of code from other files.
*/

>>> this_is_a_hook <<<

Another file that will be concatenated into the final story:

---
id: this_could_extend_another_file
type: hook
---

// This is the notation for targeting a hook block
>>> main_story.this_is_a_hook

* [Something goes here]
    -> side_story_knot
* [Another thing goes there]
    -> side_story_knot.other_route

// This is the notation for the end of the hook block
<<<

>>> main_story.this_is_a_hook
// We could define multiple hooks in one file if we wanted to
<<<

// The rest of this syntax won't be injected into the hook block,
// but instead will be concatenated into the final story file.
// Ink should compile it all the same, meaning that the hooks above can
// refer to these knots, even though they will end up being
// split up during compilation.

= side_story_knot

This knot will end up in the final story file.

-> DONE

== other_route

More options here.

-> some_knot.keep_going

Hooks make it possible to take different bits of content, author them all separately, and stitch them together into a single story file that can then be compiled and executed by Ink.

This approach aims to replicate the “deck of cards with expansion packs” metaphor, with the inherent flexibility of Ink. Each of these hooks is like an expansion pack card, and the hook invitations are how we indicate where expansion packs can extend a given story.

This makes it possible for a flow of story to remain coherent without interference from any story extensinos, since we can explicitly invite new content into the story wherever it makes sense.

Content pack resource resolution using the @ notation would still apply to hook content. Each hook would be resolved relative to its own content pack before being injected into the hook invitation.

Authoring cue executors

As noted previously, each cue is executed by a function that takes in a parsed cue as an argument, does whatever it needs to with that input, and maybe returns a result that is stored in the it variable and possibly in a named output that can be retrieved using get. Executors would be written in whatever host language makes sense for the engine embedding this flavor of Ink. When being loaded from a content pack, the engine would need to know how to interact with a given cue script type (e.g. files ending in .js or .lua or .exs).

Because we want to have tight feedback loops when editing a story versus actually running it, each executor might have a “sandbox” mode and a “flight” mode, where the sandbox mode is used for editing and the flight mode is used for running the story with an actual crew.2 This would allow for authoring to run in a tight loop, and for us to preview the story when authoring.

Imagine sitting on the bridge of your ship in the captain’s chair with a laptop, iterating on a dramatic scene in your next story. You add some cues to set the lights and music, and author some dialog from a hostile character. When you hit “play”, the simulator reacts to each cue in order, just as you wrote it. If you make some edits, you can rewind and replay: the simulator gets reset back to the initial conditions you might have set for the scene, so that you can preview new changes as you craft them.

Repeat this for each scene and segment of the story, and before long you’ve got a fully fleshed out narrative that will run on its own with a real crew.