Theses

Xiangyu (Becky) Peng. Controlling Behavior with Shared Knowledge. Ph.D. Dissertation, , 2024.

Controlling agent behavior is a fundamental challenge across diverse domains within artificial intelligence and robotics. The central idea of this dissertation is that shared knowledge can be used as a powerful tool to control AI agents’ behavior. This dissertation explores the utilization of shared knowledge in constructing coherent narratives and enhancing the expression of shared knowledge in Reinforcement Learning agents. In this dissertation, I first investigate the utilization of shared knowledge for constructing narratives by developing a story-generation agent that emulates the cognitive processes of how human readers create detailed mental models, referred to as the “reader model”, which they use to understand and interpret stories with shared knowledge. Employing the reader model has resulted in the generation of significantly more coherent and goal-directed stories. I also explore how to input unique constraints into the story generator allowing for the modification of the shared knowledge. Subsequently, I delve into the application of shared knowledge in controlling reinforcement learning agents through the introduction of a technique called “Story Shaping.” This technique involves the agent inferring tacit knowledge from an exemplar story and rewarding itself for actions that align with the inferred reader model. Following proposing this agent, I propose the Thespian agent to leverage the knowledge learned in this technique to adapt to the new environment under a few-shot setting. Additionally, I investigate the potential of using shared knowledge to explain behavior by examining the impact of symbolic knowledge graph-based state representation and Hierarchical Graph Attention mechanism on the decision-making process of a reinforcement learning agent. The goal of this dissertation aims to create AI-driven systems that are more coherent, controllable, and aligned with human expectations and preferences, thereby fostering trust and safety in human-AI interactions.

Sarah Wiegreffe. Interpreting Neural Networks for and with Natural Language. Ph.D. Dissertation, , 2022.

In the past decade, natural language processing (NLP) systems have come to be built almost exclusively on a backbone of large neural models. As the landscape of feasible tasks has widened due to the capabilities of these models, the space of applications has also widened to include subfields with real-world consequences, such as fact-checking, fake news detection, and medical decision support. The increasing size and nonlinearity of these models results in an opacity that hinders efforts by machine learning practitioners and lay-users alike to understand their internals and derive meaning or trust from their predictions. The fields of explainable artificial intelligence (XAI) and more specifically explainable NLP (ExNLP) have emerged as an active area for remedying this opacity and for ensuring models’ reliability and trustworthiness in high-stakes scenarios, by providing textual explanations meaningful to human users. Models that produce justifications for their individual predictions can be inspected for the purposes of debugging, quantifying bias and fairness, understanding model behavior, and ascertaining robustness and privacy. Textual explanation is a predominant form of explanation in machine learning datasets regardless of task modality. As such, this dissertation covers both explaining tasks with natural language and explaining natural language tasks. In this dissertation, I propose test suites for evaluating the quality of model explanations under two definitions of meaning: faithfulness and human acceptability. I use these evaluation methods to investigate the utility of two explanation forms and three model architectures. I finally propose two methods to improve explanation quality– one which increases the likelihood of faithful highlight explanations and one which improves the human acceptability of free-text explanations. This work strives to increase the likelihood of positive use and outcomes when AI systems are deployed in practice.

Prithviraj Ammanabrolu. Language Learning in Interactive Environments. Ph.D. Dissertation, Georgia Institute of Technology, 2021.

Natural language communication has long been considered a defining characteristic of human intelligence. I am motivated by the question of how learning agents can understand and generate contextually relevant natural language in service of achieving a goal. In pursuit of this objective, I have been studying Interactive Narratives, or text-adventures: simulations in which an agent interacts with the world purely through natural language—”seeing” and “acting upon” the world using textual descriptions and commands. These games are usually structured as puzzles or quests in which a player must complete a sequence of actions to succeed. My work studies two closely related aspects of Interactive Narratives: operating in these environments and creating them in addition to their intersection—each presenting its own set of unique challenges. Operating in these environments presents three challenges: (1) Knowledge representation—an agent must maintain a persistent memory of what it has learned through its experiences with a partially observable world; (2) Commonsense reasoning to endow the agent with priors on how to interact with the world around it; and (3) Scaling to effectively explore sparse-reward, combinatorially-sized natural language state-action spaces. On the other hand, creating these environments can be split into two complementary considerations: (1) World generation, or the problem of creating a world that defines the limits of the actions an agent can perform; and (2) Quest generation, i.e. defining actionable objectives grounded in a given world. I will present my work thus far—showcasing how structured, interpretable data representations in the form of knowledge graphs aid in each of these tasks—in addition to proposing how exactly these two aspects of Interactive Narratives can be combined to improve language learning and generalization across this board of challenges.

Lara Martin. Neurosymbolic Automated Story Generation. Ph.D. Dissertation, Georgia Institute of Technology, 2021.

Although we are currently riding a technological wave of personal assistants, many of these agents still struggle to communicate appropriately. Humans are natural storytellers, so it would be fitting if artificial intelligence (AI) could tell stories as well. Automated story generation is an area of AI research that aims to create agents that tell good stories. With goodness being subjective and hard-to-define, I focus on the perceived coherence of stories in this thesis. Previous story generation systems use planning and symbolic representations to create new stories, but these systems require a vast amount of knowledge engineering. The stories created by these systems are coherent, but only a finite set of stories can be generated. In contrast, very large neural language models have recently made the headlines in the natural language processing community. Though impressive on the surface, even the most sophisticated of these models begins to lose coherence over time. My research looks at both neural and symbolic techniques of automated story generation. In this dissertation, I created automated story generation systems that improved coherence by leveraging various symbolic approaches for neural systems. I did this through a collection of techniques; by separating out semantic event generation from syntactic sentence generation, manipulating neural event generation to become goal-driven, improving syntactic sentence generation to be more interesting and coherent, and creating a rule-based infrastructure to aid neural networks in causal reasoning.

Matthew Guzdial. Combinational machine learning creativity. Ph.D. Dissertation, Georgia Institute of Technology, 2019.

Computational creativity is a field focused on the study and development of behaviors in computers an observer would deem creative. Traditionally, it has relied upon rules-based and search-based artificial intelligence. However these types of artificial intelligence rely on human-authored knowledge that can obfuscate whether creative behavior arose due to actions from an AI agent or its developer. In this dissertation I look to instead apply machine learning to a subset of computational creativity problems. This particular area of research is called combinational creativity. Combinational creativity is the type of creativity people employ when they create new knowledge by recombining elements of existing knowledge. This dissertation examines the problem of combining combinational creativity and machine learning in two primary domains: video game design and image classification. Towards the goal of creative novel video game designs I describe a machine-learning approach to learn a model of video game level design and rules from gameplay video, validating the accuracy of these with a human subject study and automated gameplaying agent, respectively. I then introduce a novel combinational creativity approach I call conceptual expansion, designed to work with machine-learned knowledge and models by default. I demonstrate conceptual expansion’s utility and limitations across both domains, through the creation of novel video games and applied in a transfer learning framework for image classification. This dissertation seeks to validate the following hypothesis: For creativity problems that require the combination of aspects of distinct examples, conceptual expansion of generative or evaluative models can create a greater range of artifacts or behaviors, with greater measures of value, surprise, and novelty than standard combinational approaches or approaches that do not explicitly model combination.

Alexander Zook. Automated Iterative Game Design. Ph.D. Dissertation, Georgia Institute of Technology, 2016.

Computational systems to model aspects of iterative game design were proposed, encompassing: game generation, sampling behaviors in a game, analyzing game behaviors for patterns, and iteratively altering a game design. Explicit models of the actions in games as planning operators allowed an intelligent system to reason about how actions and action sequences affect gameplay and to create new mechanics. Metrics to analyze differences in player strategies were presented and were able to identify flaws in game designs. An intelligent system learned design knowledge about gameplay and was able to reduce the number of design iterations needed during playtesting a game to achieve a design goal. Implications for how intelligent systems augment and automate human game design practices are discussed.

Hong Yu. A Data-Driven Approach for Personalized Drama Management. Ph.D. Dissertation, Georgia Institute of Technology, 2015.

An interactive narrative is a form of digital entertainment in which players can create or influence a dramatic storyline through actions, typically by assuming the role of a character in a fictional virtual world. The interactive narrative systems usually employ a drama manager (DM), an omniscient background agent that monitors the fictional world and determines what will happen next in the players’ story experience. Prevailing approaches to drama management choose successive story plot points based on a set of criteria given by the game designers. In other words, the DM is a surrogate for the game designers. In this dissertation, I create a data-driven personalized drama manager that takes into consideration players’ preferences. The personalized drama manager is capable of (1) modeling the players’ preference over successive plot points from the players’ feedback; (2) guiding the players towards selected plot points without sacrificing players’ agency; (3) choosing target successive plot points that simultaneously increase the player’s story preference ratings and the probability of the players selecting the plot points. To address the first problem, I develop a collaborative filtering algorithm that takes into account the specific sequence (or history) of experienced plot points when modeling players’ preferences for future plot points. Unlike the traditional collaborative filtering algorithms that make one-shot recommendations of complete story artifacts (e.g., books, movies), the collaborative filtering algorithm I develop is a sequential recommendation algorithm that makes every successive recommendation based on all previous recommendations. To address the second problem, I create a multi-option branching story graph that allows multiple options to point to each plot point. The personalized DM working in the multi-option branching story graph can influence the players to make choices that coincide with the trajectories selected by the DM, while gives the players the full agency to make any selection that leads to any plot point in their own judgement. To address the third problem, the personalized DM models the probability that the players transitioning to each full-length stories and selects target stories that achieve the highest expected preference ratings at every branching point in the story space. The personalized DM is implemented in an interactive narrative system built with choose-your-own-adventure stories. Human study results show that the personalized DM can achieve significantly higher preference ratings than non-personalized DMs or DMs with pre-defined player types, while preserve the players’ sense of agency.

Boyang Li. Learning Knowledge to Support Domain-Independent Narrative Intelligence. Ph.D. Dissertation, Georgia institute of Technology, 2014.

Narrative Intelligence is the ability to craft, tell, understand, and respond appropriately to narratives. It has been proposed as a vital component of machines aiming to understand human activities or to communicate effectively with humans. However, most existing systems purported to demonstrate Narrative Intelligence rely on manually authored knowledge structures that require extensive expert labor. These systems are constrained to operate in a few domains where knowledge has been provided. This dissertation investigates the learning of knowledge structures to support Narrative Intelligence in any domain. I propose and build a system that, from an corpus of simple exemplar stories, learns complex knowledge structures that subsequently enable the creation, telling, and understanding of narratives. The knowledge representation balances the complexity of learning and the richness of narrative applications, so that we can (1) learn the knowledge robustly in the presence of noise, (2) generate a large variety of highly coherent stories, (3) tell them in recognizably different narration styles and (4) understand stories efficiently. The accuracy and effectiveness of the system have been verified by a series of user studies and computational experiments. As a result, the system is able to demonstrate Narrative Intelligence in any domain where we can collect a small number of exemplar stories. This dissertation is the first step toward scaling computational narrative intelligence to meet the challenges of the real world.

Brian O’Neill. A Computational Model of Suspense for the Augmentation of Intelligent Story Generation. Ph.D. Dissertation, Georgia institute of Technology, 2013.

In this dissertation, I present Dramatis, a computational human behavior model of suspense based on Gerrig and Bernardo’s de nition of suspense. In this model, readers traverse a search space on behalf of the protagonist, searching for an escape from some oncoming negative outcome. As the quality or quantity of escapes available to the protagonist decreases, the level of suspense felt by the audience increases. The major components of Dramatis are a model of reader salience, used to determine what elements of the story are foregrounded in the reader’s mind, and an algorithm for determining the escape plan that a reader would perceive to be the most likely to succeed for the protagonist. I evaluate my model by comparing its ratings of suspense to the self-reported suspense ratings of human readers. Additionally, I demonstrate that the components of the suspense model are sufficient to produce these human-comparable ratings.

Mark O. Riedl. Narrative Generation: Balancing Plot and Character. Ph.D. Dissertation, North Carolina State University, 2004.

The ability to generate narrative is of importance to computer systems that wish to use story effectively for a wide range of contexts ranging from entertainment to training and education. The typical approach for incorporating narrative into a computer system is for system builders to script the narrative features at design time. A central limitation of this pre- scripting approach is its lack of flexibility – such systems cannot adapt the story to the user’s interests, preferences, or abilities. The alternative approach is for the computer systems themselves to generate narrative that is fully adapted to the user at run time.

A central challenge for systems that generate their own narrative elements is to create narratives that are readily understood as such by their users. I define two properties of narrative – plot coherence and character believability – which play a role in the success of a narrative in terms of the ability of the narrative’s audience to comprehend its structure. Plot coherence is the perception by the audience that the main events of a story have meaning and relevance to the outcome of the story. Character believability is the perception by the audience that the actions performed by characters are motivated by their beliefs, desires, and traits.

In this dissertation, I explore the use of search-based planning as a technique for generating stories that demonstrate both strong plot coherence and strong character believability. To that end, the dissertation makes three central contributions. First, I describe an extension to search-based planning that reasons about character intentions by identifying possible character goals that explain their actions in a plan and creates plan structure that explains why those characters commit to their goals. Second, I describe how a character personality model can be incorporated into planning in a way that guides the planner to choose consistent character behavior without strictly preventing characters from acting “out of character” when necessary. Finally, I present an open-world planning algorithm that extends the capabilities of conventional planning algorithms in order to support a process of story creation modeled after the process of dramatic authoring used by human authors. This open-world planning approach enables a story planner not only to search for a sequence of character actions to achieve a set of goals, but also to search for a possible world in which the story can effectively be set.

The planning algorithms presented in this dissertation are used within a narrative generation system called Fabulist. Fabulist generates a story as a sequence of character actions and then recounts the story by first generating a discourse plan that specifies how the story content should be told and then realizing the discourse plan in a storytelling medium. I present the results of an empirical evaluation that demonstrates that narratives generated by Fabulist have strong plot coherence and strong character believability. The results clearly indicate how a planning approach to narrative generation that reasons about plot coherence and character believability can improve the audience’s comprehension of plot and character.