Hack the Experience: Tools for Artists from Cognitive Science

Front matter

This is a book for artists, but it is also for curators, art school faculty, landscape architects, gallerists, archivists, postdisciplinary multi-hyphenates, museum program staff, and anyone who wants to know about the ways art ne science come together to engage an audience.

BEFORE YOU START TO READ THIS BOOK, take this moment to think about making a donation to punctum books, an independent non-profit press,

@ https://punctumbooks.com/support/

Contents

Introduction 9 Chapters

1. Breaking Down Experiences to Find Backdoors for Hacking 15

2. Control Attention 19 3. Experience Hacking Is Fundamentally a Form of Design 31 4. Your Cognitive Toolbox 41 5. Story, World Building, and Belief 109 6. Tying Things Together: Paths & Nodes 125 7. Documentation for Planning, Archiving, and Reproducing 137 8. Making This Work in a Museum or Gallery 145

Introduction

Artists and scientists do the same thing: they observe something, analyze their observations, and present their findings in a way that untrained people can see with relative ease what the artist or scientist worked to see in their own analyses.

The core argument of this book centers on the premise that art is a form of cognitive engineering and that the physical environment (or objects in the physical environment) can be shaped using empirically validated models from cognitive science to maximize emotional and sensory experience. Like the title suggests, this is a kind of hacking. As the book unfolds, this kind of hacking will take various environmental resources and interventions and apply them to different conditions (emotions, senses, general experience, etc.), in order to design integrated experiences that help awaken your audience in new ways.

Cognitive engineering blends the role of artist and scientist into a process of building experiences that evoke responses of some kind. This book will give you a toolkit for planning experiences and events of all sorts and will guide you through the process of bringing elements of cognitive engineering into your creative practice. After reading this short book, your work will help your audience connect with your work in more meaningful and memorable ways. Cognitive engineering in this sense provides a way to promote inquiry, and to make people curious about the world. The experiences you design will inform the way that people live in the world and make meaning in their day-to-day life.

Artists naturally evoke responses from their viewing audience, such as the visceral responses of disgust, of joy, and also cognitive responses of indifference, bewilderment, confusion, hatred, or suspicion. This in part comes from the content of the work, partly from the context, partly from the execution, partly from the past experiences and openness of the audience, and partly from scores of other factors. All art evokes responses. “My kid could do that” is a response, and, like it or not, it is a valid response that was evoked by the work. The goal of this book is to help you shape the viewers’ experience to leverage basic cognition and attention patterns to push people toward a set of responses you want to evoke. For instance, if you want to create a feeling of remoteness in the middle of a city, the tools in this book will help you evoke that feeling by showing you which perceptual elements you need to shape in the physical space and how.

Disciplines like new media studies, neuroscience, biosemiotics, and data studies already define significant corners of the contemporary art world, so there is a context for creating experiences that make sense out of the body and the brain. This book contributes to that effort by putting ideas together that can move your existing creative practice up onto new levels of experiential grounding and perceptual relevance. It is designed for people who are actively working on projects, and it intervenes into your creative process like a critic and a consultant, guiding you with new insights into your process of creation. This book is dense, like a bullion cube of concepts and methods. It is good advice to read this book at least twice, and to read it while working on specific projects. Also, as soon as you learn a concept from this book, try to teach it to someone, explaining it in your own words so as to further solidify the concept in your own memory.

This guide is also useful to people working in museums and galleries. In the turn toward experience in museum interpretation, this guide makes sense of the ways that audiences engage with stories told through juxtaposition, organization, and collection, helping to bring an outside eye to curatorial practice. Creating experiences from the ground up in a requirements-driven process lets experience designers play around with a variety of methods to engage the public on emotional, visceral, and cognitive levels. It makes sense then to have some understanding of the way general cognition affects experience to better exhibit certain works, and to design exhibitions that better expose the relationship between cognition and experience.

Outside of the gallery in the applied arts (like design), engineering response-evoking user experiences is also a large part of brand development, and this simple guide can even serve as a playbook for designing transtormational experiences that add perceptually relevant texture to brand narratives. While the focus of this book is art installation, the presented ideas equally apply to building stories and worlds, whether for scripts, storyboards, or game design.

What constitutes “hacking”? Hacking in this book is an intervention that organizes the physical world in a way that leverages natural cognitive structures to evoke some kind of response in the viewer. It’s “hacking” because it is a kind of short-cut into a mental state through various openings in the body-mind-experience system.

What exactly does “experience” mean? This word has two senses in focus here. First, experience refers to the fabric of moments in general. Second, and more specifically, experience refers to your neural integration of stimuli in the environment to create your perception of a moment in time. This first sense is the perceived state: the experience of the participant. The second sense refers to the atmospheric and event-like nature of experience: experience as a moment in time in some designed environment, installation, or intervention (i.e., the moment created by the artist or designer), and how that is comprehended by the viewer-participant. In other words, the first sense is your perspective and the second sense is the context in which you have your perspective —your experience (first sense) of an experience (second sense).

This second sense of experiences includes things like performances, gallery exhibits, parties, festivals, events, happenings, immersive theater, installations, new media art, pop-up spaces, stable architecture, urban fabric, cultural narratives, and virtual and physical worlds. They can even be small-scale, private events like brushing your teeth, or drinking a glass of water, but they can also be massive in scale, like the string of activities involved in rebranding post-industrial cities. Experience can be shaped as it scales up or down.

Experiences can easily be expanded to include theme park design, museums, restaurants, brand encounters, advertising campaigns, and film. An experience can take place in a single location (such as a standard event or party) or it can extend over several locations (such as a hike or even a tourist walk). An experience can have a single audience or a very large audience. lt can have one or many goals, it can consist of one activity or many activities, it can occur once and be over or it can extend over a long period of time, sequentially in some predetermined order or ad hoc as determined by happenstance or by the random paths of audience encounter. There are no limits to the characteristics of experience, except for the requirement that it has to happen somewhere to someone at some time and it has to do something to that person, such as evoke a response.

Experiences in the first sense happen whether or not experiences in the second sense exist at all. But experiences in the second sense are designed to intentionally influence first-sense experience. Second-sense experiences center around intentionality of design, intending something to mean something for someone else. It is a kind of language through which the artist communicates something to the audience. The artist means something for the audience. As an artist you already know how to communicate to your audience through the traditional channels of form and subject and content. This book works to uncover how structural elements of cognition occur pervasively in daily life and how they can be harnessed in immersive, spatial approaches to art.

Cognitive engineering helps orchestrate any kind of experience: dinner for two, a site-specific installation, an exhibit, a building, an event, a relationship, an identity, a life story, or a brand narrative. This handbook sets out to show you how to set up scenarios for your audience to engage with experiences in ways that encourage them to oscillate between the roles of participant and spectator. Follow this guide to help your audience think through action. Your audience will experience your work and ideally have a richer experience of the world at large because of how you bring in awareness, the senses, and basic narrative forms to create compelling installations.

In his work on cognition and aesthetics, neuroscientist Merin Donald (2009) defines art as a form of cognitive engineering and argues that the main goal of art is to evoke responses of some ind in the viewer. The production of art is a process of changing he way people see some element of experience by engaging people through their experience of a specific work of art. Artists have always engineered viewer experiences, but new tools from cogniive science (like conceptual metaphor, blending, and cognitive simulation) enable artists to refine this process through the use of cognitive models (of attention and embodiment) as foundational elements in the same way that they use materials and the range of techniques in traditional and contemporary art making.

Taking the mind as the target audience of an artwork, it makes sense for artists to use models that come from the empirical findings of the sciences of the mind to enrich and underpin an art practice.

Pay attention to this statement, it will shape the way you make sense out of the models presented in this book: Understanding the basic patterns of human attention unlocks the door to how you shape the viewer’s experience of an artwork. Everything is about making the most use of attention.

The book is divided into four sections:

INTRODUCTION

Section 1: Hacking & Engineering Experiences

This section looks at the process of hacking cognition, the identification of backdoors for hacking, the role of design, understanding the human attention system in order to direct it, design armatures, and practices of thinking about design problems in experience hacking. This section presents some of the ideas that act as underlying skeletal structures in designing experiences.

Section 2: Toolbox of Cognitive Tools

This section has seven tools that give you important background information on different elements of experience and what they are useful for in designing experiences. These are topical sections that get you thinking about the tools that are available to you as you design experiences. This section presents some of the musculature of everyday experience and outlines ways that these experience elements can be combined, modified, and experimented with in the process of designing physical interventions in the design space.

Section 3: Stories & Paths

This section ties together the musculature from Section 2 and the skeleton from Section 1 and grounds experience design in storytelling processes along paths and at intervention nodes along these paths.