From left: Nehemiah Spencer and Brandon Gregoire, graduate students in the Department of Dance, performed on a bare stage in this scene from “Imagined Odyssey.” But the audience, wearing Microsoft HoloLens devices, saw them surrounded and confined by holographic shapes. Brad Petot (photo) / Interactive Commons (HoloLens imagery)
Moving in circles on a darkened stage, two dancers conjure up a fiery storm.
It’s not immediately clear what is happening. The dancers’ bodies throw off streaks of light that whirl around them like straw in the wind. Then the streaks proliferate and gather speed until they form two glowing tornadoes. Unlike a conventional animation, this display evolves in real time, during a live performance. In the end, the tornadoes merge and tower over the figures who unleashed them—an intensely bright, convincingly three-dimensional illusion.
Karen Potter, professor and chair in Case Western Reserve’s Department of Dance, caught her first glimpse of this mysterious scene last fall, during a rehearsal at Mather Dance Center. She was assisting with preparations for Imagined Odyssey, a new work by Professor Gary Galbraith, the department’s artistic director.
Potter looked on as Galbraith coached two students from the department’s Master of Fine Arts (MFA) Program. She found the choreography exciting in itself: “the shapes, the energy, the highly dynamic and technically challenging movements.” But she hadn’t yet watched it in tandem with the storm: a special effect produced with Microsoft HoloLens.
The HoloLens is a computerized device worn like a wrap-around visor. Its name refers both to its main component (a literal lens) and to the images (holograms) that it produces. In part, the lens functions as a window onto a viewer’s surroundings. But it also, surprisingly, projects laser light into the viewer’s eyes, painting holograms directly onto the retinas. As a result, HoloLens users perceive an augmented reality, in which the holograms appear alongside the forms and objects of the actual world.
This is what Potter observed when she put on a HoloLens at the rehearsal. Galbraith had told her about the images that would surround the two women performing this scene, but his descriptions hadn’t prepared her for the moment when all the elements came together.
“It was a real high point of the dance,” she recalls. “There was a crescendo in the music when the dancers were in that frenetic swirl, and the swirling of the ladies matched the swirling of the tornadoes. The holograms were vibrant red and orange—colors that were chosen to match the costumes. It was just a brilliant meshing, an overlay or interplay of the choreography and the holograms. And to see the culmination of what Gary had envisioned was so compelling.”
Potter wasn’t alone in her enthusiasm. Imagined Odyssey, the closing piece in a concert of works by faculty members and guest artists, played to six sold-out houses last November. When a seventh performance was added, it sold out, too. Audiences seized the chance to attend a first-of-its-kind production. Yet Galbraith has a history of creating works that integrate dance and technology, enriching the viewers’ experience as well as the art form to which he has devoted his career.
For Gary Galbraith, “Imagined Odyssey” was the latest in a series of experimental works integrating technology into contemporary dance. Photo by Keli Schimelpfenig
A CWRU faculty member since 1999, Galbraith was a longtime principal dancer with the Martha Graham Dance Company, where he performed many of the major roles in the repertory that Graham herself created. Like Graham, Galbraith infuses his work with mythic elements, and he has been influenced by the modernist sets she commissioned from artists such as the sculptor Isamu Noguchi. Transporting those sets and associated props when the company went on tour was a daunting enterprise. How much simpler it would have been, Galbraith now thinks, if they had been made of light.
Yet integrating dance with holograms was also a massive undertaking. Galbraith achieved it by joining forces with Case Western Reserve’s Interactive Commons, an institute where artists, illustrators, programmers and network engineers deploy advanced visualization technologies to present information and foster collaboration across the university.
The Interactive Commons was founded in 2014 by Mark Griswold, professor of radiology in the CWRU School of Medicine, and Erin Henninger, who serves as executive director. Griswold is an internationally recognized innovator in the field of biomedical imaging. Among his major contributions, he co-led the research team that developed magnetic resonance fingerprinting, a clinical tool that analyzes tissue changes for early indications of cancer, multiple sclerosis, heart disease and other serious medical conditions.
Soon after the Interactive Commons was launched, Case Western Reserve’s partnership with Cleveland Clinic on the new Health Education Campus led to an opportunity for Griswold to view a demonstration of Microsoft HoloLens before it was released. The university quickly realized the Interactive Commons was the ideal location to develop applications for the technology.
Since then, Griswold and his team have created anatomy lessons in which medical students interact with a life-size, transparent hologram of the human body. The students examine not only bones and muscles, but also the interiors of organs. The hologram can be programmed to illustrate evidence of disease as well. And professors can easily point out features of the body to their students, since it seems to float, ghostlike, in their midst.
The idea of incorporating holograms into a CWRU dance production began taking shape in summer 2016. Dean Cyrus Taylor had introduced Galbraith to Griswold and Henninger a year before, and from the outset they’d had plenty to talk about. The team from the Interactive Commons was interested in what Galbraith had already done to blend technology and dance, and Galbraith was always on the lookout for new approaches. He remembers a meeting that summer when he first saw what HoloLens could do. Immediately, he says, he recognized both the technology’s potential and the tremendous amount of time a successful project would require. At that point, he and the Interactive Commons team decided to give it a try.
Galbraith spent much of 2017, including a spring sabbatical, working on the new piece and generating ideas for holograms. Some of those ideas came from unlikely sources. During a stroll in Brooklyn Bridge Park, for example, Galbraith noticed a work of public art by the British sculptor Anish Kapoor. Descension, a temporary installation, consisted of a whirlpool whose waters spiraled down into the earth. The moment he saw it, Galbraith pictured a vortex engulfing the protagonists toward the end of his work in progress: a 12–minute dance epic.
Imagined Odyssey follows two brothers, or comrades in arms, who set off on a quest, prompted by a masked dancer whom Galbraith calls “the trickster.” Along the way, they encounter the women who command the winds—“beguiling ones” who alternately guide, tempt and obstruct them. Like Homer’s Odyssey, Galbraith’s work invests such figures with powers that the male wanderers lack. And HoloLens gave him a vivid means of showing those powers in action.
During the process of designing the holograms, Galbraith met regularly with the Interactive Commons team. Initially, he laid out his conception and said, “This is sort of what it looks like. These are the feelings I have about it.” Then they worked together to realize his vision.
Sometimes, Galbraith would describe an image he had in mind and ask, “Can you give me this?” Just as often, though, he would request a simple geometric shape—a sphere, a cone, a cube—and experiment with it himself, applying programming skills he had acquired from earlier projects.
A bristling forest of shapes levitated during “Imagined Odyssey” and turned upside down, forming a jagged canopy above the dancers. Brad Petot (photo) / Interactive Commons (HoloLens imagery)
For one especially ambitious scene, Galbraith started with half of a shining gold arch. Then he created enough replicas to fill the stage, where they looked like abstract trees in a virtual forest. In the narrative of Imagined Odyssey, the trickster brings this forest into being. Then it rises high in the air and rotates until the trees are pointing downward. To Galbraith, the inverted forest resembled a mass of stalactites in a cave where his wanderers are momentarily entrapped.
Along with the artistic challenges posed by Imagined Odyssey, the collaborators addressed technological issues that hadn’t come up in previous HoloLens presentations. For example, Griswold had conducted holographic anatomy lessons for 50 or so students at once. This in itself was a breakthrough; it was the first time someone had created the network infrastructure for a large-group experience with HoloLens. But Mather Dance Center accommodates an audience of 80 people, so the infrastructure there would have to be even more extensive.
Undaunted, Galbraith turned to Dennis Risen, a project manager in University Technology. Risen and his colleagues enhanced the center’s wireless capability, turning the 111-year-old building’s stadium seating area into the most powerful hotspot on campus.
Galbraith himself also made technological contributions to Imagined Odyssey. For example, because the holograms were in constant motion, he had to find a way to coordinate their movements with those of the dancers. His solution involved a projector and camera mounted above the stage. The projector beamed tiny circles of light to designated spots on the floor. When dancers hit their mark, the camera detected the action and relayed a cue to a central computer, which in turn triggered a hologram’s appearance. In a sense, then, the dancers really were summoning those images to the stage.
Griswold says that these innovations were “really at the forefront of technology anywhere.” He predicts they will have an impact on how HoloLens is used in other fields, such as engineering, physics and medicine. And he sees the joint effort by the Interactive Commons and the dance department as a model of the silo-busting collaborations that take place at Case Western Reserve. Before he met Galbraith, he confesses, “I had no idea that a dancer even paid attention to this kind of technology.”
At the end of “Imagined Odyssey,” the trickster played by Karen Opper, no longer masked, exults in her capture of the wanderers, who appear trapped in a holographic orb. Brad Petot (photo) / Interactive Commons (HoloLens imagery)
While he was choreographing Imagined Odyssey, Galbraith had already selected a five-member cast for the premiere. All of the dancers came from the MFA Program: third-year student Karen Opper, second-year students Yizhen Hu and Xiaomeng Zhao and first-year students Brandon Gregoire and Nehemiah Spencer. Their performance backgrounds included roles in undergraduate dance and theater productions; appearances with ballet, musical theater and contemporary dance companies; and participation in major dance festivals. Now Galbraith was inviting them to venture into unknown territory.
The greatest challenge the dancers faced was this: They were interacting with objects that only the audience could see. When the trickster sent a translucent orb sailing across the stage, Gregoire and Spencer had to pretend to follow it with their gaze. When Hu and Zhao stirred up the tornadoes, they had to picture their limbs emitting light. The holograms themselves were invisible to them. Or, as Opper puts it, “We had this whole other partner that wasn’t there.”
Galbraith adopted several strategies to help the dancers meet this challenge. When they were first learning their roles, he indicated where the holograms would be or tried to explain what they would look like. But since he was still in the process of creating the images, his descriptions were provisional at best. Once prototypes of the holograms were available, he had students take turns wearing a HoloLens when they weren’t onstage so that they could become familiar with the imagery.
In addition, Galbraith recorded selected rehearsals through a HoloLens he placed on a chair in the center aisle of the audience section. After students practiced a segment, they would gather around his computer to observe their movements in context. In one scene, Gregoire and Spencer ran in place for several seconds. The HoloLens footage revealed the objects that framed their actions: a series of tall, elegant gateways flying toward them and disappearing. The holograms gave the impression that the wanderers were racing across one threshold after another, covering immense distances and emerging at last into another world.
Galbraith wondered, going into the project, how it would affect the dancers’ approach to their work. Spencer recalls seeing the holograms for the first time and saying to himself, “Wow, I have to change my whole way of perceiving and performing now.” Ordinarily, he explains, “I’m performing and my mind is blank because my body is in the moment.” But in Imagined Odyssey, “I feel like I’m constantly thinking. I keep reminding myself: ‘The hologram’s right there.’”
From left: In their roles as the “beguiling ones,” graduate students Yizhen Hu and Xiaomeng Zhao had the power to radiate streaks of light that erupted into virtual tornadoes. Brad Petot (photo) / Interactive Commons (HoloLens imagery)
Zhao recalls the great precision Imagined Odyssey required of the dancers. Their timing had to be perfect, and they had to arrive at the exact locations Galbraith had assigned to them. Otherwise, they would activate a hologram at the wrong moment, or fail to activate it at all. Then, too, once a hologram materialized, the dancers had to be careful not to position themselves behind it (where their movements would be obscured) or in front of it (where they would block the audience’s view).
And yet, while Zhao recognized the importance of “cooperation with the HoloLens stuff,” she didn’t want to rely on it to make her performance meaningful. “These are real movements,” she says, referring to the role Galbraith had choreographed for her. And she felt responsible for ensuring that those movements were expressive in their own right and didn’t look “weird” apart from the holograms.
Zhao’s fellow dancers shared her aspiration, and they knew it was consistent with Galbraith’s artistic objectives. “Gary is all about finding the balance between the dancer and the technology,” Opper explains. “He didn’t want the dance to overpower the holograms, or the holograms to overpower the movements.”
You might think the dancers would find it odd to have rows of people watching them through visors. But there is no sign that HoloLens created a barrier between the performers and the audience. On the contrary, Hu felt that the technology deepened the connection between them.
Dancers, she notes, often draw inspiration from mental pictures that “give the movements more meaning.” They may create these pictures themselves, or a director may suggest them. (Gregoire cites one rehearsal in which Galbraith wanted him to rest heavily on one leg and keep still. “Imagine,” Galbraith said, “that your leg is growing into the ground.”) To Hu, the holograms were a kindred form of visualization. But instead of existing solely in the dancers’ memories and imaginations, they were also present to the spectators. “What I think,” she says, “the audience can see.”
“I feel like this was definitely a gateway experience to show where the performing arts can go,” Spencer says. “Technology is constantly developing. Modern dance is also capable of developing, and I think this piece is a showcase to see how they can develop together.”