Augmented reality art creates new opportunities—and new contexts—for works

Augmented reality (AR) art is most concisely defined as mobile device–viewable animation and audio placed in a specific spatial location. But AR art is much more than that: It’s an exploding art medium coming to the forefront thanks to technological advancements and pandemic-induced isolation.

AR art can place an Old Master’s tour-de-force in your living room or put works that push the edge of digital craftsmanship in your backyard. It can revise and recontextualize existing works of art within and outside of galleries and museums. And as more time and attention are drawn into digital spaces, its growing cultural import signals a transition toward experiences of art that are ephemeral, mutable, and increasingly interactive.

Augmented reality art compositions can be viewed with a mobile device app, presented with varying levels of animation, audio content, and viewer-driven mutability. The art itself can be new compositions, replicated existing compositions, or revised and recontextualized compositions. These appear in space as 3D artwork objects in designated areas or anywhere the viewer points the camera.

AR mashes up reality and illusion, adding layers of form, motion, and meaning to extant artworks or extant spaces. Unlike virtual reality (VR), which is an entirely synthetic and immersive experience (and requires a headset and a controller), AR is often less expensive to produce and requires less specialized hardware for both creators and users.

AR is taking traditional works of art out of the gallery, giving them new life and relevance. AR makes the art experience more like a collaboration between viewer and artist than a lecture. Superficially, it gives the viewer the chance to “place” art works anywhere. A scientifically exacting Audubon Society avian illustration will look and feel different next to a window looking onto a garden with bird feeders than on the white walls of a museum or gallery. Some AR art installations also allow viewers to manipulate the 3D elements of the composition, such as Adana Tillman’s Interplay: Art Play for All (produced in conjunction with the Akron Art Museum), which placed collage-style posters all over the city. When a QR code is scanned, it displays the poster’s objects—human figures, abstracted shapes, and more—which the viewer can move around from their mobile device.

This is one example of a dispersed installation that can take place anywhere a poster can be hung, which is ideal for the social distance requirements of the COVID-19 pandemic. The most obvious parallels to AR art in the traditional art world might be light-projection art, but one advantage of AR is that by bringing the art experience to smartphones, it doesn’t concentrate large crowds into a single public space. During a pandemic or not, AR art is a way to disperse culture through space. This changes the price of access from museum admission fees or geographic access to mobile device ownership, a shift new enough that it’s not yet clear who will benefit. AR could be a way for cultural institutions to extend their reach to audiences that might not normally visit a museum at all—or might not do so during a pandemic.

But this mutability prompts new questions about ethics and ownership in art. First, there’s the ability to use AR to essentially “write over” existing artworks. Brazilian artist Mauricio Nocera embraces this approach, sprucing up Botticelli’s The Birth of Venus with graffiti tags and swipes of digital spray paint. But it’s easy to imagine living artists objecting to such revisions of their work. And given AR art’s free-floating nature, placing AR artworks next to other art could radically recontextualize or contradict them in ways their creators would refuse.

That was the case for artist Sebastian Errazuriz, who used his own app to disrupt a Snapchat collaboration with sculptor Jeff Koons in an act of “symbolic vandalism.” The collaboration placed geotagged digital models of Koons’s sculptures in specific locations, and Errazuriz tagged Koons’s digital model, graffitied over in his own app. As much as AR and social media offer artists new frontiers for aesthetic expression, they also provide fresh tools to question creators’ authorship.

The application of current laws around AR art is something of a gray area, waiting for test cases to suss out the limits of copyright and fair use, determining what level of spatial control artists or museums can demand when intrusions are purely digital. Perhaps the most unprecedented and ambiguous change is how AR art creates new tools to make pieces of art interact with each other.

Given that AR art is created and displayed with digital tools, it lends itself well to showcasing the process and nitty-gritty details of its creation. Many artists working in this medium are sharing design-software screenshots and skeletal animations of their early iterations alongside the final compositions. For example, behind the scenes of her whimsical Etsy-shop AR art aesthetic, multimedia artist Heather Dunaway Smith lays out her process on Instagram with video clips of audio mixes, sketching tools, and modeling software. Autodesk’s AUGmentectureextension app can similarly generate 3D imagery from Autodesk Revit that can be viewed with a mobile device.

Given this relatively democratized access, new tools for making AR art are being democratized, as well. Instagram is the most popular social media platform for sharing AR art, and as such, Spark AR software helps users make their own AR Instagram and Facebook filters without having to code, importing objects and sounds, animating objects, and creating custom textures.

As part of the Invasive Species exhibit at the Pérez Art Museum Miami (PAMM), Felice Grodin designed a gigantic biomorphically ambiguous sea creature called Terrafish. This AR art piece was placed on the museum’s signature feature: its 45-foot-long hanging gardens, designed by French botanist Patrick Blanc in collaboration with the museum’s architects, Herzog & de Meuron.

Grodin previously practiced within the field of architecture, and it shows in her rigorous sense of proportion and facility with architectural design tools. She began by making 3D models of the PAMM space in Autodesk AutoCAD, working from construction drawings supplied by the building’s engineer. Grodin and PAMM worked with Cuseum (which specializes in digital-engagement technology for cultural institutions) to create 3D models in Autodesk 3ds Max, integrating the models into the museum’s multimedia app AR kit. Grodin’s AR installations were developed from previous works, including hand-drawn illustrations rendered with the digital sharpness of someone who’s spent years plotting plans in AutoCAD. The show will be the focus of related STEAM educational programming during its run.

Just as AR art is a merger of the real and the ephemeral, Grodin’s Invasive Species is a merger of flora, fauna, and architecture. The neon viscera of Grodin’s Terrafish pulses with a respiratory rhythm, contrasting with the natural flora of the garden. Grodin’s slickly digital, animation-like shapes are striking. “It’s almost like they’re trying to reach out to the hanging gardens for a food source or to become a part of the architecture,” says Jennifer Inacio, the exhibit’s curator.

It’s also huge and is Grodin’s largest piece. Its stalks are 49 feet tall, and the base is approximately 100 feet wide. The installation received a grant from the John S. and James L. Knight Foundation as part of the Knight Foundation initiative, which seeks to empower museums to reach their community via digital tools.

Terrafish was inspired by non-native jellyfish populating South Florida’s waters and evokes the specter of climate change and human disruption of nature. Invasive Species acknowledges the mutability of ecosystems, pondering climate change as a generative force that might introduce new fauna into existence. “It’s about how this place could potentially evolve or mutate,” Grodin says. Given the context of this and other AR art installations, the landscape for AR art is likely to mutate just as quickly as Grodin’s otherworldly creations.

AR art can be expressive (like Grodin’s Invasive Species) or didactic and educational, like the Smithsonian app Skin and Bones, which peels back the skin and muscles of mounted animal skeletons on display, offering new views into zoological physiology.

Alex Mayhew’s ReBlink at the Art Gallery of Ontario lies somewhere in between. For the installation, selected paintings in the collection are given AR updates: images of modernization, alienation, and distraction. In one, a fire-haired early-modern maiden holds out a selfie stick with a smirk. In another, a portrait of a 17th-century couple with a bounty of wild-caught boar and grapes is replaced with their 21st-century equivalents: canned fruit and hot dogs. The clear satirical take gains an element of meta-commentary, critiquing online absorption in an online medium.

The artist Marc-O-Matic creates giddy Saturday morning cartoon steampunk AR art such as axonometric exploded diagrams of airships that blast out of drawings and off gallery walls, as well as menacing mechanized angler fish and bookish hermit crabs with entire libraries on their backs. It’s an intensely intricate and self-contained world.

The Serpentine Gallery in London used AR to offer artist Christo, who died in 2020, a one-of-kind posthumous swan song. Using the Acute Art app, the museum placed an exact virtual replica of Christo and Jeanne-Claude’s final large-scale public art work on the Serpentine River. The piece, London Mastaba, Serpentine Lake, Hyde Park, 2016–2018, is replicated in a virtual model of the original pyramidal stack of 7,500 red, pink, and blue barrels, part beached barge and part ageless neon ziggurat, as its name (“Mastaba”) refers to a Mesopotamian tomb.

The Google Arts and Culture app offers several AR art features, including the Art Projector, which places art objects and images from more than 2,000 cultural institutions anywhere you point your phone. The app is also used as a platform for more experimental applications of AR art. This includes AR Synth, which shows off AR art’s audio capabilities, aurally and visually modeling five famous vintage synthesizers, which users can drop into any ready-made studio space.

Also hosted by the Google Arts and Culture app is a collaboration with Serpentine Augmented Architecture. This installation, The Deep Listener by Jakob Kudsk Steensen, selects five locations in London’s Hyde Park landscape to showcase original 3D animations and sounds usually hidden from human ears. These emanate from five native and non-native species common to the area: There are bats’ nocturnal chirps; parakeet shrieks; reedbeds rattling in the wind; azure blue damselflies; and perhaps most subtle, the deep earth respiration and tunneling roots of the London Plane Tree. Each creature is abstracted and deconstructed to be somehow both flora and fauna, writhing elemental avatars of one of London’s most celebrated public spaces.

This article has been updated. It was originally published in April 2018.