My Contributions: Technical Art, XR UX Design, Asset Creation
The COVID-19 pandemic has made it difficult for live events, such as concerts, to take place. This unfortunate problem was the inspiration for my augmented reality app: "Rat Parade AR." What I love about working with AR projects is that many of the conventions do not exist yet. I can approach the problem whichever way I think is best, which I find greatly rewarding.
Since we can't easily gather at a venue to watch a band perform, I wanted to try to bring the live rock show experience to the user, wherever they are, by utilizing Apple's ARKit. This proof of concept involved shooting video of the band on a green screen, adding 3D elements like the drums and amps, as well as adding interactive elements like the ability to spawn rats and record video using ReplayKit.
The most important part of all this is getting the band into the scene in AR. The most logical way to pull this off was to have the band members all perform individually on a green screen, and then have the videos play back in Unity with alpha to sound sync. After adding the 3D models the experience was starting to look a lot more like a real show. I used Cinema Director, a tool I previously assisted in the development of, to sequence the events that occur throughout the song. The backdrop comes in and out, rats spawn in and run around, and other surprises are had throughout the experience.
Next, I wanted to expand the app by adding interactive features. I added a video recorder using ReplayKit and a share button in order to share the recorded videos via the iOS Share Sheet. I also added a play/pause button and some links to the song on Apple Music and Spotify at the end. My goal was to keep the UI as unobtrusive as possible and only consists of a few circular buttons on the edge of the display.
The reception to our "AR Music Video" was great, and I'm currently working to expand the scope of the app to include more songs through a DLC system.
Rat Parade AR is now available to download on the Apple App Store. Feel free to check it out!
