AR Activism
So with all this stuff going on in the world: COVID, Social Unrest, Police Brutality, I decided to make a statement with my work. I originally thought about going out and protesting with the rest of everybody else, but I was worried about the risked that was posed to myself and my girlfriend. To make it a long story short, my girlfriend works at a nursing home and cares for 40+ elderly residents 4 days out of the week. If I were to go out, I would be putting more than myself at risk. I am also applying to medical school, so I am worried that anything that were to happen would ultimately affect my goal of being a Black physician and combat health disparities we see in this nation. If I were to get arrested, or hit with tear gas, that would affect my aspirations for being a doctor.
Like many Black people seeing this crazy world, I think to myself what I could do to support the Black Lives Matter movement without putting my health and well being at risk. Mind you this is not because I am think about myself, but because the impact that I have right now is promoting diversity in art, tech, and the field of medicine, and also working to help Black creators create in XR. There is not many of us in particular industries outside of sports and entertainment, so we have to make the decisions on when we put ourselves at risk. Especially now because it takes time to even get to these positions. Part of the reason is because you have to go against the grain of what is expected of you and you have to navigate different areas without any guidance until you have the confidence, the skills, and insights for people to finally take you seriously. We are often worried that if we do not make it through, there probably will not be another one of us coming down the pipeline anytime soon.
So, I thought about what my biggest strength is. It has not really been going out and doing things. The coronavirus is very real, and I cannot afford to be sick. I have been doing a lot of YouTubing and Podcasting, so I decided to let my creativity drive my activism. If I can put the pen to the paper, I can make stuff that garners a community and pushes the conversation forward.
I really thought hard about this because when we interact and engage with these ideas, they are not in person. Most of them are through a screen. Literally all our understanding of the coronavirus and social issues is filtered through a screen from the transmitter to our senses. I think that is such a profound realization because there are so many filters that the information passes through before it comes to us. With that I can use the filtering of a screen to promote the causes I feel strongly about. I can Photoshop something and dig into a lot of creative efforts, but that would in many ways be antithetical to the whole point of this. The goal is to exist in spaces that are otherwise unwanting of you. This is a concept that Black people are well aware of. Whether we belong in places or not. If we do, we worry whether existing in those spaces will cause uproar from the majority. This is not without actual references I can find in my lifetime. It was not until 2002 when racist language in Oregon’s Constitution referring to Black people at Free Negroes and mulattoes removed. 30% of the population was opposed to that change too. Literally 1 in 3 Oregonians supported/felt comfortable referring to Black people in public policy as Free Negroes. Is it really hard to believe that 30% has not gone down given the climate we are living in right now?
So, I came up with the idea of using augmented reality for activism. To me it is the marrying of art and technology to express myself. A picture is worth a thousand words and technology can place that picture into a real world setting to contextualize that statement. I thought the concept would be great to explore because I could build an installation on the computer, go to a location, install the artwork, and share it with the world. More importantly, I did not have to worry about the physical limitations that come with traditional installations. If I were to make a 30 foot installation, I would have to worry about location, time, and resources to pull it off, then I would have to take a picture to memorialize the moment, and post it on the internet for the world to see. Then, you would have to deal with the consequences of that action: vandalism, trespassing, etc. In reality, most people will only know about this from pictures and videos anyway. Those are the things that get shared, and the things that people reference as a receipt for its existence. All this for a picture.
So how about we reduce the risk by focusing on the picture and video rather than the physical installation: replacing the analog with the digital to make an even more powerful statement. I could create an augmented reality app to make a 30-foot monument that I can place anywhere in the world I travel to, and install them freely. I can then take the install and share it with the world like other installations. I can repeat this multiple times with many other installations and reduce the amount of time it would take for me to create them because it is completely digital. Then I could share the app with others to participate in a movement that is of the times and on par with the way we intersect with the world during COVID.
The journey was really simple to be honest. It first started when I stumbled upon a 3D model of George Floyd. The creator posted it and I thought about how cool it would be to make an AR app for it. Adding some simple user controls and ground plane tracking can go a long way.
Setup Unity
So I started by downloading the Unity Game Engine. I opened up and switched the build settings to Android because it is much easier to build an app for android than iOS.
Setup Vuforia
I went on to Vuforia and downloaded the package for the AR engine to work. This is always pretty simple to do: download, install, and add the license key.
Afterwards, I made a new scene, added the AR camera, changed the ratio of the game view to 1920x1080 and landscape right, added a plane finder and ground plane stage to track the ground, deleted the main camera, turned off the duplicate stage to make sure we don’t keep added models to the AR scene, and I set the ground plane anchor.
Added the model
I downloaded the model from Sketchfab and added it to the AR scene as an OBJ. I added some lights to make the model more visible and resized it so that it wasn’t too big for the stage.
I tested it out with a pdf emulator that come with the Vuforia SDK so I can use it with my webcam instead of having to build it out to my phone all the time.
Added some UI
With the model done, I added a ui to the app to control the model. I went with some simple scale and rotation features. I added a canvas to the app, a panel to hold the assets, and two sliders. One slider for the scale, and another for the rotation. I then anchored them to the bottom of the screen and labeled them with some text.
Added Scale and Rotate features
In order to have the features work, I needed to add some code. It was only a few lines of code, so it was not too bad. I opened visual studio and made a script for the scale, added a slider reference, and had the value of the slider reference scale the x, y, and z, areas of the model.
I did the same thing for the rotation slider except for I only reference the y axis for the rotation value. The x and z axis were 0. I wanted it to rotate around a vertical pole only.
After coding, I added the scripts to the sliders, respectively. I first made an empty gameobject to place the model in the container. This is the container that I will use to actually rotate and scale the model. I dragged the scripts to the container then added the sliders to the reference area of the scripts. I then modified the values for the sliders. Rotation would go from 0 to 360 degrees, and the scale would go from .05 to 4.
I tested it out and was pleased.
Added a toggle for UI
I figured it would be nice to toggle the UI on and off with a subtle button so that it would not cover up half the screen when I wanted to take a picture or video. I added the toggle to the game object, removed the text, and changed the toggle indicator to a green circle icon. I anchored the toggle to the bottom center of the screen and created another panel to separate toggle from the other UI layers. The last thing I want to do is actually hide the toggle button when I hide everything else. Last thing I did was add a function to the toggle button by clicking + sign in the on value changed section and added the UI panel to it.
I clicked the play button to make sure it worked.
Created an icon
After the app was done, I put my designer hat on and decided to make a branded icon that spoke to the app. Default unity apps just have the unity icon on them. I didn’t feel like that was fitting of the experience so I made a custom one.
I took a screenshot of the 3D model on Sketchfab, opened up Photoshop and started building the icon. I made the icon a 1080 x 1080 square with some rounded corners, superimposed the image over the square and added it as a clipping mask. I added some flashy icon style effects to get the final icon look and exported it to Unity as a .png with transparency to maintain the rounded look. I then added it to the default icon list.
Created splash screens
After I finished the Icon, I wrapped up the app experience with some splash screens. I decided to add some intro screens into the app that gave some more information about the app. I added some memorial text and a message to support the cause, then I added some credits and references. I exported those from Photoshop and added them to the splash screens section. After organizing the slides and giving them the right timing, I exported the app out and went out into the world to test the app in the real world.
Testing the App
I live in Portland, Oregon. It is kind of crazy that I Portland is one of the epicenters of social unrest. Given that Black people only make up 5.8% of the Portland’s population and 3% of Oregon’s population, yet they have been protesting for almost 100 days straight, the feds have been snatching people up, white supremacist have been infiltrating police, and even the mayor got tear gassed. It is wild. Funny thing is that all of this has been happening around a few blocks in downtown. Portland blocks are small, so it really is only 3/4 of a New York block that got national attention. If you walk around Portland when the sun is out, it is a typical lily-white day with people jogging, cars passing, and dogs barking. When the sun goes down, that is when it really goes down.
Definitely, decided to go out during the day. With my mask on, I went out into the world of Portland and started putting the concept to practice. I traveled to the waterfront where most people go to walk and run, the many parking lots that have idled cars, and all the murals that have sprouted up as things began to shut down.
My girlfriend and I went on a journey to catch Pokémon and sight-see. To be honest, it was a great experience to go around Portland and see pro-Black icons and murals all over boarded buildings. It is one of the very few times where I felt accepted. Understanding the history of Oregon and their exclusion laws that hindered Black people from moving in and owning stuff, what we saw around the city was the antithesis of that history. We are really curious where they will put all this magnificent artwork. Unlike my project where I can put it in my pocket when I am done, this has a physical element that takes up space.
There were just so many great areas to place the Model. I varied the size of it, I did creative shots, I did gimmicky shots, I did full 360 shots, and I just had fun. I really tried to see how far I could push this project. For the most part, people thought I was just really into my phone and I was, but it was because I was admiring something that was very personal for me. One could argue that the whole point of monuments and murals is to exist in the public, but that was not the goal for me. As a Black man in Portland walking around a very white city, I wanted to feel comfortable. I did not want to try and admire something in a public forum looking over my shoulder worried about someone saying some racist stuff in response. IT is exhausting and sometimes I just do not want to deal with it. To be able to experience something that is grand and fantastical, but also discrete and personal is a major strength of immersive technology and augmented reality. I felt fortunate to be able to create such an intimate experience that I would be able to share with others.
Posting on Twitter
So after the journey, I posted the pictures I took on Twitter and boy was it a statement. Of course, there was some positive responses. Before I know it the post was shared across the XR industry by hundreds of people. It was great because people I followed on Twitter were interacting with it and receiving it well. This made me feel great because I was getting the responses that I wanted from this.
Unfortunately, when you put yourself out there, you must deal with the risk and consequences of it. For me, I know that saying Black Lives Matter on a public platform comes with some inherent risks. That risk often aligns with white supremacy and anti-Blackness. For the most part, I knew what types of responses from certain types of people I was inviting with my post. Interestingly enough they were so similar in nature and from pretty much all versions of white people.
In a sense, I feel like my goal was accomplished because I was able to create an installation in Portland that was shared all over the world in support of Black Live Matter movement, and I didn’t have to put my health or safety at risk in the process. Plus, I triggered a bunch of racists, and they could not do anything about it, but comment on Twitter about how much they don’t like it. Very rarely can a Black person make a statement and have an experience that has such a minimal risk and a maximum reach. The only thing I really must deal with is racists in my comments.
It was really fun to see how I was able to just add stuff to the world without any consequence. I was not worried about how white people felt as I put a 30-foot mural of a Black man in the center of the city. I could be unapologetically Black without feat of white supremacy rearing its ugly head. If only I could feel like this in any other area of my life. Despite people’s opinions of the installation on Twitter, there is a freedom and liberty to continue doing work that makes a statement and not have to worry about the repercussions. If they do not like it, oh well. They cannot do anything about it.
If you want to learn how to make AR apps, you can learn by taking the course I made based on the project on Skill share. You will learn the basics of augmented reality, how to use art in activism, how to build an augmented reality experience in Unity without any code, and how to enhance the experience of thought provoking art with a user interface and user controls. The app will place a 3D model into augmented reality and control the 3D model with onscreen buttons and sliders. You can check out the course here: https://skl.sh/2O2VRuO
If you like this, feel free to subscribe on Patreon to get a promo code to the course I made for this on SKillshare, and also the project files I used for this whole project!
Patreon: https://patreon.com/iltopia
