One of the things that I am trying to do is figure out a way to have a hands free experience for mobile AR. One thing that I am looking it with this is using low-cost VR headsets and converting them into AR headset. With that, the books I make can be used with both hands rather than one hand with the phone in the other. One problem is that the UI buttons would not be interactable. In order to change that, I am trying to find ways to either use hand tracking or a Bluetooth remote to interact with the buttons. For the books, I thing the UI would come second to making a seamless handsfree AR experience with a reticle/crosshair and a timer to initiate the page experience. The reason I would do that is because I don't want there to be problems with the AR experience when the camera has multiple pages with multiple experiences.
Gaze interactions and virtual buttons are some of the areas in which I want to explore with this. It is focused on making a low-cost hololens alternative that everyone can utilize with their smart phones. The purpose is to increase immersion rather than making a robust life changing experience. Since I am doing that with books and I understand the push back books have particularly comic books in this scene, I am developing an approach that adds to the preexisting experience. People want to pick up a book and experience the art and story naturally. They want to open a double page spread and admire the visuals. What I am trying to do is take that experience and build on it. I want to let them have that same experience, but when they integrate their phone into it, they are able to still see the real world and the book in their hands. The only difference is that the pages they look at come to life. The characters speak, The speech bubbles pop out and possibly look towards the camera. The pictures move and show motion beyond an image.
I played around with some Gaze input and some cool camera effects on signs. I feel like I can implement this into some book project for handsfree interactions with AR headsets.
Links I used: