Point to Point Rear Projection Feedback

For this project, I decided to tackle something that was more of an instalation piece and an expmeriment more than anything else. It was with this that I decided to create a rear projected display that went hand in hand with a C++ OpenGL simulation that enabled people to expierience something new and unique.

What isn't clear in the video, and isn't clear in any video, is what exactly is happening. Where the user is touching the display, they are actually feeling a vibrating feedback that lets them know something is happening. This is an experiment in point to point feedback and how it can affect people's experiences.

Front view of the simulation

The Simulation itself is a gravity experiment in which all of the particles are controlled by the gravity being applied to them. This gravity can be changed by touching any of the "stars" in the simulation which will increase their size and mass. This simulation, although not the best to emphasize the point to point feedback feature, allowed this prototype to be a great example of the advantage of point ot point feedback. It forced the users to only touch points which had bneen wired for feedback, rather than attempting to touch freely.

This effect was accomplished by using a Kinect V2 to track people's hands from the left of the Display. This confused many people, who spent some time looking for a tracking device, as tracking something from the left is an unusual concept. I mapped the Z and Y values to the simulation and then passed these serially to an arduino to control how which point was being vibrated. Behind the scenes, the people controlled a box which would collide with a set of spheres which would then pass that information serially. Again, this was in a C++ engine I wrote entirely myself.

You can view the code on github. The code for the arduino control circuit can be viewed here. This code allows me to interface with the Arduino and to interface back to the c++ program. Reading serial ports from within c++ actually proved to be a much more difficult problem than I had originally expected as you need to block a thread to read the input. I took a multi-threaded approach to this and added a switch to my arduino that allowed me to change experiements in my Engine.

I have not included a circuit diagram for this project as the circuit itself is rather simple: just wiring into a shift register. The hard part was the wiring of this into the display, which require dme to drill into the pvc piping and running wires through the piping to try and have as little appear on the display as possible.

Angled view of hte Rear Projection dispaly Back view of the rear projection display with the wiring

The most interesting aspect of this experiment was enabling people to do things they weren't used to. For one I allowed and encouraged people to touch a projector screen, which many people have been told to refrain from doing. Two, People got geniune feedback from a display of that size. Lastly, people were suprised when they didn't block the image by being in front of it. Seeing people's suprised faces and excitment while using the display made this entire project worth it. I am excited to take on my next challenge.

-Aidan McInerny