This project was done to test what is possible with some of the hardest situations to deal with in real time 3D, reflections. Movies have been able to represent accurate reflections for a very long time but the time it took for rendering two objects that reflect each other was astronomical.
Game engines have gotten really good at representing opaque physical materials that react accurately and predictably to light but it wasn’t until the latest generation of engines released that reflections became more than just a faint shimmer of something blurry moving across a surface.
So we set out to find a suitable candidate to see how far we can push real-time 3D while maintaining accurate reflections and solid performance on a mid-range computer.
The Nanyang Technological University Arts, Design and Media building has been a long time favorite and inspiration so it was selected as the ideal scenario to test what modern game engines are capable of.
Everything about this building is fascinating, from the three elliptical wings, to the way they for the reflection pool at the center and of course the fact that they create a 3-way reflection where you can see the other two wings reflected in every angle.
We realized quickly that the modern advance in reflections for game engines are perfectly suited for surfaces that are in view of the camera but are unable to capture anything that is outside the cameras view. So we had to come up with a custom system of capturing the environment when it’s out of view.
For that purpose we placed 295 cameras that capture a 360° texture close to the windows in an even spacing. We then applied those captured reflections onto the window panes with a vector looking up the direction that the reflected material should display.
This approach is actually very old in real-time 3D and has been around since the early 00s.
The resulting reflections are relatively inaccurate because they can only be drawn from the point of view of the capturing camera and games and visualizations used to be able to render only a small handful of them. But modern hardware and software optimizations allowed us to have a hundreds of them in our scene without noticeable performance loss and so we were able to capture very accurate reflections.
The old way is still the best way when paired with modern advances in glossy screen space reflections and optimized through the use of instanced materials that cut down on complex calculations and reliance on system memory.