• The realm of visual effects is primarily about creating pseudo-realities – putting a dinosaur into a scene, replacing a blue screen stage with a stunning backdrop, creating a completely digital dogfight. For the most part, the goal of visual effects is to make the results as realistic and compelling as possible, and this has been the greatest challenge. While a lot of attention is focused purely on 3D and animation, there exists a whole area of research and development that is pushing the envelope in digital visual effects. Over the last decade, much research has been poured into ‘Computer Vision’ – trying to get computers to understand images and obtain usable information from them. This research has resulted in tools and concepts that are now finding their way into visual effects studios and changing the way that we produce images digitally.

    Famous for its image-based creation software is REALVIZ– a French company started in 1998 with the goal of commercializing products based on ten years of ‘Computer Vision’ research for robotics by INRIA (the French National Institute for Research in Computer Science and Control). Some of the core technologies brought to the table by REALVIZ have become ubiquitous in production environments worldwide: MatchMover for camera tracking, Image Modeler for modeling and texturing objects based on photographs, Stitcher for creating seamless panoramas and ReTimer for converting footage between frame rates.

    I was was given a unique opportunity to interview Dr. Luc Robert, Chief Technical Officer for REALVIZ about the current state of this segment of the industry.

    Leonard Teo: REALVIZ was founded in March 1998 based on research by INRIA. At the time of inception, did REALVIZ identify a market segment in the digital visual effects & entertainment industry? Is the current REALVIZ pantheon of products what was originally envisioned?

    Dr. Luc Robert: Yes, we especially identified a clear need for matchmoving products. Most of the post-production matchmoving work was done manually, except for companies using home-brewed software. Image-Based Modeling and Image-Based Rendering were also generating a big buzz, and we felt a big potential for these technologies in this market. In some sense, the collection of REALVIZ products is quite close to what we initially thought (except maybe for retiming which we discovered "by chance") -- that there was a real need, which could generate significant business. We also felt that computer vision technology was a way to make content creation easier and more productive for a variety of people, from the professional photographer to the 3D artist.

     Next Page (page 2 of 3) >>
  • Leonard Teo: What was the research at INRIA originally intended for?

    Dr. Luc Robert: The goal of the ROBOTVIS Computer Vision lab run by Pr Olivier Faugeras was to understand how vision works on live beings, and simulate it on robotic systems. One important application that resulted in many usable algorithms was a robot equipped with video cameras which could perceive its 3D environment, move while avoiding obstacles and track its 3D location within its environment.

    In the early 1990's, the focus of interest moved towards more visual applications such as Image-Based Modeling and Rendering. People started combining research on geometric perception, which allows you to reconstruct 3D shape and motion from images with image processing techniques that could, for instance, produce realistic textures. There started to be crossovers between the Computer Vision and Computer Graphics communities as well, to reproduce better, more realistic lighting as well as shape and materials. These topics are still hot in both research communities.

    Leonard Teo: On Matchmoving -- there is still a lot of skepticism regarding the use of automatic tracking, as the results may not be accurate or prone to "sliding/slipping". What's your take on this? Will automated matchmoving ever completely replace manual tracking?

    Dr. Luc Robert: The main benefit from matchmoving software is an increase in productivity. Automatic matchmoving is definitely a good help in this respect, since on a significant fraction of shots it produces a solution which requires no human interaction at all. Most post-production companies using automatic matchmoving have integrated it into their pipeline so that, for virtually no overhead cost, they can benefit from these automatic solutions. Despite this, there will always be shots which automatic matchmoving software will not be able to solve (the extreme case being where there is absolutely no element visible in the footage on which the tracking algorithms can rely). It is crucial for professionals to solve 100% of the shots they want to matchmove, and for that, the software has to allow them to control and guide the process if necessary.

    << Previous page (page 1 of 3)
  • Leonard Teo: On Image Modeling: Controversy surrounds the effectiveness of image-based modeling. Should image-based modeling be regarded as a replacement to traditional modeling or a complement to the existing tools available?

    Dr. Luc Robert: Image-Based Modeling allows one to reconstruct 3D shapes and textures representing something (or someone) which exists. It will never be a way to invent creatures or worlds, unless you first sketch them on paper or in plaster and then shoot pictures. In this respect, Image-Based Modeling is a complement to traditional modeling, just like 3D hardware scanners are. It is an incredible productivity-buster. No 3D artist really needs Image-Based Modeling to re-create existing material in 3D, but using image-based modeling they can do it much faster.

    Our tests with video game and post-production partners have proved that it reduces the cost of creating realistic 3D decors and objects by a factor of 7 to 10. Its main advantages compared to other 3D scanning approaches are a complete flexibility regarding the size of the object scanned (it works the same way for an ant or a mountain), complete control over the topology and the size of the mesh, and a very short acquisition time. Flexibility and productivity are definitely the two main arguments for professional 3D artists. Another interesting characteristic of Image-Based Modeling is that it makes the 3D modeling process easier, and accessible to a wider audience.

    Leonard Teo: What sort of technological enhancements do you expect to see permeate into the digital effects industry in the next few years?

    Dr. Luc Robert: Clean plate generation is easier, so it will most probably come first as a really useable feature, and it is what products like Pixeldust or Mokey do reasonably well in some cases. It is up to post-production companies to decide if the impact on production costs is worth the buy.

    Images: Two REALVIZ applications. (Left) Stitcher allows you to create full 360-degree cubic panoramas by using an array of stills. These can be used as background plates and reflection maps for difficult 3D shots. (Right) ImageModeller allows you to create a fully textured model based on a series of photographs of an object. This is extremely efficient and is increasingly being used in visual effects for CG replacements.

    Automated rotoscopy is another story. Roto is a huge budget-eater, and there is very little automation embedded in the available tools. It is also a difficult process to automate while meeting the professional’s quality requirements. A very interesting technological challenge!

    Leonard Teo: What's next for REALVIZ?

    Dr. Luc Robert: What I foresee for REALVIZ regarding these technologies is threefold. First, they will be improved and extended: more automatic, faster, better. Second, the integration of these techologies together and within the professional user's workflow will be made tighter to improve productivity. Third, thanks to a higher degree of automation, it will be possible for a wider audience, even down to the consumer, to use these technologies.

    REALVIZ website

    Words: Leonard Teo
    Special thanks to Liz Tjostolvsen and Dr. Luc Robert of REALVIZ.

    Back to 3D Festival

    << Previous page (page 2 of 3) 

blog comments powered by Disqus