Thursday, July 7, 2022

Nvidia’s new AI magic turns 2D photos into 3D graphics

HometechNvidia’s new AI magic turns 2D photos into 3D graphics

Nvidia has made another effort to add depth to images that lack graphics, The technique targets video game designs.

After transforming 2D photographs into 3D sceneries, models, and films, the business is now concentrating on editing.

Today, the GPU titan presented a new AI technology that converts still images into easily modifiable 3D models.

ALSO READ: Facebook, US Settle Lawsuit Over Discriminatory Housing Advertising System

The approach, dubbed 3D MoMa, might provide game producers with an easy way to modify graphics and settings. This often requires time-consuming photogrammetry, which measures objects based on photographs.

Using inverse rendering, 3D MoMa accelerates the work. Analyzing still photos, this procedure use artificial intelligence to determine a scene’s physical qualities, from geometry to illumination. The images are then rebuilt in a 3D format that is realistic.

David Luebke, vice president of graphics research at Nvidia, refers to the method as “the holy grail of computer vision and computer graphics.”

Triangle meshes define the shapes used in 3D graphics and modeling.
Triangle meshes define the shapes used in 3D graphics and modeling. Credit: Nvidia

“By formulating each component of the inverse rendering problem as a GPU-accelerated differentiable component, the NVIDIA 3D MoMa rendering pipeline uses the machinery of modern AI and the raw computational power of NVIDIA GPUs to produce 3D objects that creators can import, edit, and extend without limitation using existing tools,” said Lubeke.

3D MoMa creates models as triangular meshes, a format that is easily editable with common software. The models are produced on a single NVIDIA Tensor Core GPU in about an hour.

The meshes can then be covered with materials similar to skins. Also anticipated is the scene’s lighting, allowing designers to customise its impact on the items.

This week’s Computer Vision and Pattern Recognition Conference (CVPR) in New Orleans included 3D MoMa. In honour to the origin of jazz, Nvidia researchers created a visual representation of the musical genre using this method.

Initially, the team gathered many images of trumpets, trombones, saxophones, drums, and clarinets. The photos were then rebuilt into 3D representations via 3D MoMa.

The instruments were then revised and provided with new components. For instance, the trumpet was converted from inexpensive plastic to luxurious gold.

The freshly altered instruments were then suitable for use in any virtual scenario. They were placed in a Cornell box, which is used to evaluate rendering quality.

According to the business, all of the instruments reacted to light like they would in the real world, with brass instruments reflecting light brightly and drum skins absorbing light.

The 3D items were then recreated into an animated scenario.

Nvidia hopes that 3D MoMa will enable game developers and other designers to rapidly change 3D objects and add them to any virtual scenario.

This might also facilitate our transformation into metaverse forms.

Here you may read the research paper underlying 3D MoMa.