Mixture of Volumetric Primitives

We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a deconvolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions. Our parameterization supports the integration of correspondence and tracking constraints, while being robust to areas where classical tracking fails, such as around thin or translucent structures and areas with large topological variability. MVP is a hybrid that generalizes both volumetric and primitive-based representations.

Read the paper
Download the code

Example Results

Bibtex

@article{Lombardi21,
 author = {Lombardi, Stephen and Simon, Tomas and Schwartz, Gabriel and Zollhoefer, Michael and Sheikh, Yaser and Saragih, Jason},
 title = {Mixture of Volumetric Primitives for Efficient Neural Rendering},
 year = {2021},
 issue_date = {August 2021},
 publisher = {Association for Computing Machinery},
 address = {New York, NY, USA},
 volume = {40},
 number = {4},
 issn = {0730-0301},
 url = {https://doi.org/10.1145/3450626.3459863},
 doi = {10.1145/3450626.3459863},
 journal = {ACM Trans. Graph.},
 month = {jul},
 articleno = {59},
 numpages = {13},
 keywords = {neural rendering}
}