Compressible-composable NeRF via Rank-residual Decomposition

NeurIPS 2022


Jiaxiang Tang1, Xiaokang Chen1, Jingbo Wang2, Gang Zeng1

1 Peking University   2 Chinese University of Hong Kong  

Abstract


teaser

Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering. However, its implicit representation causes difficulty in manipulating the models like the explicit mesh representation. Several recent advances in NeRF manipulation are usually restricted by a shared renderer network, or suffer from large model size. To circumvent the hurdle, in this paper, we present an explicit neural field representation that enables efficient and convenient manipulation of models. To achieve this goal, we learn a hybrid tensor rank decomposition of the scene without neural networks. Motivated by the low-rank approximation property of the SVD algorithm, we propose a rank-residual learning strategy to encourage the preservation of primary information in lower ranks. The model size can then be dynamically adjusted by rank truncation to control the levels of detail, achieving near-optimal compression without extra optimization. Furthermore, different models can be arbitrarily transformed and composed into one scene by concatenating along the rank dimension. The growth of storage cost can also be mitigated by compressing the unimportant objects in the composed scene. We demonstrate that our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition.


Compression


Our model supports near-optimal low-rank approximation of neural fields without extra optimization, similar to the SVD compression of images.

  • Optimal: retrained at each different rank.
  • Baseline: sort-and-truncated from the optimal model at full rank.
  • Proposed: sort-and-truncated from our proposed model at full rank.

optimal baseline proposed

Composition


Since our model describes each 3D scene with a set of rank components, composability can be naturally accomplished by concatenating along the rank dimension.


LoD

Citation


@article{tang2022compressible,
  title={Compressible-composable NeRF via Rank-residual Decomposition},
  author={Tang, Jiaxiang and Chen, Xiaokang and Wang, Jingbo and Zeng, Gang},
  journal={arXiv preprint arXiv:2205.14870},
  year={2022}
}