Hash3D: Training-free Acceleration
for 3D Generation
Preprint

Xingyi Yang, Xinchao Wang
National University of Singapore
Project Page arXiv Code

Examples by applying our Hash3D on Gaussian-Dreamer and Dream-Gaussian. We accelerate Gaussian-Dreamer by 1.5X and Dream-Gaussian by 4X with comparable visual quality.

Abstract

The evolution of 3D generative modeling has been notably propelled by the adoption of 2D diffusion models. Despite this progress, the cumbersome optimization process per se presents a critical hurdle to efficiency. In this paper, we introduce Hash3D, a universal acceleration for 3D generation without model training.

Central to Hash3D is the insight that feature-map redundancy is prevalent in images rendered from camera positions and diffusion time-steps in close proximity. By effectively hashing and reusing these feature maps across neighboring timesteps and camera angles, Hash3D substantially prevents redundant calculations, thus accelerating the diffusion model's inference in 3D generation tasks. We achieve this through an adaptive grid-based hashing. Surprisingly, this feature-sharing mechanism not only speed up the generation but also enhances the smoothness and view consistency of the synthesized 3D objects. Our experiments covering 6 text-to-3D and 3 image-to-3D models, demonstrate Hash3D's versatility to speed up optimization, enhancing efficiency by $1.3\sim 4\times$. Additionally, Hash3D's integration with 3D Gaussian splatting largely speeds up 3D model creation, reducing text-to-3D processing to about 10 minutes and image-to-3D conversion to roughly 30 seconds. The code is provided in https://github.com/Adamdad/Hash3D.

Main Method

Hash3D speeds up 3D generative model by caching and reusing these feature maps across neighboring timesteps and camera angles. We make this retrieval highly efficient through an adaptive grid-based hashing. It speeds up the running time by 1.3-4X.

Image-to-3D Results

Input Image Zero-1-to-3 Hash3D + Zero-1-to-3(Speed X3.3)
Input Image DreamGaussian Hash3D + DreamGaussian (Speed X4.0)
Sofa
Sofa
Sofa

Text-to-3D Results

Prompt Gaussian-Dreamer Hash3D + Gaussian-Dreamer
(Speed X1.5)
A bear dressed as a lumberjack
A chimpanzee dressed like Napoleon Bonaparte
A dachsund wearing a boater hat
A plate of delicious tacos
A train engine made out of clay

Related Resource

Our work is inspired by several work that focus on accelerating the inference of diffusion model and 3D reconstruction.

BibTeX

@misc{yang2024hash3d,
      title={Hash3D: Training-free Acceleration for 3D Generation}, 
      author={Xingyi Yang and Xinchao Wang},
      year={2024},
      eprint={2404.06091},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}