Review) Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion

🖼️ 단일 이미지에서 3D 형태, 포즈 및 외관 복구를 위한 주요 기법 제안.  
🔄 하이브리드 인버전 기법을 통해 초기 추정치를 최적화하여 정확도 향상.  
📏 SDF 표현을 통해 복잡한 형상도 효과적으로 모델링 가능.

 

단일 이미지로부터 3D 형상, 자세 및 외관을 복원하는 방법론을 제시

- Neural Radiance Fields(NeRF)와 GANs를 효과적으로 결합하여, 단일 뷰에서 3D 복원을 가능

- 기존의 연구들이 주로 정밀한 카메라 자세가 알려진 합성 데이터셋에 초점을 맞추었던 반면, 이 연구는 자연 이미지에서의 포즈 추정 문제

 

1. 3D-aware GAN을 기반으로 하는 조건 없는 생성기 G를 훈련

2. 생성기 G가 훈련된 후, 이미지를 입력받아 초기 잠재 코드와 자세를 추정하는 인코더 E를 훈련

3. 경량화된 GAN 역전 방법을 사용하는데, 이는 기존의 스스로 코드를 검색하는 방식과 결합하여 초기 예측을 최적화

 

 

git : https://github.com/google-research/nerf-from-image.git

GitHub - google-research/nerf-from-image: Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Invers

- 싱글 이미지 input

- pretrained model github에서 제공

- detectron2 설치해야함 

!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'