Gallery #5

New Issue

2023-08-03T10:46:04+08:00

guochengqian commented

2023-08-03 10:46:04 +08:00

(Migrated from github.com)

Demo

A demo example using an ironman image from internet. We run Magic123 without textural inversion, taking 1 hour on a 32G V100 to finish.

Renderings:

https://github.com/guochengqian/Magic123/assets/48788073/d894e504-90f7-4b07-b0c7-ca644f759f57

Meshes:
ironman.zip

Command:

Run Magic123 coarse stage without textural inversion, takes ~40 mins

export RUN_ID='default-a-full-body-ironman'
export DATA_DIR='data/demo/ironman'
export IMAGE_NAME='rgba.png'
export FILENAME=$(basename $DATA_DIR)
export dataset=$(basename $(dirname $DATA_DIR))
CUDA_VISIBLE_DEVICES=0 python main.py -O \
--text "A high-resolution DSLR image of a full body ironman" \
--sd_version 1.5 \
--image ${DATA_DIR}/${IMAGE_NAME} \
--workspace out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse \
--optim adam \
--iters 5000 \
--guidance SD zero123 \
--lambda_guidance 1.0 40 \
--guidance_scale 100 5 \
--latent_iter_ratio 0 \
--normal_iter_ratio 0.2 \
--t_range 0.2 0.6 \
--bg_radius -1 \
--save_mesh

Run Magic123 fine stage without textural inversion, takes around ~20 mins

export RUN_ID='default-a-full-body-ironman'
export RUN_ID2='dmtet'
export DATA_DIR='data/demo/ironman'
export IMAGE_NAME='rgba.png'
export FILENAME=$(basename $DATA_DIR)
export dataset=$(basename $(dirname $DATA_DIR))
CUDA_VISIBLE_DEVICES=0 python main.py -O \
--text "A high-resolution DSLR image of a full body ironman" \
--sd_version 1.5 \
--image ${DATA_DIR}/${IMAGE_NAME} \
--workspace out/magic123-${RUN_ID}-${RUN_ID2}/$dataset/magic123_${FILENAME}_${RUN_ID}_${RUN_ID2} \
--dmtet --init_ckpt out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse/checkpoints/magic123_${FILENAME}_${RUN_ID}_coarse.pth \
--iters 5000 \
--optim adam \
--known_view_interval 4 \
--latent_iter_ratio 0 \
--guidance SD zero123 \
--lambda_guidance 1e-3 0.01 \
--guidance_scale 100 5 \
--rm_edge \
--bg_radius -1 \
--save_mesh

# Demo A demo example using an ironman image from internet. We run Magic123 without textural inversion, taking 1 hour on a 32G V100 to finish. Renderings: https://github.com/guochengqian/Magic123/assets/48788073/d894e504-90f7-4b07-b0c7-ca644f759f57 Meshes: [ironman.zip](https://github.com/guochengqian/Magic123/files/12255259/ironman.zip) ![image](https://github.com/guochengqian/Magic123/assets/48788073/f8b98e3e-764d-43c0-bda0-9e806cdd6d5a) ![image](https://github.com/guochengqian/Magic123/assets/48788073/4e9b9f1f-f877-4d37-afbd-96e012028ec1) Command: - Run Magic123 coarse stage without textural inversion, takes ~40 mins ``` export RUN_ID='default-a-full-body-ironman' export DATA_DIR='data/demo/ironman' export IMAGE_NAME='rgba.png' export FILENAME=$(basename $DATA_DIR) export dataset=$(basename $(dirname $DATA_DIR)) CUDA_VISIBLE_DEVICES=0 python main.py -O \ --text "A high-resolution DSLR image of a full body ironman" \ --sd_version 1.5 \ --image ${DATA_DIR}/${IMAGE_NAME} \ --workspace out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse \ --optim adam \ --iters 5000 \ --guidance SD zero123 \ --lambda_guidance 1.0 40 \ --guidance_scale 100 5 \ --latent_iter_ratio 0 \ --normal_iter_ratio 0.2 \ --t_range 0.2 0.6 \ --bg_radius -1 \ --save_mesh ``` - Run Magic123 fine stage without textural inversion, takes around ~20 mins ``` export RUN_ID='default-a-full-body-ironman' export RUN_ID2='dmtet' export DATA_DIR='data/demo/ironman' export IMAGE_NAME='rgba.png' export FILENAME=$(basename $DATA_DIR) export dataset=$(basename $(dirname $DATA_DIR)) CUDA_VISIBLE_DEVICES=0 python main.py -O \ --text "A high-resolution DSLR image of a full body ironman" \ --sd_version 1.5 \ --image ${DATA_DIR}/${IMAGE_NAME} \ --workspace out/magic123-${RUN_ID}-${RUN_ID2}/$dataset/magic123_${FILENAME}_${RUN_ID}_${RUN_ID2} \ --dmtet --init_ckpt out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse/checkpoints/magic123_${FILENAME}_${RUN_ID}_coarse.pth \ --iters 5000 \ --optim adam \ --known_view_interval 4 \ --latent_iter_ratio 0 \ --guidance SD zero123 \ --lambda_guidance 1e-3 0.01 \ --guidance_scale 100 5 \ --rm_edge \ --bg_radius -1 \ --save_mesh

guochengqian commented

2023-08-16 02:37:14 +08:00

(Migrated from github.com)

Failure Case

Not optimal for human reconstruction. Suggest to use SMPL model as an additional prior, which I believe will directly boost the performance to realism.

https://github.com/guochengqian/Magic123/assets/48788073/188af740-ad27-4e52-a562-d9a0d2338639

# Failure Case Not optimal for human reconstruction. Suggest to use SMPL model as an additional prior, which I believe will directly boost the performance to realism. https://github.com/guochengqian/Magic123/assets/48788073/188af740-ad27-4e52-a562-d9a0d2338639

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dearsky/Magic123#5