Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 1bde1c1

Browse filesBrowse files
mingyuan-zhangmingyuan-zhang
committed
Update visualization tools for T2M
1 parent 8aa226e commit 1bde1c1
Copy full SHA for 1bde1c1

File tree

Expand file treeCollapse file tree

15 files changed

+1258
-5
lines changed
Filter options
Expand file treeCollapse file tree

15 files changed

+1258
-5
lines changed

‎README.md

Copy file name to clipboardExpand all lines: README.md
+4-4Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,10 @@ This repository contains the official implementation of _MotionDiffuse: Text-Dri
5252

5353
[10/2022] Code release for text-driven motion generation!
5454

55+
## Text-driven Motion Generation
56+
57+
You may refer to [readme](text2motion/README.md) for detailed introduction.
58+
5559
## Citation
5660

5761
If you find our work useful for your research, please consider citing the paper:
@@ -65,10 +69,6 @@ If you find our work useful for your research, please consider citing the paper:
6569
}
6670
```
6771

68-
## Text-driven Motion Generation
69-
70-
You may refer to [readme](text2motion/README.md) for detailed introduction.
71-
7272
## Acknowledgements
7373

7474
This study is supported by NTU NAP, MOE AcRF Tier 2 (T2EP20221-0033), and under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).

‎figures/gallery_t2m/gen_00.gif

Copy file name to clipboard
530 KB
Loading

‎figures/gallery_t2m/gen_01.gif

Copy file name to clipboard
844 KB
Loading

‎figures/gallery_t2m/gen_02.gif

Copy file name to clipboard
578 KB
Loading

‎figures/gallery_t2m/gen_03.gif

Copy file name to clipboard
310 KB
Loading

‎figures/gallery_t2m/gen_04.gif

Copy file name to clipboard
714 KB
Loading

‎figures/gallery_t2m/gen_05.gif

Copy file name to clipboard
371 KB
Loading

‎figures/gallery_t2m/gen_06.gif

Copy file name to clipboard
537 KB
Loading

‎figures/gallery_t2m/gen_07.gif

Copy file name to clipboard
612 KB
Loading

‎text2motion/README.md

Copy file name to clipboardExpand all lines: text2motion/README.md
+36Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,42 @@ python -u tools/evaluation.py checkpoints/kit/kit_motiondiffuse/opt.txt GPU_ID
4949
python -u tools/evaluation.py checkpoints/kit/kit_motiondiffuse/opt.txt
5050
```
5151

52+
## Visualization
53+
54+
You can visualize human motion with the given language description and the expected motion length.
55+
56+
```shell
57+
# Currently we only support visualization of models trained on the HumanML3D dataset.
58+
# Motion length can not be larger than 196, which is the maximum length during training
59+
# You can omit `gpu_id` to run visualization on your CPU
60+
61+
python -u tools/visualization.py \
62+
--opt_path checkpoints/t2m/t2m_motiondiffuse/opt.txt \
63+
--text "a person is jumping" \
64+
--motion_length 60 \
65+
--result_path "test_sample.gif" \
66+
--gpu_id 0
67+
```
68+
69+
Here are some visualization examples. The motion lengths are shown in the title of animations.
70+
71+
<table>
72+
<tr>
73+
<td><img src="../figures/gallery_t2m/gen_00.gif" width="100%"/></td>
74+
<td><img src="../figures/gallery_t2m/gen_01.gif" width="100%"/></td>
75+
<td><img src="../figures/gallery_t2m/gen_02.gif" width="100%"/></td>
76+
<td><img src="../figures/gallery_t2m/gen_03.gif" width="100%"/></td>
77+
</tr>
78+
<tr>
79+
<td><img src="../figures/gallery_t2m/gen_04.gif" width="100%"/></td>
80+
<td><img src="../figures/gallery_t2m/gen_05.gif" width="100%"/></td>
81+
<td><img src="../figures/gallery_t2m/gen_06.gif" width="100%"/></td>
82+
<td><img src="../figures/gallery_t2m/gen_07.gif" width="100%"/></td>
83+
</tr>
84+
</table>
85+
86+
**Note:** You may install `matplotlib==3.3.1` to support visualization here.
87+
5288
## Acknowledgement
5389

5490
This code is developed on top of [Generating Diverse and Natural 3D Human Motions from Text](https://github.com/EricGuo5513/text-to-motion)

‎text2motion/requirements.txt

Copy file name to clipboardExpand all lines: text2motion/requirements.txt
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@ tqdm
22
opencv-python
33
clip
44
scipy
5-
matplotlib
5+
matplotlib==3.3.1
66
spacy
77
git+https://github.com/openai/CLIP.git

‎text2motion/tools/visualization.py

Copy file name to clipboard
+80Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
import os
2+
import torch
3+
import argparse
4+
from os.path import join as pjoin
5+
6+
import utils.paramUtil as paramUtil
7+
from torch.utils.data import DataLoader
8+
from utils.plot_script import *
9+
from utils.get_opt import get_opt
10+
from datasets.evaluator_models import MotionLenEstimatorBiGRU
11+
12+
from trainers import DDPMTrainer
13+
from models import MotionTransformer
14+
from utils.word_vectorizer import WordVectorizer, POS_enumerator
15+
from utils.utils import *
16+
from utils.motion_process import recover_from_ric
17+
18+
19+
def plot_t2m(data, result_path, caption):
20+
joint = recover_from_ric(torch.from_numpy(data).float(), opt.joints_num).numpy()
21+
# joint = motion_temporal_filter(joint, sigma=1)
22+
plot_3d_motion(result_path, paramUtil.t2m_kinematic_chain, joint, title=caption, fps=20)
23+
24+
25+
def build_models(opt):
26+
encoder = MotionTransformer(
27+
input_feats=opt.dim_pose,
28+
num_frames=opt.max_motion_length,
29+
num_layers=opt.num_layers,
30+
latent_dim=opt.latent_dim,
31+
no_clip=opt.no_clip,
32+
no_eff=opt.no_eff)
33+
return encoder
34+
35+
36+
if __name__ == '__main__':
37+
parser = argparse.ArgumentParser()
38+
parser.add_argument('--opt_path', type=str, help='Opt path')
39+
parser.add_argument('--text', type=str, default="", help='Text description for motion generation')
40+
parser.add_argument('--motion_length', type=int, default=60, help='Number of frames for motion generation')
41+
parser.add_argument('--result_path', type=str, default="test_sample.gif", help='Path to save generation result')
42+
parser.add_argument('--gpu_id', type=int, default=-1, help="which gpu to use")
43+
args = parser.parse_args()
44+
45+
device = torch.device('cuda:%d' % args.gpu_id if args.gpu_id != -1 else 'cpu')
46+
opt = get_opt(args.opt_path, device)
47+
opt.do_denoise = True
48+
49+
assert opt.dataset_name == "t2m"
50+
assert args.motion_length <= 196
51+
opt.data_root = './dataset/HumanML3D'
52+
opt.motion_dir = pjoin(opt.data_root, 'new_joint_vecs')
53+
opt.text_dir = pjoin(opt.data_root, 'texts')
54+
opt.joints_num = 22
55+
opt.dim_pose = 263
56+
dim_word = 300
57+
dim_pos_ohot = len(POS_enumerator)
58+
num_classes = 200 // opt.unit_length
59+
60+
mean = np.load(pjoin(opt.meta_dir, 'mean.npy'))
61+
std = np.load(pjoin(opt.meta_dir, 'std.npy'))
62+
63+
w_vectorizer = WordVectorizer('./data/glove', 'our_vab')
64+
encoder = build_models(opt).cuda()
65+
trainer = DDPMTrainer(opt, encoder)
66+
trainer.load(pjoin(opt.model_dir, opt.which_epoch + '.tar'))
67+
68+
trainer.eval_mode()
69+
trainer.to(opt.device)
70+
71+
result_dict = {}
72+
with torch.no_grad():
73+
if args.motion_length != -1:
74+
caption = [args.text]
75+
m_lens = torch.LongTensor([args.motion_length]).to(device)
76+
pred_motions = trainer.generate(caption, m_lens, opt.dim_pose)
77+
motion = pred_motions[0].cpu().numpy()
78+
motion = motion * std + mean
79+
title = args.text + " #%d" % motion.shape[0]
80+
plot_t2m(motion, args.result_path, title)

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.