[点云+深度学习]PointNetLK论文解析

发表于
精读点云配准的文章PointNetLK。

《PointNetLK: Robust & Efficient Point Cloud Registration using PointNet》

代码运行

文件 描述
ex1_train.sh 训练PointNet分类器，并转移到PointNetLK
ex1_genrot.sh 为测试生成扰动
ex1_test_pointlk.sh 测试PointNetLK
ex1_test_icp.sh 测试ICP
ex1_result_stat.sh 计算上述测试的平均误差

离线运行代码

1
nohup python train.py >> log_train.out &

ex1_train.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# 1. 切换目录，为输出创建文件夹
cd /home/amax/xw/xgy/pointcloud/PointNetLK-master/experiments
mkdir results

# 2. 首先，训练一个分类器（跑7小时）
python train_classifier.py \
 -o ex1_classifier_190408 \
 -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 \
 -c ./sampledata/modelnet40_half1.txt \
 -l ex1_classifier_190408.log \
 -j 8 \
 -b 256 \

# 结果之一保存在 results/ex1_classifier_190408_feat_best.pth
# 此文件是计算PointNet特征的模型

# 3. 训练PointNetLK，为了分类微调PointNet特征（上述文件）（跑19小时）
python train_pointlk.py \
 -o ex1_pointlk_190408 \
 -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 \
 -c ./sampledata/modelnet40_half1.txt \
 -l ex1_pointlk_190408.log \
 --transfer-from ex1_classifier_190408_feat_best.pth \
 --epochs 400 \
 -j 8 \
 
# 训练模型 results/ex1_pointlk_190408_model_best.pth

移动pth等到results文件夹中？？？

ex1_genrot.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 为每个物体生成扰动（每个'ModelNet40/[category]/test/*'）

# 1. 切换目录，为输出创建文件夹
cd /home/amax/xw/xgy/pointcloud/PointNetLK-master/experiments
mkdir -p results/ex1/gt

# 2. 测试类别（19条命令，只有最后不同"pert_000.csv --deg 0.0"）
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_000.csv --deg 0.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_010.csv --deg 10.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_020.csv --deg 20.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_030.csv --deg 30.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_040.csv --deg 40.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_050.csv --deg 50.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_060.csv --deg 60.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_070.csv --deg 70.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_080.csv --deg 80.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_090.csv --deg 90.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_100.csv --deg 100.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_110.csv --deg 110.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_120.csv --deg 120.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_130.csv --deg 130.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_140.csv --deg 140.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_150.csv --deg 150.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_160.csv --deg 160.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_170.csv --deg 170.0
python generate_rotations.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/gt/pert_180.csv --deg 180.0

ex1_test_pointlk.sh

1
2
3
4
5
6
7
8
9
10
11
# 1. 切换目录，为输出创建文件夹
cd /home/amax/xw/xgy/pointcloud/PointNetLK-master/experiments
mkdir -p results/ex1/plk

# 2. 测试和训练模型的类别、扰动
# 使用给定的扰动测试PointNetLK（19条命令，只有微小不同"result_000.csv pert_000.csv log_000.log"）
# （最后可以加上" --device cuda:0"，但会报显存溢出）
python test_pointlk.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt --pretrained ./results/ex1_pointlk_190408_model_best.pth -o ./results/ex1/plk/result_000.csv -p ./results/ex1/gt/pert_000.csv -l ./results/ex1/plk/log_000.log -j 8
python test_pointlk.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt --pretrained ./results/ex1_pointlk_190408_model_best.pth -o ./results/ex1/plk/result_010.csv -p ./results/ex1/gt/pert_010.csv -l ./results/ex1/plk/log_010.log -j 8
python test_pointlk.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt --pretrained ./results/ex1_pointlk_190408_model_best.pth -o ./results/ex1/plk/result_020.csv -p ./results/ex1/gt/pert_020.csv -l ./results/ex1/plk/log_020.log -j 8
python test_pointlk.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt --pretrained ./results/ex1_pointlk_190408_model_best.pth -o ./results/ex1/plk/result_030.csv -p ./results/ex1/gt/pert_030.csv -l ./results/ex1/plk/log_030.log -j 8

ex1_test_icp.sh

1
2
3
4
5
6
7
8
9
10
11
# 1. 切换目录，为输出创建文件夹
cd /home/amax/xw/xgy/pointcloud/PointNetLK-master/experiments
mkdir -p results/ex1/icp

# 2. 测试的类别、扰动
# 使用给定的扰动测试ICP（19条命令，只有微小不同"result_000.csv pert_000.csv log_000.log"）
# 会运行几天（5天左右），提升线程数对速度似乎没有影响
python test_icp.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/icp/result_000.csv -p ./results/ex1/gt/pert_000.csv -l ./results/ex1/icp/log_000.log -j 8
python test_icp.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/icp/result_010.csv -p ./results/ex1/gt/pert_010.csv -l ./results/ex1/icp/log_010.log -j 8
python test_icp.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/icp/result_020.csv -p ./results/ex1/gt/pert_020.csv -l ./results/ex1/icp/log_020.log 
python test_icp.py -i /home/amax/xw/xgy/pointcloud/PointNetLK-master/ModelNet40 -c ./sampledata/modelnet40_half1.txt --format wt -o ./results/ex1/icp/result_030.csv -p ./results/ex1/gt/pert_030.csv -l ./results/ex1/icp/log_030.log

ex1_result_stat.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 添加 'result_*.csv' 到 'result.csv'
#RES= ./results/ex1/plk
# （19条命令，只有微小不同"result_000.csv --val 0"）
python result_stat.py --hdr > ./results/ex1/plk/result.csv
python result_stat.py -i ./results/ex1/plk/result_000.csv --val 0 >> ./results/ex1/plk/result.csv
python result_stat.py -i ./results/ex1/plk/result_010.csv --val 10 >> ./results/ex1/plk/result.csv
python result_stat.py -i ./results/ex1/plk/result_020.csv --val 20 >> ./results/ex1/plk/result.csv
python result_stat.py -i ./results/ex1/plk/result_030.csv --val 30 >> ./results/ex1/plk/result.csv

#RES= ./results/ex1/icp
python result_stat.py --hdr > ./results/ex1/icp/result.csv
python result_stat.py -i ./results/ex1/icp/result_000.csv --val 0 >> ./results/ex1/icp/result.csv
python result_stat.py -i ./results/ex1/icp/result_010.csv --val 10 >> ./results/ex1/icp/result.csv
python result_stat.py -i ./results/ex1/icp/result_020.csv --val 20 >> ./results/ex1/icp/result.csv
python result_stat.py -i ./results/ex1/icp/result_030.csv --val 30 >> ./results/ex1/icp/result.csv

ex1_genpert.sh

针对的是CADset4tracking_fixed_perturbation数据集

代码解析

文件 描述
train_classifier.py 训练PointNet分类器（用于迁移学习）
train_pointlk.py 训练PointNetLK
generate_rotation.py 生成6自由度的扰动（旋转和平移）（为了测试）
test_pointlk.py 测试PointNetLK
test_icp.py 测试ICP
result_stat.py 计算上述测试的平均误差

generate_rotations.py

实测下来，wv与wt的旋转相同（前3位），平移不同（后3位）（疑似是wv的实现有问题）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import torch
import numpy as np
import math

# 命令行指定的参数
_deg = 60.0 # 扰动角度
_max_trans = 0.3 # 最大偏移
_format = 'wt' # wv (twist-vector), wt means rotation- and translation-vector)
_outfile = 'pert_060.csv'

# batch_size由训练集长度决定，batch_size = len(testset)
batch_size = 1202

amp = _deg * math.pi / 180.0 # 角度转弧度
w = torch.randn(batch_size, 3)
w = w / w.norm(p=2, dim=1, keepdim=True) * amp
t = torch.rand(batch_size, 3) * _max_trans

if _format == 'wv':
	# the output: twist vectors.
	import ptlk
	R = ptlk.so3.exp(w) # (N, 3) --> (N, 3, 3) # 实际是罗德里格斯变换，等价于cv2.Rodrigues(np.array(w[0]))[0]
	G = torch.zeros(batch_size, 4, 4)
	G[:, 3, 3] = 1
	G[:, 0:3, 0:3] = R
	G[:, 0:3, 3] = t

	x = ptlk.se3.log(G) # --> (N, 6) # 前3位旋转可以用罗德里格斯变换求解，等价于cv2.Rodrigues(np.array(G[0][0:3,0:3]))[0]，后3位未知
else:
	# rotation-vector and translation-vector
	x = torch.cat((w, t), dim=1) # w是旋转、t是平移

# np.savetxt(_outfile, x, delimiter=',')

result_stat.py

读取 test_pointlk.py/test_icp.py 生成的csv，计算平均误差

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
1. test_pointlk.py/test_icp.py 保存的csv有12列，
前6列是预测的值，后6列是真实的值：

['h_w1', 'h_w2', 'h_w3', 'h_v1', 'h_v2', 'h_v3', \
 'g_w1', 'g_w2', 'g_w3', 'g_v1', 'g_v2', 'g_v3']
# h: estimated
# g: ground-truth twist vectors

2. result_stat.py 保存的csv有5列
第1个值是文件索引
第2、3个值是估计的姿态的平移、旋转误差


['val, me_pos, me_rad, me_twist, me_vel']
# val: given value (for x-axis)
# me_pos: mean error of estimated position (distance)
# me_rad: mean error of estimated rotation (angle in radian)
# me_twist: mean error represented as norm of twist vector
# me_vel: translation part of the twist. (rotation part is me_rad)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import torch
import numpy as np
import ptlk

_infile = 'result_010.csv'
_val = 20

# 1. 读取csv，并分离为预测值、真实值
npdata = np.loadtxt(args.infile, delimiter=',', skiprows=1)  # --> (N, 12)
res = torch.from_numpy(npdata).view(-1, 12)
x_hat = res[:, 0:6]   # estimated twist vector
x_mgt = -res[:, 6:12] # (minus) ground-truth

# 2. 预测值转为4*4矩阵 乘 真实值转为4*4矩阵的逆，如果两者很接近，相乘结果应接近单位阵
g_hat = ptlk.se3.exp(x_hat) # [N, 4, 4], estimated matrices
g_igt = ptlk.se3.exp(x_mgt) # [N, 4, 4], inverse of ground-truth
dg = g_hat.bmm(g_igt) # [N, 4, 4]. if correct, dg == identity matrices.

# 3. 取相乘结果的平移部分，转为长度为6的向量，取向量前3个数（旋转），取向量后3个数（平移）
dp = dg[:, 0:3, 3]    # [N, 3], position error
dx = ptlk.se3.log(dg) # [N, 6], twist error
dw = dx[:, 0:3]       # [N, 3], rotation part of the twist error
dv = dx[:, 3:6]       # [N, 3], translation part

# 4. 求上述4个数据的2范数，作为结果输出
ep = dp.norm(p=2, dim=1) # [N]
ex = dx.norm(p=2, dim=1) # [N]
ew = dw.norm(p=2, dim=1) # [N]
ev = dv.norm(p=2, dim=1) # [N]

e = torch.stack((ep, ew, ex, ev)) # [4, N]
me = torch.mean(e, dim=1) # [4]

line = ','.join(map(str, me.numpy().tolist()))
print('{},{}'.format(_val, line))

参考

官方

内容 描述
原始论文：PointNetLK 演示视频
代码：PointNetLK PyTorch 0.4.0