mirror of
https://github.com/wassname/Flex-Convolution.git
synced 2026-09-09 11:14:10 +08:00
edit readme and add project page
This commit is contained in:
PatWie
Binary file not shown.
|
After Width: | Height: | Size: 513 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 758 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 361 KiB |
@@ -1,4 +1,4 @@
|
||||
# Flex-Convolution (Million-Scale Pointcloud Learning Beyond Grid-Worlds)
|
||||
# Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)
|
||||
Fabian Groh, Patrick Wieschollek, Hendrik P.A. Lensch
|
||||
|
||||
|
||||
@@ -14,7 +14,10 @@ The following figure shows the *raw* network semantic segmentation prediction on
|
||||
<p align="center"> <img src=".github/flexconv.jpg" width="100%"> </p>
|
||||
|
||||
|
||||
This repository contains contains the source code of our FlexConv Layer from our 2018 ACCV paper "Flex-Convolution (Million-Scale Pointcloud Learning Beyond Grid-Worlds)".
|
||||
This repository contains contains the source code of our FlexConv Layer from our 2018 ACCV paper "Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)".
|
||||
|
||||
<p align="center"> <a href="https://www.youtube.com/watch?v=5ftWmuQXU_s"><img src="./.github/youtube.jpg" width="50%"></a> </p>
|
||||
|
||||
|
||||
Example - Usage
|
||||
-------------------
|
||||
@@ -25,7 +28,7 @@ We provide GPU-tailored cuda implementation of our novel FlexConv in TensorFlow.
|
||||
user@host $ cd user_ops
|
||||
user@host $ cmake . -DPYTHON_EXECUTABLE=python2 && make -j
|
||||
user@host $ cd ..
|
||||
user@host $ python example
|
||||
user@host $ python example.py
|
||||
```
|
||||
|
||||
|
||||
@@ -34,13 +37,53 @@ Experiments
|
||||
|
||||
Deep learning on point-clouds is a complex matter and our codebase reflect that complexity.
|
||||
We are currently working on refactoring our research implementation to ease the usage. Therefore,
|
||||
`layers.py` contains a Keras/tf.layers compatible implementation.
|
||||
`layers.py` contains a Keras/tf.layers compatible implementation. We will add the models later.
|
||||
|
||||
### Benchmark
|
||||
|
||||
We benchmarked the inference time of entire network on the 2D-3D-S dataset and with recent tests on V100, we were able to process ~18 Million Points.
|
||||
|
||||
<p align="center"> <img src=".github/inference_time.png" width="50%"> </p>
|
||||
|
||||
|
||||
### ShapeNet Segmentation
|
||||
|
||||
ShapeNet part segmentation results per category and mIoU (%) for different methods and inference speed (on a NVIDIA GeForce GTX 1080 Ti).
|
||||
|
||||
|| Airplane | Bag | Cap | Car | Chair | Earphones | Guitar | Knife | Lamp | Laptop | Motorbike | Mug | Pistol | Rocket | Skateboard | Table | mIoU | shapes/sec |
|
||||
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | --- | ---- | ---- | --- |
|
||||
|Kd-Network [4] | 80.1 | 74.6 | 74.3 | 70.3 | 88.6 | 73.5 | 90.2 | **87.2** | 81.0 | 94.9 | 57.4 | 86.7 | 78.1 | 51.8 | 69.9 | 80.3 | 77.4 | n.a.|
|
||||
|PointNet [1] | 83.4 | 78.7 | 82.5 | 74.9 | 89.6 | 73.0 | 91.5 | 85.9 | 80.8 | 95.3 | 65.2 | 93.0 | 81.2 | 57.9 | 72.8 | 80.6 | 80.4 | n.a. |
|
||||
|PointNet++ [2] | 82.4 | 79.0 | 87.7 | 77.3 | **90.8** | 71.8 | 91.0 | 85.9 | 83.7 | 95.3 | 71.6 | 94.1 | 81.3 | 58.7 | 76.4 | 82.6 | 81.9 | 2.7 |
|
||||
|SPLATNet3D [3] | 81.9 | 83.9 | 88.6 | **79.5** | 90.1 | 73.5 | 91.3 | 84.7 | **84.5** | 96.3 | 69.7 | 95.0 | 81.7 | 59.2 | 70.4 | 81.3 | 82.0 | 9.4 |
|
||||
|SGPN [5] | 80.4 | 78.6 | 78.8 | 71.5 | 88.6 | **78.0** | 90.9 | 83.0 | 78.8 | 95.8 | **77.8** | 93.8 | **87.4** | 60.1 | **92.3** | **89.4** | 82.8 | n.a. |
|
||||
Ours | **83.6** | **91.2** | **96.7** | **79.5** | 84.7 | 71.7 | **92.0** | 86.5 | 83.2 | **96.6** | 71.7 | **95.7** | 86.1 | **74.8** | 81.4 | 84.5 | **85.0** | **489.3** |
|
||||
|
||||
|
||||
|
||||
### 2D-3D-S dataset
|
||||
|
||||
Class specific average precision (AP) on the 2D-3D-S dataset.
|
||||
|
||||
| | Table | Chair | Sofa | Bookcase | Board | Ceiling | Floor | Wall | Beam | Col. | Wind. | Door | mAP |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
Armeni [6] | 46.02 | 16.15 | 6.78 | 54.71 | 3.91 | 71.61 | 88.70 | 72.86 | 66.67 | **91.77** | 25.92 | 54.11 | 49.93|
|
||||
Armeni [6] | 39.87 | 11.43 | 4.91 | **57.76** | 3.73 | 50.74 | 80.48 | 65.59 | 68.53 | 85.08 | 21.17 | 45.39 | 44.19|
|
||||
PointNet [1] | 46.67 | 33.80 | 4.76 | n.a. | 11.72 | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. |
|
||||
SGPN [5] | 46.90 | 40.77 | 6.38 | 47.61 | 11.05 | 79.44 | 66.29 | **88.77** | **77.98** | 60.71 | **66.62** | **56.75** | 54.35 |
|
||||
Ours | 66.03 | 51.75 | 15.59 | 39.03 | 43.50 | 87.20 | 96.00 | 65.53 | 54.76 | 52.74 | 55.34 | 35.81 | 55.27|
|
||||
Ours** | **67.02** | **52.75** | **16.61** | 39.26 | **47.68** | **87.33** | **96.10** | 65.52 | 56.83 | 55.10 | 57.66 | 36.76 | **56.55**|
|
||||
|
||||
|
||||
|
||||
|
||||
More Resources
|
||||
-------------------
|
||||
|
||||
- [Arxiv Pre-Print](https://arxiv.org/abs/1803.07289)
|
||||
- [Video](https://www.youtube.com/watch?v=5ftWmuQXU_s)
|
||||
- [Project-Page](https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/computergrafik/lehrstuhl/publications/publications-since-2012/flex-convolution-million-scale-point-cloud-learning-beyond-grid-worlds/)
|
||||
|
||||
|
||||
|
||||
Citation
|
||||
@@ -52,9 +95,19 @@ If you use the code in this repository, please cite our paper:
|
||||
Patrick Wieschollek and
|
||||
Hendrik P. A. Lensch
|
||||
},
|
||||
title = {Flex-Convolution (Million-Scale Pointcloud Learning Beyond Grid-Worlds)},
|
||||
title = {Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)},
|
||||
booktitle = {Asian Conference on Computer Vision (ACCV)},
|
||||
month = {Dezember},
|
||||
year = {2018}
|
||||
}
|
||||
```
|
||||
|
||||
References
|
||||
-------------------
|
||||
|
||||
[1] C. Qi, H. Su, K. Mo, L. Guibas, "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017.<br/>
|
||||
[2] C. Qi and L. Yi, H. Su, L. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space", Advances in Neural Information Processing Systems (NIPS) 2017.<br>
|
||||
[3] H. Su, V. Jampani, D.Sun, S. Maji, E. Kalogerakis, M.-H. Yang, J. Kautz, "SPLATNet: Sparse Lattice Networks for Point Cloud Processing", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018.<br>
|
||||
[4] R. Klokov, V. Lempitsky, "Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models", Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2017. <br>
|
||||
[5] W. Wang, R. Yu, Q. Huang, U. Neumann, "Sgpn: Similarity group proposal network for 3d point cloud instance segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018.<br>
|
||||
[6] I. Armeni, A. Sax, A.-R. Zamir, S. Savarese, "Joint 2D-3D-Semantic Data for Indoor Scene Understanding", ArXiv e-prints 2017.
|
||||
|
||||
Reference in New Issue
Block a user