Weakly-Supervised Component Suggestions for Assembly Modeling

Today, I’ve read a paper “ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling” by Sung, Minhyuk; Su, Hao; Kim, Vladimir; Chaudhuri, Siddhartha; Guibas, Leonidas.

What is assembly-based modeling?

It is one of the methods of modeling in which user selects and arrange components into the completed object. For example, a chair model is composed of components of back, seat, foot and handrail.

In incremental interactive assembly modeling, user chooses a complementary component one by one to add it into the partial assembly.
It is desirable for such modeling systems to suggest applicable parts as complementary components so that they require two key technical components, retrieval and placement.

The problem is that it takes too much time and effort to label components and create dataset for component suggestion.

This work proposes a method of weakly-supervised component suggestions for 3D modeling.

Novelty of the study

This system has 3 neural net, Retrieval Network, Embedding Network and Placement network.

Retrieval Network is trained to take partial assembly X and output a probability distribution as a weighted gaussian distribution over the embedded space for each complementary component.
Embedding Network is trained to take part geometry Y and maps it to low-dimensional vector.
Retrieval Network and Embedding Network are trained jointly.
Placement network is trained to take X and Y and predict 3D coordinates, strictly translation of each complementary component.

Each architecture contains PointNet which takes 3D unordered points as input and classify into k class as output.

φ is weight, μ is mean and σ is standard deviation of Gaussian in the figure below.

Future work

Because this method randomly sample points over the surface of the input shape, larger components sometimes have bigger influences than small essential components.