Plausible vegetation synthesis from a single photo

Today I’ve read the paper “Single-picture reconstruction and rendering of trees for plausible vegetation synthesis” by Argudo, Oscar; Chica, Antonio; Andujar, Carlos.

Vegetation synthesis in DTM

Vegetation is often seen in Digital Terrain Model(DTM), but it takes too much effort for artist to model. There are some techniques to improve the efficiency in vegetation synthesis, L-systems, the space colonization algorithm and so on. However they require point cloud, multiple photographs or various parameter setting.

Novelty of the study

This work proposes a pipeline to reconstruct and render plausible trees from a single photograph.

Method of crown reconstruction from the photo.

1. Envelope mesh synthesis
2. Color synthesis
3. Relief synthesis
4. Radial distance map
5. Branches and leaves generation

1, Envelope mesh synthesis

Get tree silhouette pixels S using 8-connectivity from the photo.

Fix the silhouette points on the z = 0 plane and maintain their tangents of the point (i, j, 0) to be f(i, j). Then get crown mesh C as height map. To realize this constraint, define the outer silhouette S_o as the set of pixels outside C but adjacent to a silhouette pixel.

Crown C is determined by minimizing the thin-plate energy defined by a biharmonic equation inside the crown C. In actual calculation, each of unknown(x, y, z) are figured out using equation below. The matrix M is bilaplacian filter.

2, Color synthesis

Create color map from the photo by using the texture synthesis algorithm of Image texture tools(P. Harrison, 2005). Then create cube map from the　synthesized texture.

3, Relief synthesis

The next step is estimation of relief (depth) from luminance of the photo. At first, the luminance is normalized in the [0,1] range. The system computes a blur pyramid of the luminance. Lower levels (low-frequencies) will have larger weights, while higher levels are used to add small fine details to the relief.

Cube map of the relief is generated from the cube map of color in this way.

4, Radial distance map

Create radial distance map(RDM) that stores distance from the center of the crown to the surface, and set values by projection of the cube map of relief.

5, Branches and leaves generation

Branches are created by the Space Colonization Algorithm in the crown. And leaves are represented as set of billboards for simplicity.

Rendering

There are 2 ways of rendering the tree.

1. Fragment-based relief mapping which is for distant trees
2. Billboard cloud with an RGBA texture drawing leaves and branches which is for close trees

Check the output trees generated by this method in the original paper.