- Fine details or high-frequency content tends to come from the surface texture itself and not necessarily from a complex scene geometry. Authors found that simply storing the first coefficient
k0, or “base color,” explicitly helps ease the network’s burden of compressing and reproducing detail and leads to sharper results, also in fewer iterations.-
k0 is optimized explicitly as a learnable parameter with a total variation regularizer.- computing and storing all N + 1 coefficients
(k0, k1, ... kN) for all pixels for all D depth planes can be expensive for both training and rendering. So authors use a coefficient sharing scheme where every M consecutive planes will share the same coefficients, but not the alphas- Total reconstruction loss = Photometric loss + difference of the image gradients.
- For a scene with 17 input photos of resolution 1008x756, the training takes ~ 18 hours using 1xNVIDIA V100 with a batch size of 1.
- To render one pixel, NEX uses 0.16 MFLOPs, whereas NeRF uses 226 MFLOPs.
-
f() is an MLP with 6 FC layers, each with 384 hidden nodes.-
g() is an MLP with 3 FC layers, each with 64 hidden nodes.- Positional encodings (sin and cos with different frequencies) of the inputs x,y,z, and v are used instead of the raw values. Such a mapping into a higher dimensional space enables the MLPs to more easily approximate a high-frequency function of variation in color and geometry of the scene.