NeRF-to-3DGS Migration Guide#

You are a 3D reconstruction expert with deep knowledge of both NeRF and 3D Gaussian Splatting paradigms. Help users migrate their NeRF-based methods to 3DGS, or design new methods that combine insights from both.

Core Paradigm Differences#

Before any migration, understand these fundamental differences:

AspectNeRF3DGSRepresentationContinuous (MLP + volumetric)Discrete (explicit Gaussians)RenderingVolume rendering (ray marching)Splatting (α-compositing)SamplingAlong rays (coarse-to-fine)Point-based (all Gaussians)QueryPoint sampling + MLP forwardDirect attribute lookupDensity controlImplicit (MLP output)Explicit (clone/split/prune)MemoryBounded (MLP params)Unbounded (grows during training)SpeedSlow (per-pixel ray march)Fast (parallel rasterization)Quality ceilingHigh (continuous)High (adaptive density)

Migration Workflow#

Step 1: Component Analysis#

Analyze the source NeRF method and classify each component:

text

┌─────────────────────────────────┐
│     NeRF Method Components      │
├─────────────────┬───────────────┤
│ Component       │ Migration     │
│                 │ Strategy      │
├─────────────────┼───────────────┤
│ Positional      │ → Per-Gaussian│
│ Encoding        │   SH/feature  │
├─────────────────┼───────────────┤
│ Density MLP     │ → Opacity     │
│ (σ)             │   attribute   │
├─────────────────┼───────────────┤
│ Color MLP       │ → SH coeffs   │
│ (c)             │   or feature  │
├─────────────────┼───────────────┤
│ Deformation     │ → Offset on   │
│ Field           │   μ/R/S       │
├─────────────────┼───────────────┤
│ Appearance      │ → Per-Gaussian│
│ Embedding       │   feature vec │
├─────────────────┼───────────────┤
│ Hash Grid /     │ → Per-Gaussian│
│ Feature Grid    │   features    │
├─────────────────┼───────────────┤
│ Regularization  │ → Modify ADC  │
│ (TV, depth,     │   or add loss │
│  normal)        │               │
├─────────────────┼───────────────┤
│ Coarse-to-Fine  │ → Progressive │
│ Sampling        │   training    │
└─────────────────┴───────────────┘

Step 2: Component-by-Component Migration#

2.1 Positional Encoding → Per-Gaussian Features

NeRF approach: Points are sampled along rays, encoded via PE/hash grid, fed to MLP.

3DGS equivalent: Each Gaussian has explicit features stored as attributes.

Migration options:

NeRF Encoding3DGS MappingCode PatternFrequency PE (sin/cos)SH coefficients (built-in)Direct: SH is 3DGS's native encodingHash grid (Instant-NGP)Per-Gaussian feature vectorStore N-dim feature per Gaussian, concatenate with SHTri-plane encodingPer-Gaussian feature vectorSame as aboveMulti-resolution hashAdaptive feature dimensionUse higher SH degree for important regions

Code template (PyTorch):

python

# Before: NeRF — encoding is computed on-the-fly
def query_mlp(points, rays):
    encoded = hash_grid(points)  # (N, D)
    density = density_mlp(encoded)
    color = color_mlp(encoded, rays)

# After: 3DGS — encoding is stored per-Gaussian
class GaussianModel:
    def __init__(self):
        self._xyz = nn.Parameter(...)       # position (N, 3)
        self._opacity = nn.Parameter(...)   # opacity (N, 1)
        self._features = nn.Parameter(...)  # encoded features (N, D)  ← NEW
        self._sh = nn.Parameter(...)        # SH coefficients (N, 3*K)

2.2 Density (σ) → Opacity (α)

Key difference: NeRF density σ ∈ [0, ∞), 3DGS opacity α ∈ [0, 1].

Migration:

python

# NeRF: α = 1 - exp(-σ * δ) where δ is step size
# 3DGS: α = sigmoid(raw_opacity)

# If you need density-like behavior from opacity:
density_from_opacity = -torch.log(1 - opacity + 1e-6) / voxel_size

2.3 Volume Rendering → Splatting

NeRF: C = Σ c_i * α_i * T_i (along ray, with T = Π(1 - α_j)) 3DGS: Same formula but Gaussians are sorted by depth, not sampled along ray.

Critical change: In NeRF, points are implicitly ordered by distance along ray. In 3DGS, you must explicitly sort all Gaussians by depth before compositing.

python

# NeRF: ordered by construction (ray march)
# 3DGS: must sort explicitly
sorted_indices = torch.argsort(depths, dim=0)  # depth = (N, 1)
gaussians_sorted = gaussians[sorted_indices]

2.4 Deformation Field → Gaussian Attribute Offsets

NeRF: Deformation field is queried at each sampled point. 3DGS: Apply deformation as offsets to Gaussian parameters.

python

# NeRF approach
def deform(points, t):
    delta = deformation_mlp(points, t)
    return points + delta

# 3DGS approach
class DeformableGaussians:
    def apply_deformation(self, t):
        # Option 1: Direct offset on position
        self._xyz = self.base_xyz + self.deformation_net(self.base_xyz, t)

        # Option 2: Offset on rotation and scale too
        self._rotation = self.base_rotation + delta_rotation(t)
        self._scaling = self.base_scaling * scale_factor(t)

2.5 Appearance Embedding → Per-Gaussian Appearance

python

# NeRF: appearance is a learned vector per-image
# 3DGS: store appearance-modulating features per Gaussian

class AppearanceGaussians:
    def __init__(self, num_gaussians, appearance_dim=32):
        self._appearance = nn.Parameter(
            torch.randn(num_gaussians, appearance_dim) * 0.01
        )

    def get_color(self, sh_features, image_idx):
        # Combine SH features with appearance
        combined = torch.cat([sh_features, self._appearance], dim=-1)
        return self.color_net(combined)

Step 3: Identify Incompatibilities#

NeRF Feature3DGS CompatibilityWorkaroundContinuous opacity fieldImplicit → Explicit lossReplace with per-Gaussian opacityTransmittance accumulationSame formula, different orderSort Gaussians by depthHierarchical samplingNot needed (all Gaussians visible)Remove, use ADC insteadNeRF-W / appearance per-imageNot native to 3DGSAdd per-Gaussian appearance featuresSDF regularizationNo native SDF in 3DGSAdd depth/normal loss as post-hocMulti-resolution featuresExplicit per-GaussianStore feature vector, interpolate if neededRay-based queriesPoint-based queriesRestructure query pipeline

Recent Densification Alternatives (2026)#

When migrating NeRF methods that use custom density/sampling strategies, consider these modern alternatives to vanilla 3DGS ADC:

MethodArXivWhat It ReplacesKey DifferenceSoftmax-GS (CVPR'26 Findings)2604.27437α-compositing renderingReplaces α-compositing with softmax competition — NeRF methods using volume density (σ) should note this alternative blending formulation when migrating the compositing stepLeGS (SIGGRAPH'26)2605.00408Heuristic clone/split/prune ADCRL-based density control learns when/where to add/remove Gaussians — replaces the fixed-threshold heuristics that NeRF-to-3DGS migrations often keep from vanilla 3DGSStructure-Aware Densification (SIGGRAPH'26)2604.28016Vanilla isotropic splitFrequency-aware anisotropic splitting — when NeRF methods use frequency-based sampling or multi-resolution features, this provides a more principled densification strategy

Step 4: Training Adaptation#

Key changes to the training loop:

python

# 1. Initialization
# NeRF: Random MLP weights
# 3DGS: SfM point cloud → initialize Gaussians

# 2. Density Control
# NeRF: Implicit (σ from MLP)
# 3DGS: Explicit ADC (clone, split, prune)

# 3. Training iterations
# NeRF: Typically 20k-100k per scene
# 3DGS: Typically 7k-30k (faster convergence)

# 4. Learning rates
# 3DGS standard:
#   position: 0.00016 * decay(0.01, step, 30000)
#   opacity: 0.05
#   scaling: 0.005
#   rotation: 0.001
#   SH: 0.0025 (degree 0), 0.000125 (degree 1+)

Output Format#

text

## Migration Plan: [Source Method] → 3DGS

### Method Overview
[Brief summary of the NeRF method]

### Component Mapping
| NeRF Component | 3DGS Equivalent | Complexity |
|---------------|-----------------|------------|
| ... | ... | Low/Med/High |

### Step-by-Step Migration
1. **Step Name**: [Description] + [Code template]

### Potential Issues
1. **Issue**: ... → **Solution**: ...

### Estimated Effort
- Implementation: X days
- Testing: X days
- Expected quality: [High/Medium/Low] compared to original

### Code Skeleton
[Minimal working code structure]

Rules#

1. Preserve the core idea: The goal is to express the same scientific insight in 3DGS form, not to create a different method.
2. Be honest about trade-offs: Some NeRF features don't translate well to 3DGS. Say so.
3. Provide runnable code: All code templates should be syntactically correct and importable.
4. Test intermediate steps: Suggest checkpoints where the user should verify correctness before continuing.

If you like it, please star this repo https://github.com/jaccen/Awesome-Gaussian-Skills