Optimization Landscape and Feasibility in Riemannian AmbientFlow

Problem Statement

Riemannian AmbientFlow minimizes a combined objective:

L(θ, φ) = L_AF(θ, φ) + λ · ||J_{f_θ}(0)||_F²

Three feasibility assumptions for recoverability:

(F1) Data matching: Learned distribution equals ground truth.

(F2) Posterior matching: Variational posterior equals true posterior.

(F3) Geometric constraint: ||J_f(0)||_F² ≤ C* = Tr(G*(0)).

Circle in R²

S¹, C*=1.0

1→2

d→D

Sphere in R³

S² stereographic, C*=8.0

2→3

d→D

Helix in R³

Helix, C*≈1.025

1→3

d→D

Experimental Setup

200

Data Points

Random Starts

λ Values

Experiments

Multi-Start Landscape Exploration

15 random initializations per λ. High variance indicates multiple distinct local minima.

Manifold: Metric:

Feasibility Phase Diagram

Manifold:

Complete Feasibility Table

Manifold	λ	Data MM	Post. MM	\|\|J\|\|²	F1	F2	F3	Feas.
Circle	0.0	0.079	1.761	0.415	0.924	0.172	1.0	0.159
Circle	0.1	0.089	2.527	0.389	0.915	0.080	1.0	0.073
Circle	1.0	0.147	1.050	0.260	0.864	0.350	1.0	0.302
Sphere	0.0	0.148	3.478	1.453	0.862	0.031	1.0	0.027
Sphere	0.1	0.165	3.762	0.882	0.848	0.023	1.0	0.020
Sphere	2.0	0.230	5.318	0.302	0.795	0.005	1.0	0.004
Helix	0.0	0.082	1.438	0.420	0.921	0.237	1.0	0.219
Helix	0.1	0.081	2.562	0.399	0.922	0.077	1.0	0.071
Helix	1.0	0.146	1.452	0.268	0.864	0.234	1.0	0.202

All entries reported. F3=1.0 everywhere (Jacobian norms below C*). Aggregate score dominated by F2 (posterior mismatch). Best: Circle 0.302, Sphere 0.027, Helix 0.219.

Bootstrap 95% CIs (B=200)

Manifold:

Oracle Feasibility: Capacity vs. Optimization

Fitting f_θ directly to f* (bypassing ELBO) to separate model capacity from optimization landscape effects.

Manifold	Model	Params	Recon. MSE	\|\|J\|\|²	F1	F3
Circle	Simple	9	0.0002	1.034	0.998	0.967
Circle	MLP	1,186	0.016	1.014	0.957	0.986
Sphere	Simple	19	0.162	3.764	0.909	1.000
Sphere	MLP	1,251	0.138	3.208	0.917	1.000
Helix	Simple	13	0.0002	1.059	0.998	0.967
Helix	MLP	1,219	0.021	1.176	0.952	0.860

Circle/Helix: capacity sufficient. Simple model achieves F1>0.99, MSE~0.0002. Feasibility gap is an optimization landscape phenomenon.

Sphere: genuine capacity gap. Both models achieve only F1~0.91, MSE~0.14. Stereographic projection harder to approximate.

Directional Curvature Analysis

50 random unit directions per critical point. All positive curvature provides evidence for local minimum status.

Manifold:

Manifold	λ	Min Curv.	Max Curv.	Neg. Dirs
Circle	0.0	12.28	461.67	0/50
Circle	1.0	11.52	671.04	0/50
Sphere	0.0	8.18	76.64	0/50
Sphere	1.0	62.76	859.35	0/50
Helix	0.0	141.03	1870.47	0/50
Helix	1.0	326.87	3896.46	0/50

Zero negative directions across 600+ random probes. Strong statistical evidence for local minimum status.

Parameter Continuation

Tracking a single local minimum as λ increases from 0 to 2. Smooth deformation, no bifurcation.

Manifold: Show:

Path dependence: Feasibility decreases monotonically along continuation, but fresh multi-start finds better solutions. Basin at λ=0 is not the most feasible at larger λ.

Pullback Metric Analysis

Manifold:

Manifold	λ	Tr(G_θ)	Tr(G*)	Ratio
Circle	0.0	0.450	1.000	0.450
Circle	1.0	0.261	1.000	0.261
Sphere	0.0	1.010	8.000	0.126
Sphere	1.0	0.515	8.000	0.064
Helix	0.0	0.416	1.025	0.406
Helix	1.0	0.263	1.025	0.256

Systematic underestimation: Learned metric underestimates true geometry by 2-16x. Sphere ratio drops to 0.064.

Noise Sensitivity (σ ∈ [0.01, 0.5])

Manifold:

Non-trivial noise dependence: Very low noise yields near-zero feasibility. Peaks at moderate noise (σ=0.1-0.2). Sphere consistently low across all noise levels.

Sample Size Sensitivity (n ∈ [50, 500])

Manifold:

Robust to sample size: Feasibility stable across n=50 to 500. Findings not driven by finite-sample effects.

Key Findings

1. Local minima confirmed: Zero negative curvature in 600+ probes. Multiple distinct minima exist (variance up to 2.08).

2. Feasibility trade-off: Increasing λ improves F3 but degrades F1. Non-monotonic aggregate. Best: Circle 0.302, Sphere 0.027, Helix 0.219.

3. Oracle separates capacity from optimization: Simple model achieves F1>0.99 on circle/helix when fit directly. Feasibility gap is primarily an optimization landscape phenomenon.

4. Geometric underestimation: Pullback metric underestimates true geometry by 2-16x due to Jacobian penalty.

5. Path dependence: Continuation yields worse feasibility than fresh multi-start. Basin at λ=0 not optimal at larger λ.

6. Robust across settings: Results stable across σ∈[0.05,0.5] and n∈[50,500].

Reference

Based on: Diepeveen et al., "Riemannian AmbientFlow," arXiv:2601.18728, 2026.

Read the full paper (PDF)

Problem Statement

Circle in R2

Sphere in R3

Helix in R3

Experimental Setup

Multi-Start Landscape Exploration

Feasibility Phase Diagram

Complete Feasibility Table

Bootstrap 95% CIs (B=200)

Oracle Feasibility: Capacity vs. Optimization

Directional Curvature Analysis

Parameter Continuation

Pullback Metric Analysis

Noise Sensitivity (σ ∈ [0.01, 0.5])

Sample Size Sensitivity (n ∈ [50, 500])

Key Findings

Reference

Circle in R²

Sphere in R³

Helix in R³