Gaussian Splatting: From Research to Reality

February 7, 2025 – Written by Paula Ramos

Computer Vision Plugins

 

My Journey in 3D Reconstruction

Back in 2015, during my doctoral thesis, I was deep into 3D reconstruction, working with point clouds to estimate mass using the PCL (Point Cloud Library) and pure C++ code. It was challenging; reconstructing coffee branches was painstakingly slow, requiring significant computational resources and manual optimization. The process took forever; every incremental improvement meant diving deep into low-level code. The field of 3D vision was still in its early days, and tools were not as optimized as they are today.

 

Coffee Branch — 3D Reconstruction

 

Fast forward to today, and we are witnessing a revolution in 3D Gaussian Splatting (Kerbl et al., 2023), a technique that enables real-time radiance field rendering, making 3D reconstructions smoother, more efficient, and visually compelling. One of my colleagues at Voxel51, Daniel Gural, has been working tirelessly on a FiftyOne implementation that is technically impressive and immensely beneficial to the computer vision community. His work simplifies the visualization and comparison of point clouds, making it easier than ever to explore 3D reconstructions with minimal effort. (Check out the plugin here). When I first learned about the new Gaussian Splatting capabilities in FiftyOne, I was immediately eager to integrate them into my workflow, as it finally offers the streamlined experience I had hoped for in 3D reconstruction.

 

The Challenge: Implementing Gaussian Splatting

Let’s be real and honest, the implementation process for the reconstruction wasn’t easy. Getting Gaussian Splatting to work required following the original paper, setting up the pipeline, drivers, dependencies, and handling 3D data efficiently. There were multiple iterations and debugging sessions to get the best visual results. However, the potential benefits were too significant to ignore. Unlike traditional point cloud methods, Gaussian Splats allow for smoother, more dynamic reconstructions without heavy post-processing.

Here, I am sharing my whole setup process.

# Installing and Running 3D Gaussian Splatting on Linux
# Pipeline running by Paula Ramos https://www.linkedin.com/in/paula-ramos-phd/

# 1. System Requirements
# Ensure you have:
# - Ubuntu 22.04+
# - NVIDIA GPU (RTX 3000/4000 series recommended)
# - NVIDIA Driver 535+
# - CUDA 12.2 installed
# - Python 3.10+

# 2. Install System Dependencies
sudo apt update && sudo apt upgrade -y
sudo apt install -y git cmake libxmu-dev libxi-dev libgl-dev libomp-dev \
                    python3-dev python3-venv python3-pip \
                    ninja build-essential

# Clone the Repository & Set Up Virtual Environment
git clone https://github.com/graphdeco-inria/gaussian-splatting.git
cd gaussian-splatting

# Create and activate a virtual environment
python3 -m venv gs_env
source gs_env/bin/activate

# Upgrade pip
pip install --upgrade pip

# Install PyTorch with CUDA 12.1+
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Verify installation
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"

# Install Gaussian Splatting Dependencies 
# https://github.com/graphdeco-inria/gaussian-splatting/blob/main/environment.yml
# Carefull with the versions of Cuda and Torch
pip install -r requirements.txt

# Initialize Git Submodules
git submodule update --init --recursive

# Install required submodules
pip install ./submodules/diff-gaussian-rasterization
pip install ./submodules/simple-knn
pip install ./submodules/fused-ssim

# Fix Common Build Errors
# Missing Python Development Headers (`Python.h` not found)
sudo apt install -y python3.10-dev python3-dev

# CUDA Compilation Issues (`TORCH_CUDA_ARCH_LIST` Not Set)
export TORCH_CUDA_ARCH_LIST="8.6 8.9+PTX"
echo 'export TORCH_CUDA_ARCH_LIST="8.6 8.9+PTX"' >> ~/.bashrc
source ~/.bashrc

# Ensure CUDA 12.2 is Used
export PATH=/usr/local/cuda-12.2/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64:$LD_LIBRARY_PATH
echo 'export PATH=/usr/local/cuda-12.2/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

#!wget https://huggingface.co/camenduru/gaussian-splatting/resolve/main/tandt_db.zip

#!unzip tandt_db.zip

#!python train.py -s /content/gaussian-splatting/tandt/train

# Run Gaussian Splatting
python train.py -s <path to COLMAP or NeRF dataset>

# Verify Everything
nvidia-smi
nvcc --version
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available())"

 

The Breakthrough: Using FiftyOne as a Visualizer

One of the biggest bottlenecks in 3D research is visualization. While traditional 3D viewers like Meshlab or Open3D work well, they lack the ease of dataset management and structured analysis. This is where FiftyOne comes in.

 

With FiftyOne’s plugin for Gaussian Splatting, visualizing multiple point clouds becomes insanely easy , just one click, and you can seamlessly switch between different reconstructions. The ability to compare different iterations of the same 3D scene (e.g., 7000 vs. 30000 iterations) within the same tool is a game-changer for debugging and analysis. It’s something I wish I had back in 2010.

 

 

Here’s how simple it is to visualize Gaussian Splats in FiftyOne: Install FiftyOne and the changes in this branch https://github.com/voxel51/fiftyone/tree/ksplat_panel

!fiftyone plugins download https://github.com/danielgural/ksplats_panel

import fiftyone as fo
from fiftyone.utils.splats import SplatFile

dataset = fo.Dataset(name=”splat-test”, overwrite=True)

sample = fo.Sample(filepath="/path/to/reference_image.png")
sample["splat"] = SplatFile(filepath="/path/to/reconstruction.ply")
dataset.add_sample(sample)
session = fo.launch_app(dataset, auto=False, port=5152)

That’s it. From spending weeks debugging point clouds in C++ to a few lines of Python code. This is what progress in AI and computer vision looks like!

Try this by your own, I have created a dataset you can download and play with FiftyOne. https://huggingface.co/datasets/Voxel51/gaussian_splatting

 

Why This Matters for the AI & Computer Vision Community

The impact of Gaussian Splatting goes beyond personal convenience, representing a paradigm shift in 3D data representation. Combined with FiftyOne’s powerful dataset visualization capabilities, researchers and developers can now:

  • Quickly inspect and compare multiple 3D reconstructions.
  • Debug and refine 3D models with intuitive dataset querying.
  • Reduce engineering overhead by leveraging FiftyOne’s integration with AI pipelines.

 

Just Wrapping Up! 😀

Looking back, I realize the dream of real-time 3D visualization is finally here. Gaussian Splatting is a massive leap forward, and with FiftyOne as a visualization tool, we can finally make 3D dataset exploration as simple as working with images.

If you’re working on 3D reconstruction, I highly recommend giving this a try. You might feel like you’ve been waiting for this forever, just like I did. I would love to hear your thoughts. Let’s push the boundaries of 3D AI together!

I would love to hear about your experiences! Please Share Your Thoughts, Ask Questions, and Provide Testimonials. Your insights might help others in our next posts.

Together, we can learn more effectively and contribute to improving FiftyOne!

Stay tuned for the next post, in which we’ll explore FiftyOne’s essential features, manipulate the dataset, and evaluate the model.

Let’s make this journey with FiftyOne a collaborative and enriching experience. Happy coding!

 

Stay Connected

 

What Is Next?

I’m excited to share more about my journey at Voxel51! 🚀 If you’re interested in following along as I dive deeper into the world of AI and continue to grow professionally, feel free to connect or follow me on LinkedIn. Let’s inspire each other to embrace change and reach new heights!

You can find me at some Voxel51 events (https://voxel51.com/computer-vision-events/), or if you want to join this fantastic team, it’s worth taking a look at this page: https://voxel51.com/jobs/