Skip to content

Exploring the Berkeley Deep Drive Autonomous Vehicle Dataset

Welcome to the latest installment of our ongoing blog series where we highlight a dataset from the FiftyOne Dataset Zoo! FiftyOne provides a Dataset Zoo that contains a collection of common datasets that you can download and load into FiftyOne via a few simple commands. In this post, we explore the Berkeley Deep Drive dataset.

Wait, what’s FiftyOne?

FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.

FiftyOne quick overview gif

The FiftyOne Dataset Zoo comprises more than 30 datasets, with new datasets being added all the time! They cover a variety of computer vision use cases including:

  • Video
  • Location
  • Point-cloud
  • Action-recognition
  • Classification
  • Detection
  • Segmentation
  • Relationships

About the Berkeley Deep Drive dataset

The Berkeley Deep Drive (BDD) dataset is one of the largest and most diverse video datasets for autonomous vehicles and heterogeneous multitask learning.

The BDD100K dataset contains 100,000 video clips collected from more than 50,000 rides covering New York, San Francisco Bay Area, and other regions. The dataset contains diverse scene types such as city streets, residential areas, and highways. Furthermore, the videos were recorded in diverse weather conditions at different times of the day.

The videos are split into training (70K), validation (10K), and test (20K) sets. Each video is 40 seconds long at 720p resolution and a frame rate of 30fps. The frame at the 10th second of each video is annotated for image classification, detection, and segmentation tasks.

The version of the dataset we’ll be working with contains the 100K images extracted from the videos as described above, together with the image classification, detection, and segmentation labels.

Dataset quick facts

Step 1: Download the dataset

In order to load the BDD100K dataset into FiftyOne, you must download the source data manually with your directory organized in the following manner:

Step 2: Install FiftyOne

If you don’t already have FiftyOne installed on your laptop, it takes just a few minutes! For example on MacOs:

  • Verify your version of Python
  • Create and activate a virtual environment
  • Install IPython (optionall)
  • Upgrade your Setuptools
  • Install FiftyOne

Learn more about how to get up and running with FiftyOne in the Docs.

Step 3: Import the dataset

Now that you have the dataset downloaded and FiftyOne installed, let’s import the dataset into FiftyOne and launch the FiftyOne App. This should take just a few minutes and a few more lines of code. Make sure to swap in the correct path to your downloaded source files.

import fiftyone as fo
import fiftyone.zoo as foz

# The path to the source files that you manually downloaded
source_dir = "/path/to/dir-with-bdd100k-files"

dataset = foz.load_zoo_dataset(

session = fo.launch_app(dataset)

Your output should look similar to:

Preparing split 'validation' in '/path/to/dir-with-bdd100k-files/fiftyone/bdd100k/validation'
Preparing training images...
Preparing training labels...
Preparing validation images...
Preparing validation labels...
Preparing test images...
Parsing dataset metadata
Found 10000 samples
Dataset info written to '/path/to/dir-with-bdd100k-files/fiftyone/bdd100k/info.json'
Loading 'bdd100k' split 'validation'
 100% |███| 10000/10000 [3.2m elapsed, 0s remaining, 62.9 samples/s]      
Dataset 'bdd100k-validation' created
App launched. Point your web browser to http://localhost:5151

The last line in the code snippet we submitted will launch the FiftyOne App in your default browser. You should see the following initial view of the dataset in the FiftyOne App:

Ok, let’s start exploring the Berkeley Deep Drive dataset!

Object detection

For object detection, there are 10 classes to be evaluated. They include:

1: person
2: rider
3: car
4: truck
5: bus
6: train
7: motor (motorcycle)
8: bike (bicycle)
9: traffic light
10: traffic sign

Note: The field category_idrange starts at 1 instead of 0.

Their distribution within the dataset can be seen in the FiftyOne App under detections > label.

We can limit the detections view to specific labels by selecting detections > label > truck for example:

Frame attributes

The BDD100K dataset has frame attributes including weatherscene, and timeofday.


As the name implies, these labels help us understand what the general weather conditions are in the sample.

weather: rainy, snowy, clear, overcast, undefined, partly cloudy, foggy

For example weather > label > snowy:


These labels help us understand the general vehicle-centric environment the objects are moving though.

scene: tunnel, residential, parking lot, undefined, city street, gas stations, highway

For example scene > label > tunnel:

Time of day

These labels help us understand at what time of day (which tells us something about the lighting conditions) the sample was taken.

timeofday: daytime, night, dawn/dusk, undefined

For example timeofday > label > dawn/dusk:

Drivable area

As you can imagine, with this dataset roads and pathways are going to be key elements of just about any model you develop. Linearly drawn polylines help us trace road and pathway structures connected at individual vertices.

For example polylines > label > drivable area:

Zooming in on a specific sample:


Dashed and solid roadway lanes that influence the behavior of vehicles, pedestrians, and bicycles can be viewed at polylines > label > lane:

Zooming in on a specific sample:

Final example

Let’s say we want to limit our view to just samples where cars are present on city streets during the daytime while it is raining. Simply make the selections in the sidebar filter to view the 241 samples that meet this criteria.

Start working with the dataset

Now that you have a general idea of what the dataset contains, you can start using FiftyOne to perform a variety tasks including:

Create dataset views

Think of dataset views as pipeline operations that are applied to a dataset to extract a subset of the dataset whose samples and fields are filtered, sorted, shuffled, etc. Learn more about how to create dataset views in the Docs.

Create aggregations

If you are interested in computing aggregate statistics about datasets, such as label counts, distributions, and ranges, check out the aggregations section of the Docs.

Create interactive plots

FiftyOne provides a powerful plotting framework that contains a variety of interactive plotting methods that enable you to visualize your datasets and uncover patterns that are not apparent from inspecting either the raw media files or aggregate statistics. Learn more about the plotting capabilities in FiftyOne in the Docs.

plotting patterns in datasets, visualizing embeddings in fiftyone

Annotate datasets

FiftyOne provides a powerful annotation API that makes it easy to add or edit labels on your datasets or specific views into them. Learn more about annotating datasets in the Docs.

Evaluate models

FiftyOne provides a variety of built-in methods for evaluating your model predictions, including regressions, classifications, detections, polygons, instance, and semantic segmentations, on both image and video datasets. Learn more about evaluating models in the Docs.

FiftyOne Brain

Finally, the FiftyOne Brain provides powerful machine learning techniques you can apply to your workflows including: