Welcome to the latest installment of our ongoing blog series where we highlight a dataset from the FiftyOne Dataset Zoo! FiftyOne provides a Dataset Zoo that contains a collection of common datasets that you can download and load into FiftyOne via a few simple commands. In this post, we explore the Berkeley Deep Drive dataset.
Wait, what’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
The FiftyOne Dataset Zoo comprises more than 30 datasets, with new datasets being added all the time! They cover a variety of computer vision use cases including:
- Video
- Location
- Point-cloud
- Action-recognition
- Classification
- Detection
- Segmentation
- Relationships
About the Berkeley Deep Drive dataset
The Berkeley Deep Drive (BDD) dataset is one of the largest and most diverse video datasets for autonomous vehicles and heterogeneous multitask learning.
The BDD100K dataset contains 100,000 video clips collected from more than 50,000 rides covering New York, San Francisco Bay Area, and other regions. The dataset contains diverse scene types such as city streets, residential areas, and highways. Furthermore, the videos were recorded in diverse weather conditions at different times of the day.
The videos are split into training (70K), validation (10K), and test (20K) sets. Each video is 40 seconds long at 720p resolution and a frame rate of 30fps. The frame at the 10th second of each video is annotated for image classification, detection, and segmentation tasks.
The version of the dataset we’ll be working with contains the 100K images extracted from the videos as described above, together with the image classification, detection, and segmentation labels.
Dataset quick facts
- Research Paper: BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
- Authors: Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, Trevor Darrell
- GitHub: README and Code
- Docs: BDD100K Documentation
- Dataset Source: BDD100K data and annotations
- Dataset Size: 7.10 GB
- License: BSD 3-Clause License
- Last Release: September 21, 2020
- FiftyOne Dataset Name: bdd100k
- Tags: image, multilabel, automotive, manual
- Supported Splits: train, validation, test
- Zoo Dataset class: BDD100KDataset
Step 1: Download the dataset
In order to load the BDD100K dataset into FiftyOne, you must download the source data manually with your directory organized in the following manner:
Step 2: Install FiftyOne
If you don’t already have FiftyOne installed on your laptop, it takes just a few minutes! For example on MacOs:
- Verify your version of Python
- Create and activate a virtual environment
- Install IPython (optionall)
- Upgrade your Setuptools
- Install FiftyOne
Learn more about how to get up and running with FiftyOne in the Docs.
Step 3: Import the dataset
Now that you have the dataset downloaded and FiftyOne installed, let’s import the dataset into FiftyOne and launch the FiftyOne App. This should take just a few minutes and a few more lines of code. Make sure to swap in the correct path to your downloaded source files.
import fiftyone as fo
import fiftyone.zoo as foz
# The path to the source files that you manually downloaded
source_dir = "/path/to/dir-with-bdd100k-files"
dataset = foz.load_zoo_dataset(
"bdd100k",
split="validation",
source_dir=source_dir,
)
session = fo.launch_app(dataset)
Your output should look similar to:
Preparing split 'validation' in '/path/to/dir-with-bdd100k-files/fiftyone/bdd100k/validation' Preparing training images... Preparing training labels... Preparing validation images... Preparing validation labels... Preparing test images... Parsing dataset metadata Found 10000 samples Dataset info written to '/path/to/dir-with-bdd100k-files/fiftyone/bdd100k/info.json' Loading 'bdd100k' split 'validation' 100% |███| 10000/10000 [3.2m elapsed, 0s remaining, 62.9 samples/s] Dataset 'bdd100k-validation' created App launched. Point your web browser to http://localhost:5151
The last line in the code snippet we submitted will launch the FiftyOne App in your default browser. You should see the following initial view of the dataset in the FiftyOne App:
Ok, let’s start exploring the Berkeley Deep Drive dataset!
Object detection
For object detection, there are 10 classes to be evaluated. They include:
1: person 2: rider 3: car 4: truck 5: bus 6: train 7: motor (motorcycle) 8: bike (bicycle) 9: traffic light 10: traffic sign
Note: The field category_idrange starts at 1 instead of 0.
Their distribution within the dataset can be seen in the FiftyOne App under detections > label.
We can limit the detections view to specific labels by selecting detections > label > truck for example:
Frame attributes
The BDD100K dataset has frame attributes including weather, scene, and timeofday.
Weather
As the name implies, these labels help us understand what the general weather conditions are in the sample.
weather: rainy, snowy, clear, overcast, undefined, partly cloudy, foggy
For example weather > label > snowy:
Scene
These labels help us understand the general vehicle-centric environment the objects are moving though.
scene: tunnel, residential, parking lot, undefined, city street, gas stations, highway
For example scene > label > tunnel:
Time of day
These labels help us understand at what time of day (which tells us something about the lighting conditions) the sample was taken.
timeofday: daytime, night, dawn/dusk, undefined
For example timeofday > label > dawn/dusk:
Drivable area
As you can imagine, with this dataset roads and pathways are going to be key elements of just about any model you develop. Linearly drawn polylines help us trace road and pathway structures connected at individual vertices.
For example polylines > label > drivable area:
Zooming in on a specific sample:
Lanes
Dashed and solid roadway lanes that influence the behavior of vehicles, pedestrians, and bicycles can be viewed at polylines > label > lane:
Zooming in on a specific sample:
Final example
Let’s say we want to limit our view to just samples where cars are present on city streets during the daytime while it is raining. Simply make the selections in the sidebar filter to view the 241 samples that meet this criteria.
Start working with the dataset
Now that you have a general idea of what the dataset contains, you can start using FiftyOne to perform a variety tasks including:
Create dataset views
Think of dataset views as pipeline operations that are applied to a dataset to extract a subset of the dataset whose samples and fields are filtered, sorted, shuffled, etc. Learn more about how to create dataset views in the Docs.
Create aggregations
If you are interested in computing aggregate statistics about datasets, such as label counts, distributions, and ranges, check out the aggregations section of the Docs.
Create interactive plots
FiftyOne provides a powerful plotting framework that contains a variety of interactive plotting methods that enable you to visualize your datasets and uncover patterns that are not apparent from inspecting either the raw media files or aggregate statistics. Learn more about the plotting capabilities in FiftyOne in the Docs.
Annotate datasets
FiftyOne provides a powerful annotation API that makes it easy to add or edit labels on your datasets or specific views into them. Learn more about annotating datasets in the Docs.
Evaluate models
FiftyOne provides a variety of built-in methods for evaluating your model predictions, including regressions, classifications, detections, polygons, instance, and semantic segmentations, on both image and video datasets. Learn more about evaluating models in the Docs.
FiftyOne Brain
Finally, the FiftyOne Brain provides powerful machine learning techniques you can apply to your workflows including:
