fiftyone.utils.video¶

Video utilities.

Copyright 2017-2022, Voxel51, Inc.
voxel51.com

Functions:

`concat_videos`(input_paths, output_path[, …])	Concatenates the given list of videos, in order, into a single video.
`extract_clip`(video_path, output_path[, …])	Extracts the specified clip from the video.
`reencode_video`(input_path, output_path[, …])	Re-encodes the video using the H.264 codec.
`reencode_videos`(sample_collection[, …])	Re-encodes the videos in the sample collection as H.264 MP4s that can be visualized in the FiftyOne App.
`sample_frames_uniform`(frame_rate[, …])	Returns a list of frame numbers sampled uniformly according to the provided parameters.
`sample_video`(input_path, output_patt[, …])	Samples the video into a directory of per-frame images according to the provided parameters using `ffmpeg`.
`sample_videos`(sample_collection[, …])	Samples the videos in the sample collection into directories of per-frame images according to the provided parameters using `ffmpeg`.
`transform_video`(input_path, output_path[, …])	Transforms the video according to the provided parameters using `ffmpeg`.
`transform_videos`(sample_collection[, fps, …])	Transforms the videos in the sample collection according to the provided parameters using `ffmpeg`.

fiftyone.utils.video.extract_clip(video_path, output_path, support=None, timestamps=None, metadata=None, fast=False)¶

Extracts the specified clip from the video.

Provide either suppport or timestamps to this method.

When fast=False, the following ffmpeg command is used:

# Slower, more accurate option
ffmpeg -ss <start_time> -i <video_path> -t <duration> <output_path>

When fast is True, the following two-step ffmpeg process is used:

# Faster, less accurate option
ffmpeg -ss <start_time> -i <video_path> -t <duration> -c copy <tmp_path>
ffmpeg -i <tmp_path> <output_path>

Parameters

video_path – the path to the video
output_path – the path to write the extracted clip
support (None) – the [first, last] frame number range to clip
timestamps (None) – the [start, stop] timestamps to clip, in seconds
metadata (None) – the fiftyone.core.metadata.VideoMetadata for the video
fast (False) – whether to use a faster-but-potentially-less-accurate strategy to extract the clip

fiftyone.utils.video.reencode_videos(sample_collection, force_reencode=True, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Re-encodes the videos in the sample collection as H.264 MP4s that can be visualized in the FiftyOne App.

The filepath of the samples are updated to point to the re-encoded videos.

By default, the re-encoding is performed via the following ffmpeg command:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

You can configure parameters of the re-encoding such as codec and compression by passing keyword arguments for eta.core.video.FFmpeg(**kwargs) to this function.

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
force_reencode (True) – whether to re-encode videos that are already MP4s
delete_originals (False) – whether to delete the original videos after re-encoding
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be re-encoded
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.transform_videos(sample_collection, fps=None, min_fps=None, max_fps=None, size=None, min_size=None, max_size=None, reencode=False, force_reencode=False, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Transforms the videos in the sample collection according to the provided parameters using ffmpeg.

The filepath of the samples are updated to point to the transformed videos.

In addition to the size and frame rate parameters, if reencode == True, the following basic ffmpeg command structure is used to re-encode the videos as H.264 MP4s:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

Note

This method will not update the metadata field of the collection after transforming. You can repopulate the ``metadata` field if needed by calling:

sample_collection.compute_metadata(overwrite=True)

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
fps (None) – an optional frame rate at which to resample the videos
min_fps (None) – an optional minimum frame rate. Videos with frame rate below this value are upsampled
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
reencode (False) – whether to re-encode the videos as H.264 MP4s
force_reencode (False) – whether to re-encode videos whose parameters already satisfy the specified values
delete_originals (False) – whether to delete the original videos after re-encoding
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be transformed
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_videos(sample_collection, frames_patt=None, frames=None, fps=None, max_fps=None, size=None, min_size=None, max_size=None, original_frame_numbers=True, force_sample=False, save_filepaths=False, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Samples the videos in the sample collection into directories of per-frame images according to the provided parameters using ffmpeg.

The frames for each sample are stored in a directory with the same basename as the input video with frame numbers/format specified by frames_patt.

For example, if frames_patt = "%%06d.jpg", then videos with the following paths:

/path/to/video1.mp4
/path/to/video2.mp4
...

would be sampled as follows:

/path/to/video1/
    000001.jpg
    000002.jpg
    ...
/path/to/video2/
    000001.jpg
    000002.jpg
    ...

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
frames_patt (None) – a pattern specifying the filename/format to use to store the sampled frames, e.g., "%%06d.jpg". The default value is fiftyone.config.default_sequence_idx + fiftyone.config.default_image_ext
frames (None) – an optional list of lists defining specific frames to sample from each video. Entries can also be None, in which case all frames will be sampled. If provided, fps and max_fps are ignored
fps (None) – an optional frame rate at which to sample frames
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
original_frame_numbers (True) – whether to use the original frame numbers when writing the output frames (True) or to instead reindex the frames as 1, 2, … (False)
force_sample (False) – whether to resample videos whose sampled frames already exist
save_filepaths (False) – whether to save the sampled frame paths in a filepath field of each frame of the input collection
delete_originals (False) – whether to delete the original videos after sampling
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be sampled
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.reencode_video(input_path, output_path, verbose=False, **kwargs)¶

Re-encodes the video using the H.264 codec.

By default, the re-encoding is performed via the following ffmpeg command:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

You can configure parameters of the re-encoding such as codec and compression by passing keyword arguments for eta.core.video.FFmpeg(**kwargs) to this function.

Parameters

input_path – the path to the input video
output_path – the path to write the output video
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.transform_video(input_path, output_path, fps=None, min_fps=None, max_fps=None, size=None, min_size=None, max_size=None, reencode=False, verbose=False, **kwargs)¶

Transforms the video according to the provided parameters using ffmpeg.

In addition to the size and frame rate parameters, if reencode == True, the following basic ffmpeg command structure is used to re-encode the video as an H.264 MP4:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

Parameters

input_path – the path to the input video
output_path – the path to write the output video
fps (None) – an optional frame rate at which to resample the videos
min_fps (None) – an optional minimum frame rate. Videos with frame rate below this value are upsampled
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
reencode (False) – whether to reencode the video (see main description)
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_video(input_path, output_patt, frames=None, fps=None, max_fps=None, size=None, min_size=None, max_size=None, original_frame_numbers=True, verbose=False, **kwargs)¶

Samples the video into a directory of per-frame images according to the provided parameters using ffmpeg.

Parameters

input_path – the path to the input video
output_patt – a pattern like /path/to/images/%%06d.jpg specifying the filename/format to write the sampled frames
frames (None) – an iterable of frame numbers to sample. If provided, fps and max_fps are ignored
fps (None) – an optional frame rate at which to sample the frames
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
original_frame_numbers (True) – whether to use the original frame numbers when writing the output frames (True) or to instead reindex the frames as 1, 2, … (False)
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_frames_uniform(frame_rate, total_frame_count=None, support=None, fps=None, max_fps=None, always_sample_last=False)¶

Returns a list of frame numbers sampled uniformly according to the provided parameters.

Parameters

frame_rate – the video frame rate
total_frame_count (None) – the total number of frames in the video
support (None) – a [first, last] frame range from which to sample
fps (None) – a frame rate at which to sample frames
max_fps (None) – a maximum frame rate at which to sample frames
always_sample_last (False) – whether to always sample the last frame

Returns

a list of frame numbers, or None if all frames should be sampled

fiftyone.utils.video.concat_videos(input_paths, output_path, verbose=False)¶

Concatenates the given list of videos, in order, into a single video.

Parameters

input_paths – a list of video paths
output_path – the path to write the output video
verbose (False) – whether to log the ffmpeg command that is executed