fiftyone.utils.video¶

Video utilities.

Copyright 2017-2022, Voxel51, Inc.
voxel51.com

Functions:

`concat_videos`(input_paths, output_path[, …])	Concatenates the given list of videos, in order, into a single video.
`extract_clip`(video_path, output_path[, …])	Extracts the specified clip from the video.
`reencode_video`(input_path, output_path[, …])	Re-encodes the video using the H.264 codec.
`reencode_videos`(sample_collection[, …])	Re-encodes the videos in the sample collection as H.264 MP4s that can be visualized in the FiftyOne App.
`sample_frames_uniform`(frame_rate[, …])	Returns a list of frame numbers sampled uniformly according to the provided parameters.
`sample_video`(input_path, output_patt[, …])	Samples the video into a directory of per-frame images according to the provided parameters using `ffmpeg`.
`sample_videos`(sample_collection[, …])	Samples the videos in the sample collection into directories of per-frame images according to the provided parameters using `ffmpeg`.
`transform_video`(input_path, output_path[, …])	Transforms the video according to the provided parameters using `ffmpeg`.
`transform_videos`(sample_collection[, fps, …])	Transforms the videos in the sample collection according to the provided parameters using `ffmpeg`.

fiftyone.utils.video.extract_clip(video_path, output_path, support=None, timestamps=None, metadata=None, fast=False)¶

Extracts the specified clip from the video.

Provide either suppport or timestamps to this method.

When fast=False, the following ffmpeg command is used:

# Slower, more accurate option
ffmpeg -ss <start_time> -i <video_path> -t <duration> <output_path>

When fast is True, the following two-step ffmpeg process is used:

# Faster, less accurate option
ffmpeg -ss <start_time> -i <video_path> -t <duration> -c copy <tmp_path>
ffmpeg -i <tmp_path> <output_path>

Parameters

video_path – the path to the video
output_path – the path to write the extracted clip
support (None) – the [first, last] frame number range to clip
timestamps (None) – the [start, stop] timestamps to clip, in seconds
metadata (None) – the fiftyone.core.metadata.VideoMetadata for the video
fast (False) – whether to use a faster-but-potentially-less-accurate strategy to extract the clip

fiftyone.utils.video.reencode_videos(sample_collection, force_reencode=True, media_field='filepath', output_field=None, output_dir=None, rel_dir=None, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Re-encodes the videos in the sample collection as H.264 MP4s that can be visualized in the FiftyOne App.

By default, the re-encoding is performed via the following ffmpeg command:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

You can configure parameters of the re-encoding such as codec and compression by passing keyword arguments for eta.core.video.FFmpeg(**kwargs) to this function.

Note

This method will not update the metadata field of the collection after transforming. You can repopulate the metadata field if needed by calling:

sample_collection.compute_metadata(overwrite=True)

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
force_reencode (True) – whether to re-encode videos that are already MP4s
media_field ("filepath") – the input field containing the video paths to transform
output_field (None) – an optional field in which to store the paths to the transformed videos. By default, media_field is updated in-place
output_dir (None) – an optional output directory in which to write the transformed videos. If none is provided, the videos are updated in-place
rel_dir (None) – an optional relative directory to strip from each input filepath to generate a unique identifier that is joined with output_dir to generate an output path for each video. This argument allows for populating nested subdirectories in output_dir that match the shape of the input paths
delete_originals (False) – whether to delete the original videos after re-encoding. This parameter has no effect if the videos are being updated in-place
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be re-encoded
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.transform_videos(sample_collection, fps=None, min_fps=None, max_fps=None, size=None, min_size=None, max_size=None, reencode=False, force_reencode=False, media_field='filepath', output_field=None, output_dir=None, rel_dir=None, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Transforms the videos in the sample collection according to the provided parameters using ffmpeg.

In addition to the size and frame rate parameters, if reencode == True, the following basic ffmpeg command structure is used to re-encode the videos as H.264 MP4s:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

Note

This method will not update the metadata field of the collection after transforming. You can repopulate the metadata field if needed by calling:

sample_collection.compute_metadata(overwrite=True)

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
fps (None) – an optional frame rate at which to resample the videos
min_fps (None) – an optional minimum frame rate. Videos with frame rate below this value are upsampled
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
reencode (False) – whether to re-encode the videos as H.264 MP4s
force_reencode (False) – whether to re-encode videos whose parameters already satisfy the specified values
media_field ("filepath") – the input field containing the video paths to transform
output_field (None) – an optional field in which to store the paths to the transformed videos. By default, media_field is updated in-place
output_dir (None) – an optional output directory in which to write the transformed videos. If none is provided, the videos are updated in-place
rel_dir (None) – an optional relative directory to strip from each input filepath to generate a unique identifier that is joined with output_dir to generate an output path for each video. This argument allows for populating nested subdirectories in output_dir that match the shape of the input paths
delete_originals (False) – whether to delete the original videos after re-encoding
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be transformed
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_videos(sample_collection, frames_patt=None, frames=None, fps=None, max_fps=None, size=None, min_size=None, max_size=None, original_frame_numbers=True, force_sample=False, save_filepaths=False, media_field='filepath', output_field=None, output_dir=None, rel_dir=None, delete_originals=False, skip_failures=False, verbose=False, **kwargs)¶

Samples the videos in the sample collection into directories of per-frame images according to the provided parameters using ffmpeg.

By default, each folder of images is written using the same basename as the input video. For example, if frames_patt = "%%06d.jpg", then videos with the following paths:

/path/to/video1.mp4
/path/to/video2.mp4
...

would be sampled as follows:

/path/to/video1/
    000001.jpg
    000002.jpg
    ...
/path/to/video2/
    000001.jpg
    000002.jpg
    ...

However, you can use the optional output_dir and rel_dir parameters to customize the location and shape of the sampled frame folders. For example, if output_dir = "/tmp" and rel_dir = "/path/to", then videos with the following paths:

/path/to/folderA/video1.mp4
/path/to/folderA/video2.mp4
/path/to/folderB/video3.mp4
...

would be sampled as follows:

/tmp/folderA/
    video1/
        000001.jpg
        000002.jpg
        ...
    video2/
        000001.jpg
        000002.jpg
        ...
/tmp/folderB/
    video3/
        000001.jpg
        000002.jpg
        ...

Parameters

sample_collection – a fiftyone.core.collections.SampleCollection
frames_patt (None) – a pattern specifying the filename/format to use to store the sampled frames, e.g., "%%06d.jpg". The default value is fiftyone.config.default_sequence_idx + fiftyone.config.default_image_ext
frames (None) – an optional list of lists defining specific frames to sample from each video. Entries can also be None, in which case all frames will be sampled. If provided, fps and max_fps are ignored
fps (None) – an optional frame rate at which to sample frames
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
original_frame_numbers (True) – whether to use the original frame numbers when writing the output frames (True) or to instead reindex the frames as 1, 2, … (False)
force_sample (False) – whether to resample videos whose sampled frames already exist
save_filepaths (False) – whether to save the sampled frame paths in the output_field field of each frame of the input collection
media_field ("filepath") – the input field containing the video paths to sample
output_field (None) – an optional frame field in which to store the paths to the sampled frames. By default, media_field is used
output_dir (None) – an optional output directory in which to write the sampled frames. By default, the frames are written in folders with the same basename of each video
rel_dir (None) – a relative directory to remove from the filepath of each video, if possible. The path is converted to an absolute path (if necessary) via fiftyone.core.utils.normalize_path(). This argument can be used in conjunction with output_dir to cause the sampled frames to be written in a nested directory structure within output_dir matching the shape of the input video’s folder structure
delete_originals (False) – whether to delete the original videos after sampling
skip_failures (False) – whether to gracefully continue without raising an error if a video cannot be sampled
verbose (False) – whether to log the ffmpeg commands that are executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.reencode_video(input_path, output_path, verbose=False, **kwargs)¶

Re-encodes the video using the H.264 codec.

By default, the re-encoding is performed via the following ffmpeg command:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

You can configure parameters of the re-encoding such as codec and compression by passing keyword arguments for eta.core.video.FFmpeg(**kwargs) to this function.

Parameters

input_path – the path to the input video
output_path – the path to write the output video
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.transform_video(input_path, output_path, fps=None, min_fps=None, max_fps=None, size=None, min_size=None, max_size=None, reencode=False, verbose=False, **kwargs)¶

Transforms the video according to the provided parameters using ffmpeg.

In addition to the size and frame rate parameters, if reencode == True, the following basic ffmpeg command structure is used to re-encode the video as an H.264 MP4:

ffmpeg \
    -loglevel error -vsync 0 -i $INPUT_PATH \
    -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -vsync 0 -an \
    $OUTPUT_PATH

Parameters

input_path – the path to the input video
output_path – the path to write the output video
fps (None) – an optional frame rate at which to resample the videos
min_fps (None) – an optional minimum frame rate. Videos with frame rate below this value are upsampled
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
reencode (False) – whether to reencode the video (see main description)
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_video(input_path, output_patt, frames=None, fps=None, max_fps=None, size=None, min_size=None, max_size=None, original_frame_numbers=True, verbose=False, **kwargs)¶

Samples the video into a directory of per-frame images according to the provided parameters using ffmpeg.

Parameters

input_path – the path to the input video
output_patt – a pattern like /path/to/images/%%06d.jpg specifying the filename/format to write the sampled frames
frames (None) – an iterable of frame numbers to sample. If provided, fps and max_fps are ignored
fps (None) – an optional frame rate at which to sample the frames
max_fps (None) – an optional maximum frame rate. Videos with frame rate exceeding this value are downsampled
size (None) – an optional (width, height) for each frame. One dimension can be -1, in which case the aspect ratio is preserved
min_size (None) – an optional minimum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
max_size (None) – an optional maximum (width, height) for each frame. A dimension can be -1 if no constraint should be applied. The frames are resized (aspect-preserving) if necessary to meet this constraint
original_frame_numbers (True) – whether to use the original frame numbers when writing the output frames (True) or to instead reindex the frames as 1, 2, … (False)
verbose (False) – whether to log the ffmpeg command that is executed
**kwargs – keyword arguments for eta.core.video.FFmpeg(**kwargs)

fiftyone.utils.video.sample_frames_uniform(frame_rate, total_frame_count=None, support=None, fps=None, max_fps=None, always_sample_last=False)¶

Returns a list of frame numbers sampled uniformly according to the provided parameters.

Parameters

frame_rate – the video frame rate
total_frame_count (None) – the total number of frames in the video
support (None) – a [first, last] frame range from which to sample
fps (None) – a frame rate at which to sample frames
max_fps (None) – a maximum frame rate at which to sample frames
always_sample_last (False) – whether to always sample the last frame

Returns

a list of frame numbers, or None if all frames should be sampled

fiftyone.utils.video.concat_videos(input_paths, output_path, verbose=False)¶

Concatenates the given list of videos, in order, into a single video.

Parameters

input_paths – a list of video paths
output_path – the path to write the output video
verbose (False) – whether to log the ffmpeg command that is executed