Egocentric EMG-Force dataset: does EMG predict grip force?

Q: How do I load the egocentric-emg-force dataset in FiftyOne?

Call foz.load_zoo_dataset("egocentric-emg-force"). It loads as eight samples, one per task, each pointing at a native .mcap episode file.

FiftyOne 1.22.0 added a new Dataset Zoo entry pairing real wrist EMG with a real, independently computed finger-force estimate across eight household tasks. Running our own cross-correlation, sample by sample, on all 16 hand and episode pairs finds a strongest correlation of only r = 0.43 and a measured EMG-to-force lag that swings from -230 ms to +200 ms depending on the episode, a long way from the single 80 ms lead the dataset card documents.

FiftyOne 1.22.0 added the egocentric-emg-force dataset to the Dataset Zoo

FiftyOne 1.22.0 added the egocentric-emg-force dataset to the Voxel51 Dataset Zoo: someone wearing a chest-mounted RGB-D camera and an eight-channel electromyography (EMG) band on each wrist works through eight household tasks, from hand cream application to sweeping and mopping a floor. Alongside the video, each episode carries the raw EMG signal from both forearms at roughly 550 Hz, a 21-point hand skeleton, wrist IMU, and an estimate of how hard each finger is pressing, derived from hand pose and depth rather than an instrumented glove. Across the full dataset that is 38.2 minutes of recording, 67,185 camera and depth frames, 2,539,493 EMG samples, and 133,721 per-finger force readings, shipped as eight native .mcap episodes FiftyOne opens directly.

Full dataset card, with exact schema and license: https://docs.voxel51.com/dataset_zoo/datasets/egocentric_emg_force.html

Key takeaways

FiftyOne 1.22.0 added the egocentric-emg-force dataset to the Dataset Zoo, pairing eight-channel wrist EMG with a vision-derived per-finger force estimate across eight household tasks.
Across all 16 hand-episode pairs in the egocentric-emg-force dataset, the strongest EMG-to-force correlation is r = 0.426, and 13 of the 16 pairs fall below r = 0.3.
The dataset card documents a fixed 80 ms EMG lead, but a per-episode lag search finds best lags spanning -230 ms to +200 ms, with a median of 40 ms.
Treat the finger-force channel in egocentric-emg-force as an independent vision-derived signal, not a drop-in proxy for a physical force reading or a raw EMG level.
The same lag-search cross-correlation works on any FiftyOne dataset that pairs two independently sampled MCAP topics, such as an IMU against visual odometry.

Two independent sensors, one physical event: a real check, not a demo

Most multimodal datasets give you one signal per phenomenon and ask you to trust it. This one gives you two unrelated sensing paths pointed at the same physical event: an eight-channel EMG band reading the wearer's own forearm muscle activity, and a hand-pose-and-depth model estimating how hard their fingers are pressing. If contact force really tracks muscle effort, the two should move together, shifted by whatever delay the excitation-contraction coupling and the vision pipeline's own latency add up to. The dataset card states that delay as 80 ms. We do not take that on faith: we run our own lag-search cross-correlation per episode per hand and report what we actually measure, next to the documented number.

The rest of this post is one worked example: load all eight episodes, rectify and smooth each hand's EMG into a linear envelope, cross-correlate it against the measured force curve across a plus-or-minus 300 ms lag search, and check whether the two signals agree closely enough to treat one as a stand-in for the other.

Eight episodes, queryable before opening a single one

Loading the dataset gives eight samples, one per task, each pointing at an episode.fo.mcap file. FiftyOne reads scalar fields straight off each sample (task, package_id, num_emg_samples, num_force_frames, peak finger force) so the grid is filterable and sortable before any episode is opened.

The eight tasks span a real range of grip types and durations: short, fine-motor episodes like hand cream application and paper towel wiping (1,000 to 1,200 force samples each) sit next to long, whole-body tidying episodes like tidy dining room and sweep and mop floor (16,000 to 18,000 force samples each). That range matters for this post's measurement: a correlation computed on a two-minute fine-motor task and one computed on a much longer tidying task are not automatically comparable, which is why every number below is reported per episode per hand rather than pooled into one dataset-wide average.

FiftyOne sample grid showing the eight egocentric-emg-force episodes, with filterable fields including task, package_id, num_emg_samples, and num_force_frames in the left sidebar. — All eight episodes load as native .mcap files; task, package_id, and frame counts are queryable fields on the sample grid, not something you have to open a video to find.

Opening one episode (here, washing machine laundry) shows FiftyOne's real multimodal MCAP viewer: an egocentric camera feed, a depth stream, and a 3D hand-keypoints pane driven by the same wrist tracking used to derive finger force. The hand-keypoints pane starts empty. In this episode the wrist band and hand are out of frame until 0:13.30 into the recording, so the pane shows a Jump to data control rather than 13 seconds of nothing.

FiftyOne multimodal MCAP viewer on the washing machine laundry episode at 0:00, with the hand-keypoints pane empty and showing a Jump to data control because tracking starts at 0:13.30. — The wearer is still walking to the laundry room in the first 13 seconds; hand-keypoints, which needs the wrist in frame, waits for real data instead of rendering an empty skeleton.

The raw EMG channel is not the signal you correlate against force

FiftyOne's topic inspector lets you read any MCAP topic's raw messages directly, which is a useful sanity check before writing extraction code: the 8-channel EMG topic reports values clustered around 127, the center of an 8-bit analog-to-digital converter (ADC) reading, not a clean muscle-activity curve.

FiftyOne topic inspector showing raw eight-channel EMG values between 125 and 128, clustered near the ADC baseline, next to a live 3D hand-keypoints pane and the egocentric camera and depth feeds. — Raw EMG channel values hover near the ADC baseline of 127 while the hand skeleton is live in the 3D pane. Each of these per-sample values is noisy on its own; the correlation in this post uses a rectified, smoothed envelope of all eight channels, never a single raw sample.

Our extraction code rectifies each of the 8 channels around that ADC baseline, averages them into one envelope value per timestamp, resamples onto a uniform 200 Hz grid, and applies a 150 ms centered moving average, the standard linear-envelope step in EMG signal processing. Only after that smoothing does the signal become something you would expect to track a physiological output like force at all.

Background on the FiftyOne multimodal MCAP viewer used for every screenshot in this post: https://docs.voxel51.com/user_guide/multimodal.html

Does forearm EMG predict measured grip force? What we found

For each of the 8 episodes and both hands (16 pairs total, n = 8 episodes), we cross-correlated the smoothed EMG envelope against the measured finger-force curve across a -300 ms to +300 ms lag search in 10 ms steps, then kept the lag with the strongest correlation magnitude. The best pair anywhere in the dataset is washing machine laundry, left hand, at r = 0.426 and a 40 ms lag. The weakest is sink hand washing, left hand, at r = -0.042. Not one of the 16 pairs reaches r = 0.5; 13 of the 16 stay below r = 0.3.

Best-lag correlation between the smoothed EMG envelope and measured finger force, for all 16 hand and episode pairs in the egocentric-emg-force dataset. Each row reports the strongest correlation found across a -300 ms to +300 ms lag search in 10 ms steps. Not one pair reaches r = 0.5, and the best lag disagrees with the documented 80 ms in 15 of 16 cases.

Best-lag correlation between the smoothed EMG envelope and measured finger force, for all 16 hand and episode pairs in the egocentric-emg-force dataset. Each row reports the strongest correlation found across a -300 ms to +300 ms lag search in 10 ms steps. Not one pair reaches r = 0.5, and the best lag disagrees with the documented 80 ms in 15 of 16 cases.
Task	Hand	Best-lag r	Best lag (ms)	n samples
washing machine laundry	left	0.426	40	3807
paper towel hand wiping	right	0.411	60	1014
paper towel hand wiping	left	0.357	60	1014
tidy bedroom	left	0.292	40	14922
sweep and mop floor	left	0.255	60	16358
tidy living room	left	0.200	60	10264
sink hand washing	right	0.195	-210	1021
tidy dining room	left	0.156	0	18289
sweep and mop floor	right	0.143	-20	16360
washing machine laundry	right	0.132	60	3806
tidy bedroom	right	0.127	200	14919
hand cream application	right	0.111	20	1134
tidy living room	right	0.100	100	10262
hand cream application	left	-0.097	10	1213
tidy dining room	right	0.083	-160	18289
sink hand washing	left	-0.042	-230	1021

Two stacked time-series charts comparing the lag-shifted EMG envelope against measured finger force. The strongest pair, washing machine laundry left hand at r = 0.426, drifts in and out of sync. The weakest pair, sink hand washing left hand at r = -0.042, shows force spikes with no matching EMG burst. — Even the strongest pair in the dataset (top) still shows EMG bursts and force spikes drifting in and out of sync; the weakest pair (bottom) shows large force spikes with no corresponding EMG burst at all.

The measured lag disagrees with the documented number just as much as the correlation strength does. The dataset card states the EMG leads the camera by a fixed 80 ms. Our own search finds best lags ranging from -230 ms to 200 ms across the 16 pairs, a median of 40 ms and a mean of only 5.6 ms once positive and negative lags average out. The single strongest pair does land at a plausible EMG-leads-force lag (40 ms), which is at least directionally consistent with the card. But that is one pair out of 16, not a pattern reliable enough to hang a fixed offset on.

This is a small-sample finding (n = 8 episodes, one wearer, one wearing session) and it is about the dataset, not about a bug in our own code: the extraction and lag search are the standard EMG-envelope pipeline, and we are reading both signals straight off the MCAP topics the dataset ships. The honest read is that the finger-force channel here is best treated as an independent, vision-derived signal, not a proxy you can swap for a physical force reading or a raw EMG level.

Building your own: lag-search correlation between two MCAP topics

Nothing in this workflow is specific to EMG or force. Any FiftyOne dataset that pairs two independently sampled sensor streams in the same MCAP episode, an IMU against visual odometry, an audio level against a transcript's word timing, one camera's brightness against another's, can be checked the same way: read both topics' raw messages directly with the mcap library (no FiftyOne-internal API required), resample both onto a shared uniform time grid, and search a lag window with Pearson correlation rather than assuming zero lag or trusting a vendor's documented offset.

for lag_ms in np.arange(-300, 301, 10):
    shifted = signal_a_times + int(lag_ms * 1e6)  # ms -> ns
    aligned_a = np.interp(signal_b_times, shifted, signal_a_values)
    r = np.corrcoef(aligned_a, signal_b_values)[0, 1]
    # keep whichever lag maximizes abs(r), same shape as this post's
    # correlate_at_lag() in build_and_measure.py

Writing the per-pair result back onto the sample (as emg_force_best_abs_corr here) turns a one-off analysis into a field you can sort and filter on in the App, which is what makes the worst and best pairs in this post something you can click to, not just numbers in a table. The same pattern generalizes past a pairwise check: once the field exists on every sample, you can build a saved view for "pairs worth trusting" (r above some threshold) and a separate one for pairs that need a second sensor before you'd rely on either alone.

Where a shaky EMG-to-force link is not just an academic caveat

Myoelectric prosthetic control: prosthetic and exoskeleton hands infer grip intent and force from forearm EMG because there is no other channel once a limb is missing. This dataset's finding, that EMG and an independently measured force diverge sample to sample even in the best case, is the same calibration problem proportional myoelectric control already has to fight, not a new one.
Industrial ergonomics monitoring: warehouse and assembly-line ergonomics programs use wearable EMG armbands to flag high-exertion grips and pinches. Camera-based systems that try to estimate hand force from RGB-D and pose alone, to avoid fitting every worker with an EMG kit, are making exactly the substitution this post tested. At r = 0.43 in the best case, that substitution is not yet solid enough to base a safety threshold on.
Teleoperation haptic feedback: a remote operator's haptic glove needs a grip-force signal to render feedback when the manipulated hand is a robot gripper watched only by a camera. A vision-only force proxy this weakly correlated with ground truth would produce haptic cues that mislead as often as they help.

Try it

The dataset, the extraction and lag-search code, the chart script, and this post's generator are all re-runnable end to end. A short video of the real camera feed and hand-keypoints pane playing back together is included alongside this post.

import fiftyone.zoo as fozdataset = foz.load_zoo_dataset("egocentric-emg-force")

Dataset card and full schema: https://docs.voxel51.com/dataset_zoo/datasets/egocentric_emg_force.html

FAQ

Not well, at least in this dataset. The strongest of the 16 hand-episode pairs in egocentric-emg-force reaches only r = 0.426, and 13 of the 16 pairs fall below r = 0.3. No pair reaches r = 0.5.

It varies by episode. The dataset card documents a fixed 80 ms EMG lead, but a per-episode, per-hand lag search across a -300 ms to +300 ms window finds best lags from -230 ms to +200 ms, with a median of 40 ms and a mean of 5.6 ms.

It is estimated, not physically measured. A model derives per-finger force from hand pose and depth rather than from an instrumented glove, which is what makes it an independent second signal to check EMG against.

Raw values sit near 127, the center of an 8-bit ADC reading, and are noisy sample to sample. The signal has to be rectified around that baseline, averaged across all eight channels, resampled to a uniform grid, and smoothed with a 150 ms moving average to form a linear envelope first.

Call foz.load_zoo_dataset("egocentric-emg-force"). It loads as eight samples, one per task, each pointing at a native .mcap episode file.

Egocentric EMG-Force in FiftyOne: When Forearm Muscle Signal Doesn't Predict Measured Grip Force

Talk to an AI expert

FiftyOne 1.22.0 added the egocentric-emg-force dataset to the Dataset Zoo

Key takeaways

Two independent sensors, one physical event: a real check, not a demo

Eight episodes, queryable before opening a single one

The raw EMG channel is not the signal you correlate against force

Does forearm EMG predict measured grip force? What we found

Building your own: lag-search correlation between two MCAP topics

Where a shaky EMG-to-force link is not just an academic caveat

Try it

FAQ

Talk to an AI expert

Related posts

Related posts

Egocentric EMG-Force in FiftyOne: When Forearm Muscle Signal Doesn't Predict Measured Grip Force

Talk to an AI expert

FiftyOne 1.22.0 added the egocentric-emg-force dataset to the Dataset Zoo

Key takeaways

Two independent sensors, one physical event: a real check, not a demo

Eight episodes, queryable before opening a single one

The raw EMG channel is not the signal you correlate against force

Does forearm EMG predict measured grip force? What we found

Building your own: lag-search correlation between two MCAP topics

Where a shaky EMG-to-force link is not just an academic caveat

Try it

FAQ

How well does forearm EMG predict measured grip force?

What is the EMG-to-force lag in the egocentric-emg-force dataset?

How is finger force measured in the egocentric-emg-force dataset?

Why does the raw EMG channel need processing before you correlate it against force?

How do I load the egocentric-emg-force dataset in FiftyOne?

Talk to an AI expert

Related posts

Related posts