Estimating Finger Force from Forearm Ultrasound

Core Idea

Finger force estimation is important for robotic teleoperation, rehabilitation robotics, haptic feedback, and human-machine interaction. In these applications, the system may need to know not only the user’s hand gesture, but also the force being applied during interaction.

Forearm ultrasound provides a view of internal muscle and tendon deformation. When a finger applies force, the forearm anatomy changes in a way that can be captured by ultrasound. This work investigates whether those image-level changes can be mapped to finger force.

The learning problem can be written compactly as

where (x_i) is a forearm ultrasound image and (f_i) is the corresponding force measurement.

Task Formulation

We considered two related tasks.

The first task is binary force classification:

where (y_i=1) indicates force application and (y_i=0) indicates no force.

The second task is continuous force regression:

The classification task asks whether force is being applied. The regression task asks how much force is being applied.

Experimental Setup

A wireless ultrasound probe was strapped to the forearm to acquire B-mode ultrasound images. A force sensor provided the corresponding ground-truth force measurement. The synchronized ultrasound-force pairs were used for both classification and regression.

Both shuffled and non-shuffled train-test splits were evaluated. The shuffled split measures performance when samples are randomly mixed from the same acquisition distribution. The non-shuffled split is more restrictive because it preserves greater temporal separation between training and testing samples.

Results

For force/no-force classification, the model achieved 98.4% average accuracy on shuffled data and 82% average accuracy on non-shuffled data.

For continuous force estimation, the average RMSE was 0.02 N for shuffled data and 0.2 N for non-shuffled data. Since the maximum force range was 5 N, the non-shuffled RMSE corresponds to

of the maximum force range.

These results suggest that forearm ultrasound contains force-relevant information and can support both force-state classification and continuous force estimation.

Interpretation

This result extends ultrasound-based hand decoding beyond gesture classification. Gesture classification estimates hand state. Force estimation adds information about interaction intensity.

For robotic and virtual interaction, this distinction is important. A teleoperated robot, prosthetic device, or haptic interface may need to infer both what the user intends to do and how strongly the user intends to interact with the environment.

In this context, forearm ultrasound is useful because it provides a musculoskeletal view of the forearm rather than only a surface-level electrical measurement.

Limitations and Next Steps

The non-shuffled results show a clear performance drop relative to the shuffled split, indicating that temporal separation, acquisition variability, and probe placement remain important challenges.

Future work should evaluate cross-session and cross-subject generalization, improve robustness to probe placement, and integrate force estimation into real-time human-machine interfaces. A natural next step is joint estimation of hand gesture, hand pose, and finger-applied force from the same ultrasound stream.

Citation

@inproceedings{bimbraw2023force,
  title={Estimating Force Exerted by the Fingers Based on Forearm Ultrasound},
  author={Bimbraw, Keshav and Zhang, Haichong K.},
  booktitle={2023 IEEE International Ultrasonics Symposium (IUS)},
  pages={1--4},
  year={2023},
  organization={IEEE},
  doi={10.1109/IUS51837.2023.10306652}
}