Inertial Measurement Unit (IMU) sensors are electronic devices that measure and report a body's specific force, angular rate, and sometimes the magnetic field surrounding the body, using a combination of accelerometers, gyroscopes and magnetometers. Most people interact with these devices daily, via the IMU in their smartphone that allows them accurate navigating ability. Recent advances in wearable IMU technology, however, are allowing users to capture and store hours of kinematic data. Traditionally, kinematic data was only attainable in motion capture laboratories equipped with multiple cameras that tracked a subject's motion via reflective markers worn on the body. With wearable, wireless IMU sensors, we have a unique opportunity - the ability to capture human motion in "natural" settings (outside of the laboratory) lasting multiple hours. One such IMU company, Xsens, uses a 3D accelerometer, 3D gyroscope, and 3D magnetometer in each of their IMU sensors to capture full 6 degree-of-freedom tracking of body segments.
In this study, we started with a simple application using two Xsens IMUs to measure knee angle. Our experiment calculated the knee angle as a person is sitting in a chair with their initial knee flexion angle at 0 degrees (full extension) and flexing to 90 degrees at frequencies similar to that of walking and running. The knee flexion angle is calculated with measurements from IMUs attached to the shank and thigh. Using 2D video, markers and a goniometer, the subject's leg motion was be measured with video analysis to obtain a "ground truth" knee angle measurement. This was compared to the knee angle determined from the IMU sensor data combined with the Xsens biomechanical model, output as a data file of joint angles over time. The knee joint angles was then input into OpenSim to observe the motion. In addition to directly outputting the joint angles, each individual IMU sensor's orientation was output prior to feedback from the Xsens biomechanical model. These orientations, output as quaternions, were transformed into rotation matrices and input into an OpenSim Inverse Kinematics (IK) script. IK in OpenSim determined the positions that best matched the input rotation matrices at each time step, and the resulting knee angle was observed. Once model kinematics were successfully reconstructed in OpenSim, we validated the determined knee joint angle from both the Xsens biomechanical model and using IK in OpenSim with our ground truth motion capture data.
The goal of this project was to determine if knee angle could be measured accurately with the IMU sensors. The motivation for this goal is that if validated, the IMUs can be used for more complex movements, such as gait, to characterize kinematics without the restrictions of a motion capture system. Many questions can be addressed with the ability to analyze kinematics in a natural setting and for long periods of time.
I. Pipeline for Processing IMU Data
II. Knee flexion experiment (IMU and 2D video marker tracking)
III. Reconstructing motion in OpenSim using Xsens-estimated knee joint angle
IV. Reconstructing motion in OpenSim from Inverse Kinematics using Xsens IMU orientation data
In the pipeline for processing IMU data, the IMU sensors are used first to obtain linear acceleration (from the accelerometer), angular velocity (from the gyroscope), and magnetometer readings of the desired motion or activity. The IMU sensors we used were sampling at 60 Hz and were small and lightweight (47x30x13 mm, 16 g). Xsens developed a sensor fusion algorithm, a Kalman filter called XKF-3, such that orientation and position of the IMU sensors can be accurately estimated.
Following the lower pathway, you see that Xsens estimates kinematics using a combination of orientation and position data from the IMUs and their biomechanical model which incorporates joint characteristics and external contacts. In this way, they can update joint uncertainties and feedback updated orientation and position information to the sensor fusion algorithms. Xsens then outputs these estimated model kinematics, including the joint angles of the model at each time step. In our study, we were specifically looking at knee joint angle, so we extracted the Xsens-estimated knee rotation about the X,Y,Z axes.
We also worked through the upper pathway, using sensor orientation from the Xsens processing (prior to feedback with the biomechanical model) to estimate joint angles. This orientation data, output as quaternions, was translated into rotation matrices and input into an OpenSim IK C++ script that output various model kinematics. We specifically looked at the knee joint angle and used the gait10dof18musc OpenSim model (see below for a comparison of the Xsens vs. OpenSim knee model).
|Xsens Knee Model ||OpenSim Knee Model |
|Camera captures sagittal knee rotation.|
Marker placement on leg.
|~5 deg||~90 deg|
|2D video screen captures of the marker tracking for the subject's leg at knee angle in the fully extended position (~5 deg) versus fully flexed (~90 deg).|
Instructions for creating an OpenSim motion file from XSens .mnvx file in Matlab (See README_JointAngles.txt)
1. Run read_jointAngles_lowerbody_write_motion_file.m (Matlab)
a. This runs main_mvnx.m (this is code modified from Xsens Toolkit, Xsens North America Inc.)
This file loads two figures of the “first segment position” - the pelvis rotation in 3D and the pelvis 3D translation
i. main_mvnx.m uses the load_mvnx.m function (Xsens Toolkit)
ii. Outputs a data tree with all the biomechanical model segment and IMU sensor data (joint angles, joint velocities, joint accelerations etc.)
b. Creates a table of all joints and joint angles from IMU data at every time point (jointTable)
c. Outputs selected joint angles to a matrix with time as first row (allAnglesMatrix)
d. Creates motion file named “mvnxfilename_Xsens_jointangle_q.mot” of joint angles to input in Open Sim from Xsens IMU Data
This file has required OpenSim headings and formats data correctly for use with OpenSim model (but only has pelvis_tilt pelvis_tx pelvis_ty hip_flexion_r knee_angle_r ankle_angle_r)
2. Open your .osim model in OpenSim (the .osim models we have included correspond to each .mvnx file, and all start in the default position determined by the subject’s physical body position at the beginning of the experiment).
For example, if you used the LegSwing_40bpm_5-26-17.mvnx then you should load the gait10dof18musc_40.osim model
3. Input created .mot file into OpenSim through 'File -> Load Motion' and pick the mvnxfilename_Xsens_jointangle_q.mot file.
|~5 deg||~90 deg|
Examples of the reconstructed seated knee flexion motion in the OpenSim model.
The code then runs an inverse kinematic simulation and outputs OpenSim kinematics files:
|40 BPM (Walking)||80 BPM||120 BPM (Running)|
Root mean square error (RMSE) between Xsens joint angle and ground truth (2D video marker tracking) and OpenSim IK joint angle and ground truth (2D video marker tracking):
|RMSE (deg)||40 BPM||80 BPM||120 BPM|
*Compare with literature RMSE of 5.09 deg for knee flexion using Xsens sensors while walking (Zhang et al. 2013)
Comparing Xsens vs. OpenSim Inverse Kinematics joint estimates
In the above plots, the knee angle estimation that utilized Xsens IMU sensor orientations in combination with OpenSim Inverse Kinematics is shown in blue, the knee angle estimation from the Xsens biomechanical model is shown in dark orange, and the ground truth knee angle determined from 2D video marker tracking is shown in yellow.
In each plot, we see that the Xsens estimate is closer to the ground truth knee angle than the OpenSim_IK estimate. This is also reflected in the root mean square error values for each method: averaged across all three speeds, Xsens and OpenSim IK had an RMSE of 3.65 deg and 7.16 deg, respectively. These RMSEs are comparable with literature values of Xsens IMU sensor validation with camera-based motion capture, where they saw RMSE's of 5.09 degrees for knee flexion during walking .
The Xsens system is unique in its approach to estimate body segment orientation and position changes with a combination of the integrated inertial data as well as continuous update using a biomechanical model of the human body. By facilitating the constraints of the model, notably that the segments are connected by joints, the kinematics of body segments are corrected for drift and other errors.
We think that Xsens has a better knee angle estimate because of their sensor fusion scheme with both a prediction and correction step. In the prediction step (pictured on left below), they integrate acceleration and angular velocities on all sensors. This is followed by prediction of segment kinematics using a known sensor to body alignment and a 23 segment biomechanical model of the body. However, skin and soft tissue artifacts influence the IMU measurements, much like traditional reflective marker systems due to contracting muscles and skin/fat motion . In the correction and joint measurement update step (right), these artifacts are reduced by using the knowledge that the two segments are on average connected but with a statistical uncertainty. Detecting contact points of the body with the external world limit the boundless integration error of the assembled body model in the global frame and constrains the global position and velocity. For example, if a person is sitting but moving around (e.g. on a bike), then you could include a constraint that the pelvis is fixed. After the correction step, these updated kinematics are fed back to the sensor fusion algorithms and segment kinematic step to be used in the next time frame .
Xsens used a scaled model based on the subject's body measurements; however, our OpenSim model was not scaled. Thus, we were not able to incorporate position data and the Inverse Kinematics simulation was only based on orientation data, which makes the joint estimation less accurate. Moreover, with only orientation driven motion "capture", it is not possible to analyze the clearance of both feet (such as in running or jumping) or determine displacement of body with respect to a coordinate system not fixed to the body. Therefore, in the future we want to do real-time estimation of body segment orientation and position changes using raw IMU data and an Opensim model, which we can scale based on the subject's body measurements.
There are a number of reasons why the Xsens joint estimation is not equal to the ground truth knee angles. As mentioned in the experimental set-up, the subject must stand in a neutral pose, so that the rotation from sensor to body segment can be determined by matching the orientation of the sensor in the global frame with the known orientation of each segment in this pose. However, if the subject's sensors are placed incorrectly, (e.g. the thigh sensor is put on horizontally instead of vertically) or even shifted slightly, then the model will be expecting the sensor to be placed one way and will calibrate based on this incorrect conformation. Thus, the resulting joint angles will have slight errors. In a similar manner, if the subject does not assume the correct neutral pose, the sensors will be calibrated incorrectly and there will be error incurred. In addition, if there are magnetic disturbances, the magnetometer readings will be erroneous and this will affect the orientation estimation and thereby the joint angle estimation. Note that this would affect both the Xsens estimation and the OpenSim based method.
We thought that the accuracy in IMU-based joint angle estimation might decrease at higher velocity motions. However, from the above results at the varying speeds, we cannot conclude this to be true. Perhaps this is because we did not sustain the activity at high enough speeds and/or for long enough time periods. Xsens documentation references deteriorations in XKF-3's kinematic estimation accuracy after at least 10-20 sec of high acceleration motion. This is because the Kalman filter is constantly fusing signals from the accelerometer with those of the gyroscope to correct for angular velocity integration drift over time. If the accelerometer becomes unreliable (as it can be at high speeds), then the estimate is dependent on the gyroscope-based tracking. This loses accuracy after 10-20 seconds and begins to drift.
It would be fruitful to use these sensors for longer periods of time in high velocity motion like running. However, even if the system is able to correct for the errors introduced by higher velocity motion, we suspect that the impact forces during contact in running could also introduce error in the joint angle estimations.
Characterizing Parkinsonian Gait
Though Parkinsonian motor function has been studied heavily in clinics and research labs, there is far less known about how patients function at home or in daily life, let alone movement classifiers for such activities. IMUs that allow characterization of body kinematics outside the lab could be worn by a Parkinson’s patient to validate past motion capture experiments that concluded that Parkinson’s patients have decreased range of motion in their hips, knees and ankles during straight forward walking as compared to healthy age-matched control subjects (Morrris et al. 2005). Furthermore, they could be used to inform closed loop neurostimulation for deep brain stimulation therapy or sensory cueing to improve gait and other movement.
Characterizing Running Gait during Fatigue
Many studies have shown that fatigue alters running kinematics. A review of incidence and associated potential risk factors of lower extremity running injuries in long distance runners found that long distance training increases injury risk, with knee injuries being the most common . While some studies have been able to observe these changes, motion capture limits the studies to treadmill running and most of the studies have only studied fatigue in less than 30 minutes. IMUs can be used outside of the lab to characterize running kinematics of fatigue and the resulting injuries. Additionally, the change in kinematics while running on different terrains can be studied which could also help prevent injury. Though this has been tried by Reenalda et al. in 2016 using Xsens IMU sensors, there were limitations with data acquisition and battery life . Rapid improvements in this technology will allow for effective exploration of running kinematics and fatigue.
Though we were able to answer our first 2 research questions, we leave our last research question for the future work:
Can kinematics be reconstructed in OpenSim with IMU measurements in real-time to use OpenSim as the biomechanical model for orientation and position feedback?
We would like to thank the BIOE485 Teaching Team, class and NMBL lab for all of their guidance and support throughout this project.
Xsens Technologies (2013). Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors.
Gait 2392 and 2354 Models - OpenSim Documentation
Ferrari A, Cutti A G and Cappello A 2010a A new formulation of the coefficient of multiple correlation to assess the similarity of waveforms measured synchronously by different motion analysis protocols Gait Posture 31 540–2
Zhang, J. T., Novak, A. C., Brouwer, B., & Li, Q. (2013). Concurrent validation of Xsens MVN measurement of lower limb joint angular kinematics. Physiological measurement, 34(8), N63.
Morris, M., Iansek, R., McGinley, J., Matyas, T. and Huxham, F. (2005), Three-dimensional gait biomechanics in Parkinson's disease: Evidence for a centrally mediated amplitude regulation disorder. Mov. Disord., 20: 40–50. doi:10.1002/mds.20278
van Gent, R.N. et al. (2007). Incidence and determinants of lower extremity running injuries in long distance runners: A systematic review. British Journal of Sports Medicine.
Reenalda, J., Maartens, E., Homan, L., & Buurke, J. J. (2016). Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics. Journal of Biomechanics, 49(14), 3362-3367.
Home: BIOE-ME 485 Spring 2017