Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Add note regarding 4.2 changes

...

Info

Welcome to the OpenSense documentation! To complete this example, you will need to download OpenSim 4.1 or later. If you try the example and software, please send any issues or feedback to opensim@stanford.edu. Note that the functionality has been improved in later versions including the introduction of visualization tools for IMU data in version 4.2.


Anchor
topofpage
topofpage

OpenSense is a new workflow for analyzing movement with inertial measurement unit (IMU) data. In the page below, we introduce you to the tool, show you how to get started, and describe how to use the software to compute and analyze gait kinematics through a hands-on example.

Table of Contents
maxLevel2
minLevel2

What is OpenSense?

OpenSense is a workflow that enables users to compute the motions of body segments based on inertial measurement unit (IMU) data, as shown in the animation below. The OpenSense workflow is summarized in the text and flowchart below. 

Inputs: OpenSim Model and IMU Data

To get started, you will need an OpenSim model that includes the joints and degrees of freedom of interest. In the example below, we provide a model for studying lower-extremity kinematics during gait. There are many more models available through our Model Library. If you are solely interested in joint angles and other kinematic quantities (e.g., normalized muscle lengths and lengthening velocities), the model need not be scaled to the anthropometry of the subject.

You must also input orientations from one or more IMU sensors. We currently support Xsens and APDM file formats. You can create your own file converter to support any other IMU system and we plan to add support for additional sensor manufacturers in the future. We currently assume that sensor fusion and syncing have been performed using a vendor’s or third-party algorithm. Several open-source sensor fusion algorithms are also available on GitHub. 

Registration and Calibration

OpenSense provides an interface to associate and register each IMU sensor with a body segment of an OpenSim model (as an IMU Frame). We provide a basic calibration routine in which the first frame time step of IMU data is registered to the default pose of the model. You change the registration pose by changing the default coordinate values of the model. You can also write your own calibration procedures in Matlab, Python, etc. to optimize the initial pose of the model for calibration using other data sources (markers, goniometer, etc). Read more about these steps in our User's Guide chapter on the IMU Placer tool.

Computing Inverse Kinematics

An inverse kinematics method is used to compute the set of joint angles at each time step of a motion that minimizes the errors between the experimental IMU orientations and the model’s IMU Frames. The angles can then be used as inputs to other OpenSim tools and analyses or you can visualize these angles in the OpenSim GUI. The OpenSense capabilities are available through the command line and through scripting (Matlab or Python). The resulting Model and Motion . As of OpenSim 4.2, the calibration and inverse kinematics steps are also available through the OpenSim GUI. The resulting Model and Motion can be loaded, visualized, and analyzed in the OpenSim 4.0 GUI. In the future, we will also provide a direct GUI-based tool to run IMU-based kinematics. Read more about this step in the User's Guide chapter on IMU Inverse Kinematics.


...

How to Setup the OpenSense

...

Tools

The OpenSense workflow is available as of OpenSim 4.1, onward. You can perform the OpenSense workflow from the command line and through Matlab and Python scripting on both Mac and Windows. As with OpenSim, the OpenSense tools use XML settings files to specify the details of your workflow. Whether you are running from the command line or Matlab, you . To get started, you will first need to download and install the latest OpenSim version (minimum version is 4.1). OpenSense can be downloaded from SimTK, with both Windows and Mac builds available. 

...

Back to top

Running OpenSense to Compute Gait Kinematics

Now that you've installed OpenSim, we will show you how to use the software through a hands-on example using example experimental IMU data from a study of lower extremity gait kinematics. The example data. models, scripts, and setup files can be found in your OpenSim resources directory under /Code/Matlab/OpenSenseExample. You can also download a zip of the example files _here_.

The basic steps for running an IMU-based OpenSense kinematics analysis are the following;

The flowchart below shows the workflow for the example. We will import the IMU sensor data, calibrate our OpenSim model, compute inverse kinematics, and then visualize the results.  

Image Removed

We use Xsens sensor data in this example, but all the steps for using APDM sensors are identical except for data reading. Read how to import APDM sensor data in the section below. Please note that the data in this example is for illustrative purposes and not intended for research use.

...

The first step is to collect your data and convert it into OpenSim format. You must place the sensors on your subject, typically with one sensor per tracked segment. You then must collect calibration data, where the subject is in a known pose, followed by the movement of interest (e.g., gait). For each IMU sensor, you must keep track of which sensor is placed on which body. You must also keep track of the timestamp(s) for the calibration pose.

For example, in this study, we have eight sensors placed as follows:

...

Image Removed

  • IMUs were placed on the Trunk, Pelvis, and the right and left Thighs, Shanks, and Feet.
  • It is important to note which sensor ID number represents which body on the subject. For example, Xsens sensors will have sensor names such as MT_012005D6_009-001_00B42279 where the section in bold is different for each sensor. In this example, it is noted that sensor MT_012005D6_009-001_00B421E6 is attached to the pelvis. 

Before collecting data for the movement of interest, you must collect data with the subject in a known, calibration pose. The You can perform the OpenSense workflow on Mac or Windows through:

  • Matlab scripting: To use OpenSense in Matlab, follow the Matlab scripting setup instructions.
  • Python scripting: To use OpenSense in Python, follow the Python scripting setup instructions.
  • Command line: To use OpenSense from the command line, read about the Command Line Utilities
  • GUI: To use OpenSense tools from the application GUI (version 4.2 and later). The setup files are included under the "Models\Rajagopal_OpenSense" folder included with the distribution.

As with OpenSim, the OpenSense tools use XML settings files to specify the details of your workflow. 


Back to top

...

Running OpenSense to Compute Gait Kinematics

Now that you've setup OpenSense, we will show you how to use the software through a hands-on example using example experimental IMU data from a study of lower extremity gait kinematics. The example data, models, scripts, and setup files can be found in your OpenSim resources directory under [Your Documents Directory]/OpenSim/OpenSim 4.1/Code/Matlab/OpenSenseExample.

The basic steps for running an IMU-based OpenSense kinematics analysis are the following;

The flowchart below shows the workflow for the example. We will import the IMU sensor data, calibrate our OpenSim model, compute inverse kinematics, and then visualize the results.  

Image Added

We use Xsens sensor data in this example, but all the steps for using APDM sensors are identical except for data reading. Read how to import APDM sensor data in the section below. Please note that the data in this example is for illustrative purposes and not intended for research use.

Step One: Collect and Prepare IMU Data
Anchor
stepone
stepone

The first step is to collect your data and convert it into a format that you can read into OpenSim and process with the OpenSense workflow.

  1. You must place the sensors on your subject, typically with one sensor per tracked segment. For each IMU sensor, you must keep track of which sensor is placed on which body.
  2. You then must collect calibration data, where the subject is in a known pose (i.e., specify or record each joint angle). You must keep track of the timestamp(s) for the calibration pose. The calibration data will be used by OpenSense to register the IMUs to the OpenSim model. The subject's pose during the calibration data collection must, as closely as possible, match the (editable) default pose defined by the model. You can standardize a simple static pose for each subject by using simple measurement equipment, like a goniometer.

In our example, the calibration pose is a neutral pose where the hips, knees, and ankles were at (or close) to 0 degrees.

...

  1. The OpenSense workflow also allows you to perform a heading correction to help orient the model in 3D space during the calibration phase. To do heading correction, you need to specify a base IMU (e.g., the pelvis) and the heading of that IMU (i.e., which axis is pointing forward). So if you are going to perform the heading correction, you will need to also note the base IMU and its heading.
  2. Collect the movement of interest (e.g., gait) with the same IMU placement.
  3. Pre-process your data. IMU sensor systems, like Xsens and APDM, typically provide features that perform sensor fusion, time syncing, and data interpolation for missing entries. The current version of OpenSense assumes that this pre-processing has already been performed and that you are inputting processed rotation matrices. 

...

Once you have collected data and prepared it (i.e., compiled IMU-based orientations for calibration and movement trials), you must convert it to OpenSim's file format and associating it with an OpenSim model. Data from IMUs can be in various formats: a single file with numbered sensor names (e.g., APDM) or multiple files with sensor-specific numbering (e.g., Xsens). Upon import, OpenSim will create a single, time synced, Storage (.sto) file format for orientations, converting the rotation matrices into quaternions. 

In this example, we will be using data from an Xsens system that has been processed and exported to an Xsens text format. You can find this data in the IMUData folder. Each Xsens sensor is represented by a single text (.txt) file with time histories of the internal sensor data.

To read your data, you first need to create a file that lets OpenSense know which sensor is associated with which body segment in the Model. In our example, this file is called myIMUMappings.xml. You can open and edit this file in any text editor. In this settings/XML file you specify the following information:

Image Removed

  • <trial_prefix> This is the common prefix of all the .txt files for a given movement trial. In our example, the prefix is "MT_012005D6_009-001".
  • <ExperimentalSensor name> looks for the string after <trial_prefix> to identify the the specific sensor file. In our example, the first sensor uses the prefix "_00B42268"
  • <name_in_model> is the corresponding name of the sensor in the OpenSim model. In our example, the first sensor is associated with the torso segment of our model.

Each IMU sensor is represented as a Frame in an OpenSim Model, where a Frame is an orthogonal XYZ coordinate system. When you read in your data, OpenSense will find the appropriate IMU frame in your model (based on the mappings XML file) or create an IMU Frame, if it doesn't already exist. OpenSense uses a naming convention where we expect the sensor to be named as <bodyname>_imu. For example, the OpenSim model has a right femur body called femur_r, therefore the IMU sensor must be called femur_r_imu.

The IMU reader then creates a storage file with the orientation data for each sensor, where each column in the storage file is named according to the frame in the corresponding OpenSim model. 

Command-line tool for creating an orientations file from IMU sensor data

To read your data into OpenSense from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExampleFiles folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -ReadX IMUData/ myIMUMappings.xml 

Since we are working with Xsens data, we use the -ReadX option (read Xsens data). We next provide the directory where the IMU sensor data is (IMUData) and then the name of the local IMU Mappings file (myIMUMappings.xml).

Running this command line call will generate an orientations file called <trial_name>_orientations.sto (<trial_name> is defined in your myIMUMappings.xml file) in your OpenSenseExampleFiles folder. 

The next step is to calibrate the IMUs to an OpenSim model.

...

The OpenSense Calibration step takes an OpenSim Model and the IMU calibration data and finds the initial orientations of the IMU Frames (i.e. offsets) relative to the OpenSim body segments. We provide a basic algorithm for calibration or you can also create your own methods of calibration by developing your own algorithms (in C++ or via scripting) to compute a default pose and/or the transforms of the IMU sensors.

To use OpenSense's calibration, you must provide an OpenSim Model in the calibration step. In our example, we are using the Rajagopal (2015) model. As noted above, on data read, either your Model should have IMU frames attached that correspond to the name_in_model specified in Step Two, or if you use our assumed naming convention (<bodyname>_imu), the calibrate step will add IMU Frames to the model as long as there is a corresponding body segment with a matching <bodyname>. 

OpenSense calibration assumes that the pose of the subject in the calibration data matches the default pose of the model. In our example, the calibration pose is with the pelvis, hip, knee, and ankle at neutral, so we did not need to make any adjustments to the model's default pose. If you use a different pose, you can edit the pose of the input in the OpenSim GUI, through scripting, or in XML (see Coordinate Controls and Poses to learn how to edit the default pose through the OpenSim GUI). 

You must next provide the calibration data. OpenSense assumes the first time point corresponds to the calibration pose. If you have a trial where the calibration pose is performed at some time other than the first time row, you must edit your orientations file (or make a new one) where the first time row best corresponds to the calibration pose. 

You can also specify optional arguments that enable OpenSense to correct or adjust for the overall difference in the heading (forward direction) of the IMU data versus that of the OpenSim model. Typically, an OpenSim model is facing in the positive X direction of the ground frame in the initial pose, but the base IMU (e.g., on the pelvis or torso) can have any initial heading. Given the <base_imu_label> (the label that identifies the base IMU in the provided orientation data) and the <base_heading_axis> (the 'x', 'y' or 'z' axis of the base IMU that represents its heading direction), OpenSense will compute the angular offset between the two poses and use it to rotate all the orientation data so that the heading of the base IMU is now directed along the X-axis of the OpenSim ground reference frame (same as the model). If the <base_imu_label> is not provided, then no heading correction is applied. If the <base_imu_label> is provided and no <base_heading_axis> the 'z' axis of the base IMU is assumed to be its heading direction.

The output of the calibration step is a calibrated model, where each IMU is registered to the OpenSim model. The image below shows our example subject with IMU's on the pelvis, trunk, thighs, shanks, and feet segments and the corresponding OpenSim Model with the matching pose. 

Image Removed

Command-line tool for calibrating an OpenSim Model

To Calibrate your model with IMU Orientations from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExampleFiles folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -Calibrate myIMUPlacer_Setup.xml

In our example, the pelvis_imu is set as the base IMU and z is the axis of the base IMU that corresponds to its heading,

...

Image Removed

A visualizer window will appear, showing the calibrated model. The pose of the model is determined by the model's default pose and will not change from one calibration to the next (unless you edit the model's default pose). What will change is the orientation of the sensors attached to each body. You can zoom in on the sensors, represented as small orange bricks located at the COM of each body. 

Note: You can close the visualizer window, when selected, by using the keyboard shortcut of ctrl-Q (command-Q on Mac).

...

Image Removed

...

The terminal window will print out the calibration offset for each IMU. This is the transform between the model body and the IMU sensor. 

To continue the calibration, and print the calibrated model to file, select the visualizer window and press any key to continue. 

The Calibrated Model is written to file and will have the prefix 'calibrated_' added (i.e., if the input Model file is called model.osim, the output calibrated model file will be named calibrated_model.osim).

...

Now that you have read in your data and calibrated your model, you can use OpenSense's Inverse Kinematics to track Orientation data from IMU sensors. The Inverse Kinematics step finds the pose of the model at each time-step that minimizes, in the least-squares sense, the difference between the orientation data from the IMU sensors and the IMU Frames on your calibrated model. The computed kinematics depend on both the calibrated model and the sensor data. Thus to perform inverse kinematics tracking of orientation data you need (i) a Calibrated Model (.osim), (ii) an orientations file (as quaternions), and (iii) an Inverse Kinematics Setup file (.xml). Using the Calibrated Model we generated in the previous section, we will track orientation data for walking that we read in during Step Two. 

View and edit the IK Setup File in a text editor

In a text editor— such as Notepad++, SublimeText, Atom, or Matlab— open the IK_Setup.xml file. The setup file stores properties that tell OpenSense how to run the Inverse Kinematics simulation. In the setup file, you specify:

  • <time_range> The time range for the inverse kinematics tracking (in seconds). In our example, we use data between 7.25 and 15 seconds.

  • <sensor_to_opensim_rotations> The rotation needed to convert the IMU world frame (typically Z up, Y to the left) to the OpenSim world frame (Y up, Z to the right). 
  • <model_file_name> The name/path to the calibrated model file (.osim) to be used in tracking. In our example, this is the calibrated_Rajagopal_2015.osim file that was the output of Step Three.
  • <orientations_file_name> The name/path to a .sto file of sensor frame orientations (as quaternions) that will be tracked. In our example, this is the MT_012005D6_009-001_orientations.sto we created in Step Two.
  • <results_directory> The directory where the results will be printed to file. 

An example setup file is shown below.

Image Removed 

For now, leave these settings as they are. This settings file can be copied and edited for your own workflow.

Command-line tool for performing an Inverse Kinematics with Orientation.

To perform Inverse Kinematics with OpenSense from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExampleFiles folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -IK myIMUIK_Setup.xml

The output motion file is written to file and will have the prefix 'ik_' added (i.e., if the input orientations file is called MT_012005D6_009-001_orientations.sto, the output motion file will be named IKResults/ik_MT_012005D6_009-001_orientations.mot)

...

Visualization isn't generated when running OpenSense Inverse Kinematics through the command line, so it is useful to view the results of the simulation using the OpenSim application (GUI) visualizer. You can also use OpenSim's plotter to plot the kinematics or perform further analyses with other OpenSim pipeline tools. (Note that you will generally need to scale your model and provide ground reaction forces if you want to generate muscle driven simulations.)

To view the Inverse Kinematics results:

  • Open the OpenSim 4.1 application. (Download Link)
  • Open the model: calibrated_Rajagopal_2015.osim
  • Load the motion you created in Step Four: IKResults/ik_MT_012005D6_009-001_orientations.mot.  Since the IMUs cannot track global translations, only relative orientations, the model appears to rotate about a single place. 

Image Removed

To plot the OpenSense Kinematics

  • In the taskbar, select Tools → Plot.
  • Select Y-Quantity→ Load File, and select IKResults/MT_012005D6_009-001_orientations.sto
  • Select the hip_flexion_r, knee_angle_r, & ankle_angle_r coordinates
  • Select X-Quantity→ Time
  • Select Add
Image Removed

...

You can run OpenSense through the Matlab scripting environment. The Matlab interface provides additional tools to customize your workflow and also allows you to visualize the results of the inverse kinematics tool in real-time.

Setting up your Matlab scripting environment

To use OpenSense through Matlab, you must perform the following setup. 

Download and unzip OpenSense on your system. Add OpenSense to your system path with instructions here.

  1. Launch MATLAB. If you're using Windows make sure to "Run as administrator" (available by right-clicking the MATLAB application in the start menu, etc.).
  2. Find the configureOpenSim.m script in your OpenSense resources directory. This will be in your OpenSense install directory, <your_OpenSense_dir>/Resources/Code/Matlab.
  3. Run the configureOpenSim.m file from the current folder. 
  4. When the script finishes, you should see a dialog box notifying you if the script succeeded. If the script did not succeed, read the output in the command window for more information.
  5. Restart MATLAB; the OpenSim libraries will not be recognized until doing so.

To test that everything is configured correctly, run the following command:

...

>>> org.opensim.modeling.opensimCommon.GetVersion()    % This should print the version of OpenSim that you've configured with MATLAB (4.1)

The configureOpenSim.m file will detect any installations of OpenSim that were previously configured with MATLAB, and will "remove" them from MATLAB (the other OpenSim installations are not deleted, they are simply no longer configured with MATLAB). The configureOpenSim.m file also backs up any changes it makes to MATLAB configuration files.

Example Matlab scripts to compute gait kinematics

We have provided a set of scripts to run through the workflow from the example above in Matlab.

Matlab scripting to create an orientations file from IMU sensor data

You can read your IMU data into OpenSense through the Matlab scripting interface. Note that, as in the example above, we will still use the myIMUMappings.xml file to define the mappings from IMU sensor to OpenSim model. A feature of the scripting interface is that you can also read and export the IMU accelerations, magnetometer, and gyro data to file. 

  • Launch Matlab and navigate to the OpenSenseExampleFiles folder.
  • Open and run the OpenSense_IMUDataConverter.m script.
  • Run the script. This will generate an orientations file, MT_012005D6_009-001_orientations.sto, as well as _acceleration, _magnetometer, and _gyro data files. 

Matlab scripting for calibration

You can also perform calibration through Matlab scripting. The scripting interface is similar to the command line. You provide the model names, the orientations file, and, optionally, the base IMU name and heading. You can also specify if you want to have the calibrated model visualized.

To perform calibration, run the OpenSense_CalibrateModel.m script in Matlab. A visualization window will open, showing the calibrated model, as well as the calibrated model being written to file. 

Matlab scripting to run inverse kinematics

The Scripting interface for the OpenSense IMUInverseKinematicsTool gives you finer control over the inverse kinematic properties. In particular, you can visualize the Inverse Kinematics tracking.

To perform Inverse Kinematics, open and run the file OpenSense_OrientationTracking.m script in Matlab. A visualizer window will open, showing the kinematic tracking of the IMU orientation by the model. The results of the orientation tracking will be written to the IKResults directory. Results can be plotted in the OpenSim GUI similarly done above.

...

You can run OpenSense through the Python scripting environment. The Python interface provides additional tools to customize your workflow and also allows you to visualize the results of the inverse kinematics tool in real-time.

Setting up your Python scripting environment

To use OpenSense in Python, you must perform the following setup. 

Download and unzip OpenSense on your system. Add OpenSense to your system path with instructions here.

  1. Download and Setup Python 3 from Python.org. The installer will place Python on your system path for you. 
  2. Open a Command Prompt (Windows) or Terminal (MacOS) and navigate to <your_OpenSense_dir>/Resources/Code/Python. 
  3. Execute the following command 
Code Block
>> python setup.py install

Example Python scripts to compute gait kinematics

We have provided a set of scripts to run through the workflow from the example in Python. The steps parallel the Matlab instructions described above.

Learn More and Future Work

...

Typically, APDM exports a trial as a .h5 file and as a .csv ASCII text file that is comma delimited, grouped in order by sensor. The OpenSense APDM Reader can only read CSV file types.

To read the APDM CSV file, you must create a file that associates the column labels in the APDM .csv file with an OpenSim model body segment. You can open and edit this file in any text editor. In this settings/XML file you specify the following information:

  • <ExperimentalSensor name> A string to identify the specific column in the APDM sensor file. In our example, the first sensor uses the column name "Trunk".
  • <name_in_model> The corresponding name of the sensor in the OpenSim Model. The first sensor is associated with the torso_imu segment of the model.

An example Settings file for APDM sensors can be Downloaded here. You can open and edit this file in any text editor. A snippet of the file is shown below:

Image Removed

For our example, we have eight sensors placed as follows:

Image Added


  • IMUs were placed on the Trunk, Pelvis, and the right and left Thighs, Shanks, and Feet.
  • It is important to note which sensor ID number represents which body on the subject. For example, Xsens sensors will have sensor names such as MT_012005D6_009-001_00B42279 where the section in bold is different for each sensor. In this example, it is noted that sensor MT_012005D6_009-001_00B421E6 is attached to the pelvis. 
  • In our example, the calibration pose is a neutral pose where the hips, knees, and ankles were at (or close) to 0 degrees. The subject is in the calibration pose in the first time step of the data collection.
  • We are using the pelvis as the base IMU. For the pelvis IMU, the z-axis is pointing forward in our calibration.
  • We used the Xsens software to output quaternions for our calibration and gait data.


Step Two: Convert IMU Data into OpenSim Format 
Anchor
steptwo
steptwo

Once you have collected and pre-processed your data, you must convert it to OpenSim's file format and associate it with an OpenSim model. Data from IMUs can be in various formats: a single file with numbered sensor names (e.g., APDM) or multiple files with sensor-specific numbering (e.g., Xsens). Upon import, OpenSim will create a single, time synced, Storage (.sto) file format for orientations, converting the rotation matrices into quaternions. 

In this example, we will be using data from an Xsens system that has been pre-processed (e.g., time-syncing and sensor fusion has been performed)  and exported to an Xsens text format. You can find this data in the IMUData folder. Each Xsens sensor is represented by a single text (.txt) file with time histories of the internal sensor data. We assume the data reported by the IMU system to include orientations (in the case of XSens these are assumed to be direction-cosine-matrices).

To read your data, you first need to create a file that lets OpenSense know which sensor is associated with which body segment in the Model. In our example, this file is called myIMUMappings.xml. You can open and edit this file in any text editor. In this settings/XML file you specify the following information:

  • <trial_prefix> This is the common prefix of all the .txt files for a given movement trial. In our example, the prefix is "MT_012005D6_009-001".
  • <ExperimentalSensor name> looks for the string after <trial_prefix> to identify the the specific sensor file. In our example, the first sensor uses the prefix "_00B42268"
  • <name_in_model> is the corresponding name of the sensor in the OpenSim model. In our example, the first sensor is associated with the torso segment of our model.

An example setup file is shown below:

Image Added


Each IMU sensor is represented as a Frame in an OpenSim Model, where a Frame is an orthogonal XYZ coordinate system. When you read in your data, OpenSense will find the appropriate IMU Frame in your model (based on the mappings XML file) or create an IMU Frame, if it doesn't already exist. OpenSense uses a naming convention where we expect the sensor to be named as <bodyname>_imu. For example, the OpenSim model has a right femur body called femur_r, therefore the IMU sensor must be called femur_r_imu.

The IMU reader then creates a storage file with the orientation data for each sensor, where each column in the storage file is named according to the Frame in the corresponding OpenSim model. To read in your data, use the following steps, depending on how you are accessing the OpenSense workflow

Matlab commands to create an orientations file from IMU sensor data

Launch Matlab and set the OpenSimExampleFiles folder as your working directory. Enter the following code to read in the IMU sensor data.

Code Block
languagetext
% Import the OpenSim libraries
import org.opensim.modeling.*; 

% Create an xsensDataReader and supply the settings file that maps IMUs to your model
xsensSettings = XsensDataReaderSettings('myIMUMappings.xml'); % 
xsensReader = XsensDataReader(xsensSettings);

% Read the quaternion data and write it to a STO file for in OpenSense workflow
tables = xsensReader.read('IMUData/');
quaternionTable = xsensReader.getOrientationsTable(tables);
STOFileAdapterQuaternion.write(quaternionTable,  [char(xsensSettings.get_trial_prefix()) '_orientations.sto']);


Command-line tool to create an orientations file from IMU sensor data

Expand
titleUse the command line ...

To read your data from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExample folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -ReadXsens IMUData/ myIMUMappings.xml 

Since we are working with Xsens data, we use the -ReadX option (read Xsens data). We next provide the directory where the IMU sensor data is (IMUData) and then the name of the local IMU Mappings file (myIMUMappings.xml).

Running this command line call will generate an orientations file called <trial_name>_orientations.sto (<trial_name> is defined in your myIMUMappings.xml file) in your OpenSenseExample folder. 

Step Three: Calibrate an OpenSim Model 
Anchor
stepthree
stepthree

The next step is to calibrate the IMUs to an OpenSim model. The OpenSense Calibration step takes an OpenSim Model and the IMU calibration data and finds the initial orientations of the IMU Frames (i.e. offsets) relative to the OpenSim body segments. We provide a basic algorithm for calibration or you can also create your own methods of calibration by developing your own algorithms (in C++ or via scripting) to compute a default pose and/or the transforms of the IMU sensors.

To calibrate your model, you first need a setup file that stores information about the model, orientations file, and some settings to be used during the calibration procedure. In our example, this file is called myIMUPlacer_Setup.xml. You can open and edit this file in any text editor. In this settings/XML file you specify the following information:

  • <model_file> To use OpenSense's calibration, you must provide an OpenSim Model in the calibration step. In our example, we are using the Rajagopal (2015) model. As noted above, on data read, your Model should have IMU Frames attached that correspond to the name_in_model specified in Step Two, or if you use our assumed naming convention (<bodyname>_imu), the calibrate step will add IMU Frames to the model as long as there is a corresponding body segment with a matching <bodyname>. 

OpenSense calibration assumes that the pose of the subject in the calibration data matches the default pose of the model. In our example, the calibration pose is with the pelvis, hip, knee, and ankle at neutral, so we did not need to make any adjustments to the model's default pose. If you use a different pose, you can edit the pose of the input in the OpenSim GUI, through scripting, or in XML (see Coordinate Controls and Poses to learn how to edit the default pose through the OpenSim GUI). 

  • <orientation_file_for_calibration> You must next provide the calibration data. OpenSense assumes the first time point corresponds to the calibration pose. If you have a trial where the calibration pose is performed at some time other than the first time row, you must edit your orientations file (or make a new one) where the first time row best corresponds to the calibration pose. 

  • <sensor_to_opensim_rotations> You must also provide the rotation needed to convert the IMU world Frame (typically Z up, Y to the left) to the OpenSim world Frame (Y up, Z to the right). 

You can also specify optional arguments that enable OpenSense to correct or adjust for the overall difference in the heading (forward direction) of the IMU data versus that of the OpenSim model. Typically, an OpenSim model is facing in the positive X direction of the ground Frame in the initial pose, but the base IMU (e.g., on the pelvis or torso) can have any initial heading. To perform heading correction update the following settings:

  • <base_imu_label> This is the label that identifies the base IMU in the provided orientation data. The default is no base_imu_label provided, and thus no heading correction will be performed.
  • <base_heading_axis> This is the axis of the base IMU that represents its heading direction. The axis can be 'x', '-x', 'y', '-y', 'z' or '-z'.

An example setup file is shown below:

Image Added

OpenSense will compute the angular offset between the two poses and use it to rotate all the orientation data so that the heading of the base IMU is now directed along the X-axis of the OpenSim ground reference Frame (same as the model). In our example, the pelvis_imu is set as the base IMU and z is the axis of the base IMU that corresponds to its heading, If either <base_imu_label> or <base_heading_axis> are not provided, then no heading correction is performed. 

  • <output_model_file> The output of the calibration step is a calibrated model, where each IMU is registered to the OpenSim model. This setting allows you to specify the output model file name.

The image below shows our example subject with IMU's on the pelvis, trunk, thighs, shanks, and feet segments and the corresponding OpenSim model with the matching pose. 

Image Added

Matlab commands to calibrate a model in OpenSense

Code Block
% Setup and run the IMUPlacer tool, with model visualization set to true
myIMUPlacer = IMUPlacer('myIMUPlacer_Setup.xml'); 
myIMUPlacer.run(true);  

% Write the calibrated model to file
myIMUPlacer.getCalibratedModel().print('Rajagopal_2015_calibrated.osim');

Command-line tool to calibrate a model in OpenSense

Expand
titleUse the command line ...

To calibrate your model with IMU Orientations from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExample folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -Calibrate myIMUPlacer_Setup.xml



Image Added

A visualizer window will appear, showing the calibrated model. The pose of the model is determined by the model's default pose and will not change from one calibration to the next (unless you edit the model's default pose). What will change is the orientation of the sensors attached to each body. You can zoom in on the sensors, represented as small orange bricks located at the COM of each body. 


Note: You can close the visualizer window, when selected, by using the keyboard shortcut of ctrl-Q (command-Q on Mac).

Image Added

You will see a print out the calibration offset for each IMU. This is the transform between the model body and the IMU sensor. 

To continue the calibration, and print the calibrated model to file, select the visualizer window and press any key to continue. 

The Calibrated Model is written to file and will have the postfix '' added (i.e., if the input Model file is called model.osim, the output calibrated model file will be named model_calibrated.osim).

Using the OpenSim Application (GUI)

As of version 4.2 you can execute this step from the OpenSim application by invoking Tools→IMU Placer and loading the settings from the file myIMUPlacer_Setup.xml created above, or entering the data manually in the dialog as shown below, then hitting the Run button. After you run the tool, a new model with IMUs placed on it will appear in the application.

Image Added


Step Four: Perform IMU Sensor Tracking 
Anchor
stepfour
stepfour

Now that you have read in your data and calibrated your model, you can use OpenSense's Inverse Kinematics to track Orientation data from IMU sensors. The Inverse Kinematics step finds the pose of the model at each time-step that minimizes, in the least-squares sense, the difference between the orientation data from the IMU sensors and the IMU Frames on your calibrated model. The computed kinematics depend on both the calibrated model and the sensor data. Thus to perform inverse kinematics tracking of orientation data you need (i) a Calibrated Model (.osim), (ii) an orientations file (as quaternions), and (iii) an Inverse Kinematics Setup file (.xml). Using the calibrated model we generated in the previous section, we will track orientation data for walking that we read in during Step Two. 

In a text editor— such as Notepad++, SublimeText, Atom, or Matlab— open the myIMUIK_Setup.xml file. The setup file stores properties that tell OpenSense how to run the Inverse Kinematics simulation. In the setup file, you specify:

  • <time_range> The time range for the inverse kinematics tracking (in seconds). In our example, we use data between 7.25 and 15 seconds.

  • <sensor_to_opensim_rotations> The rotation needed to convert the IMU world Frame (typically Z up, Y to the left) to the OpenSim world Frame (Y up, Z to the right). 
  • <model_file_name> The name/path to the calibrated model file (.osim) to be used in tracking. In our example, this is the Rajagopal_2015_calibrated.osim file that was the output of Step Three.
  • <orientations_file_name> The name/path to a .sto file of sensor Frame orientations (as quaternions) that will be tracked. In our example, this is the MT_012005D6_009-001_orientations.sto we created in Step Two.
  • <results_directory> The directory where the results will be printed to file. 

An example setup file is shown below.

Image Added 

For now, leave these settings as they are. This settings file can be copied and edited for your own workflow.


Matlab commands to perform inverse kinematics

Execute the following code in Matlab to run inverse kinematics with the IMU data.

Code Block
% Setup and run the IMU IK tool with visualization set to true.
imuIK = IMUInverseKinematicsTool('myIMUIK_Setup.xml');
imuIK.run(true);


Command-line tool to perform inverse kinematics

Expand
titleUse the command line ...

To perform Inverse Kinematics with OpenSense from the command line, use the following steps.

  • Launch a terminal window (or command prompt) and navigate to the OpenSenseExample folder.
  • At the prompt, enter the following command. 
Code Block
>> opensense -IK myIMUIK_Setup.xml

The output motion file is written to file and will have the prefix 'ik_' added (i.e., if the input orientations file is called MT_012005D6_009-001_orientations.sto, the output motion file will be named IKResults/ik_MT_012005D6_009-001_orientations.mot)

Using the OpenSim Application (GUI)

As of version 4.2 you can execute this step from the OpenSim application by invoking Tools→IMU Inverse Kinematics and loading the settings from the file imuInverseKinematics_Setup.xml created above, or entering the data manually in the dialog as shown below, then hitting the Run button. The IK problem will be solved and the solution will be animated in the application.

Image Added

Step Five: Visualize the Results of IMU Tracking 
Anchor
visualizeResults
visualizeResults

You can view and plot the results of the simulation using the OpenSim application (GUI) visualizer. You can also use OpenSim's plotter to plot the kinematics or perform further analyses with other OpenSim pipeline tools. (Note that you will generally need to scale your model and provide ground reaction forces if you want to generate muscle driven simulations.)

To view the Inverse Kinematics results:

  • Open the OpenSim application.
  • Open the model: calibrated_Rajagopal_2015.osim
  • Load the motion you created in Step Four: IKResults/ik_MT_012005D6_009-001_orientations.mot.  Since the IMUs cannot track global translations, only relative orientations, the model appears to rotate about a single place. 

Image Added

To plot the OpenSense Kinematics

  • In the taskbar, select Tools → Plot.
  • Select Y-Quantity→ Load File, and select IKResults/MT_012005D6_009-001_orientations.sto
  • Select the hip_flexion_r, knee_angle_r, & ankle_angle_r coordinates
  • Select X-Quantity→ Time
  • Select Add
Image Added


Customizing the OpenSense Workflow via Matlab Scripting 
Anchor
matlabscripting
matlabscripting

We ran the OpenSense through the Matlab scripting environment in a simplified manner in the steps above. The Matlab interface provides additional tools to customize your workflow.

Example Matlab scripts to compute gait kinematics

We have provided a set of scripts to run through the workflow from the example above in Matlab.

Matlab scripting to create an orientations file from IMU sensor data

You can read your IMU data into OpenSense through the Matlab scripting interface. Note that, as in the example above, we will still use the myIMUMappings.xml file to define the mappings from IMU sensor to OpenSim model. A feature of the scripting interface is that you can also read and export the IMU accelerations, magnetometer, and gyro data to file. 

  • Launch Matlab and navigate to the OpenSenseExample folder.
  • Open and run the OpenSense_IMUDataConverter.m script.
  • Run the script. This will generate an orientations file, MT_012005D6_009-001_orientations.sto, as well as _acceleration, _magnetometer, and _gyro data files. 

Matlab scripting for calibration

You can also perform calibration through Matlab scripting. The scripting interface is similar to the command line. You provide the model names, the orientations file, and, optionally, the base IMU name and heading. You can also specify if you want to have the calibrated model visualized.

To perform calibration, run the OpenSense_CalibrateModel.m script in Matlab. A visualization window will open, showing the calibrated model, as well as the calibrated model being written to file. 

Matlab scripting to run inverse kinematics

The Scripting interface for the OpenSense IMUInverseKinematicsTool gives you finer control over the inverse kinematic properties. In particular, you can visualize the Inverse Kinematics tracking.

To perform Inverse Kinematics, open and run the file OpenSense_OrientationTracking.m script in Matlab. A visualizer window will open, showing the kinematic tracking of the IMU orientation by the model. The results of the orientation tracking will be written to the IKResults directory. Results can be plotted in the OpenSim GUI similarly done above.

Running OpenSense via Python Scripting 
Anchor
pythonscripting
pythonscripting

You can run OpenSense through the Python scripting environment. The Python interface provides additional tools to customize your workflow and also allows you to visualize the results of the inverse kinematics tool in real-time.

Example Python scripts to compute gait kinematics

We have provided a set of scripts to run through the workflow from the example in Python. The steps parallel the Matlab instructions described above. You can find these files in your OpenSim resources directory.


Learn More and Future Work

Importing APDM Sensor Data 
Anchor
readapdmfiles
readapdmfiles

Typically, APDM exports a trial as a .h5 file and as a .csv ASCII text file that is comma delimited, grouped in order by sensor. The OpenSense APDM Reader can only read CSV file types.

To read the APDM CSV file, you must create a file that associates the column labels in the APDM .csv file with an OpenSim model body segment. You can open and edit this file in any text editor. In this settings/XML file you specify the following information:

  • <ExperimentalSensor name> A string to identify the specific column in the APDM sensor file. In our example, the first sensor uses the column name "Trunk".
  • <name_in_model> The corresponding name of the sensor in the OpenSim Model. The first sensor is associated with the torso_imu segment of the model.


Download an example APDM Settings file and a corresponding example APDM sensor dataYou can open and edit this file in any text editor. A snippet of the Settings file is shown below:

Image Added

Matlab commands to create an orientations file from APDM IMU sensor data

Launch Matlab and set the folder that contains the APDM files as your working directory. Enter the following code, changing the trial name and the myIMUMappings.xml file names for your workflow.

Code Block
% Import the OpenSim libraries
import org.opensim.modeling.*;

% Define the trial name
trialName = 'exampleAPDM_Data.csv';

% Create an APDMDataReader and supply the settings file that maps IMUs to your model
apdmSettings = APDMDataReaderSettings('exampleAPDM_Settings.xml'); 
myAPDMDataReader = APDMDataReader(apdmSettings);
 
% Read the quaternion data and write it to a STO file for use in OpenSense workflow
tables = myAPDMDataReader.read( trialName );
quaternionTable = myAPDMDataReader.getOrientationsTable(tables);
STOFileAdapterQuaternion.write(quaternionTable,  strrep(trialName,'.csv', '_orientations.sto') );


To use the OpenSense APDM command-line tool to read in the APDM data and export an orientations .sto file to use in OpenSense, you would use the below call. 

Code Block
>> opensense -ReadAReadAPDM myAPDFileexampleAPDM_Data.csv myAPDMMappingsexampleAPDM_Settings.xml 

Back to top


Future Work

The current version of OpenSense is our first step in bringing IMU-based biomechanics to the research community. We plan several additional enhancements and new features in future releases, for example:

  • Direct support for additional IMU sensors
  • More advanced approaches to calibration and interfaces to better support users developing their own calibration protocolsSupport for OpenSense inverse kinematics directly in the OpenSim application (GUI)

  • Tools to easily visualize data and results for debugging (e.g., are the sensors registered to the correct body segments?)

...