This is a library and a set of utilities built on PCL to work with a CHR UM6 IMU mounted on an OpenNI 1.x compatible depth camera like the Microsoft Kinect. Classes and utilities are included to grab, save, and playback full-frame 30FPS depth data and corresponding 100Hz IMU data. The imucam::Grabber and imucam::Reader classes extend pcl::Grabber and can supply pcl::PointClouds, OpenNI depth and RGB images, or imucam::Frames which package the images with corresponding IMU data.
A calibration tool is provided to estimate the rigid rotation of the IMU relative to the camera based on captured depth images of a flat horizontal ground plane.
Though RGB-D+IMU data is the focus, most of the code and utilities can be used with any subset of the RGB, depth, and IMU datastreams. Support for IMUs other than the UM6 is not included, but could be implemented with some work by abstracting the imucam::UM6 interface. Supporting non-OpennNI 1.x cameras would also be possible with some work.
Copyright (c) 2014, Marsette Vona. All code is BSD Licensed.
imucam::UM6 is a general purpose C++ interface for communicating with the CHR UM6 IMU. It can be used independently of PCL and the other parts of imucam.imucam::Frame encapsulates a single captured depth image, RGB color image, and associated UM6 IMU data. The images are in OpenNI 1.x binary formats.imucam::Grabber is a pcl::Grabber that can produce imucam::Frames in addition to the usual OpenNI images.imucam::Writer writes imucam::Frames to disk with real-time PCLZF compression. IMU data and metadata are saved in corresponding XML files which are compatible with those saved by pcl/visualization/tools/openni_image.cpp in PCL.imucam::Reader is like imucam::Grabber but reads data from disk. Both PCLZF+XML and TUM format data are supported. The reconstructed OpenNI images are identical to those originally generated by the camera, and an effort is made to produce them with their original timing.imucam::Viewer visualizes imucam::Frames as separate images and/or as a point cloud. The live IMU orientation and gravity vector can optionally be drawn in the latter view. Interactive mouse picking is supported in both the image and cloud views.imucam::Recorder and imucam::Player are interactive tools to record and play back imucam::Frames.imucam::PickPlayer is a specialized player that replays interactive picks that were saved to a text file.imucam::Calibrator is a specialized player that performs calibration using imucam::DominantPlane, more info below.Also see imucam-classes.txt and the header files for additional doc.
record_frames - instantiates imucam::Grabber and imucam::Recorder to display live RGB-D+IMU data and optionally record it to disk in PCLZF+XML format. Recording uses a circular buffer to maintain full frame rate (30FPS 640x480 RGB-D, 100Hz IMU) without hiccups on current laptops and workstations.play_frames - instantiates imucam::Reader and imucam::Player to playback RGB-D+IMU PCLZF+XML data previously saved with record_frames, or TUM format RGB-D data. Supports variable playback speed, pause, frame stepping forward and backward, and various visualization options.play_picks - instantiates imucam::PickPlayer to replay picks that were saved to a text file.calibrate_frames - instantiates imucam::Reader and imucam::Calibrator to estimate the spatial rotation from the UM6 coordinate frame to the depth camera coordinate frame by solving the orthogonal Procrustes problem to align pairs of gravity vectors from the UM6 and from the depth camera. The latter are estimated as the doward facing normal of the dominant plane in the scene, which should be a flat horizontal surfce.irtool - view and save IR images from the depth camera, possibly useful for camera calibration.All utilties show command line option help with -h and interactive help by hitting h in a GUI window or in the terminal.
common, io, visualization and their dependencies
filters features segmentation surface. The calibration binaries are built by default but not included in the library (make WITH_CALIBRATOR=1 to change that) to limit the number of PCL modules upon which the library depends.Run make to build everything. Option VERBOSE=1 shows the commands being run. Option WITH_CALIBRATOR=1 includes the calibration code in the library.
Run make libs|bins|docs to build just the libraries, binaries, or documentation. Run make foo.bin to build just the target binary foo, e.g. make record_frames.bin.
pclzf2tum and tum2pclzf utilitiesin imucam::Reader
in imucam::Grabber
in imucam::UM6
in imucam::Viewer
in imucam::Writer
in imucam::Calibrator
This material is based upon work supported by the National Science Foundation under Grant No. 1149235. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
THIS INFORMATION AND/OR SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS INFORMATION AND/OR SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
