DSpace@MIT

Comparison of Self-Localization Methods Continued ================================================= This archive contains data logs and evaluation tools, Dieter Fox and me used for performing the self-locaization experiments reported in our paper at IROS 2002. Data Log Files -------------- The archive contains one directory 'dlogs' with the following data log files: o normal/dlog.dat (base log file), o sparse.*/dlog.dat (log files under different level of sparseness), o noise.*/dlog.dat (log files under different level of outliers), and o displace/dlog.dat (log file with parts removed to simulate kid-napping). These log files were generated from the same base log file. See our paper [Gutmann and Fox, IROS 2002] for understanding what has been done to the indiviual logs. The format of each dlog.dat file is as follows. Each line contains a position estimate from odometry and observations obs: frame x y theta num ( id1:id2 range bearing ) ... where: - frame is a timestamp (1 sec = 125 frames), - x, y, theta is the robot pose estimated by odometry and measured in mm and deg, - num is the number of landmarks seen, - id1:id2 gives the id of the landmark seen, - range and bearing are from the current robot pose (kinematic chain of head is already included) and are measured in mm and deg. The reference point on the robot is at the neck joint connecting robot body and head. You also need the following information about the landmark positions: cyan:magenta -1500 -1000 magenta:cyan -1500 1000 magenta:green 0 -1000 green:magenta 0 1000 yellow:magenta 1500 -1000 magenta:yellow 1500 1000 and 0 -> green 1 -> magenta 2 -> yellow 3 -> blue The robot started in the center of the field facing in positive x direction. It then cycled in an 8-shaped path through the following positions: (500 -500), (500, 0), (0, 0), (-1000, 0), (-1000, -500). At each of these positions the operator pressed a button on which a 'mark' was written to the data log. Potentially there are errors in the ground truth (joysticking the robot exactly on a spot is difficult, observing it is exactly on a spot contains errors, and there can be a short time delay until the mark is written to the log). See fieldSetup.gif for a visualization of landmark and marker positions. Evaluation ---------- Basically, you can completely decide by your own how to evaluate your results. Here is how we did it. If your localization program outputs the pose of the robot at each mark in the following format: < method-name > x y th where x, y are in mm and the in deg, then you can use the accuracy.sh script for computing the distance to the ground truth locations and the mean_confidence program for obtaining mean and confidence of your estimates. E.g. for MLEKF we used: accuracy.sh MLEKF < pose.log | mean_confidence Evaluation for kid-napping is a bit different. We used the output of one localization method as a reference path when processing the base log file (e.g. we used the SRL output but you are welcome to provide your own if you feel your results on the base log file are better). Your localization program should then output the robot pose at each time step in the following format: x[< frame >]=< x > y[< frame >]=< y > th[< frame >]=< th > for example: x[30048]=-102.502917; y[30048]=-4.598352; th[30048]=-105.912977; Your localization program should als copy the 'mark' lines to this output. You can then use the 'recoverTime' utility for computing the number of seconds your method needs for recovering from kid-napping: recoverTime ../reference-log/pose.log.SRL < allpose.log | mean_confidence You find our reference pose log, scripts and utils in the evaluation folder. Papers ------ J.-S. Gutmann, W. Burgard, D. Fox, and K. Konolige. An Experimental Comparison of Localization Methods, International Conference on Intelligent Robots and Systems (IROS'98), Victoria, Canada, October 1998. J.-S. Gutmann and D. Fox, An Experimental Comparison of Localization Methods Continued, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'02), Lausanne, Switzerland, October 2002. S. Kristensen and P. Jensfelt. An Experimental Comparison of Localisation Methods, the MHL Sessions, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'03), 2003. Good luck! Steffen Gutmann, 6.5.2004