Back to Homepage / Tutorial

Training application for gesture recognition

To use gesture recognition, you must create your classifier that contains the set of gesture classes for your application. Launch Training application (in package fr.lri.swingstates.gestures) using command:

   java -classpath SwingStates.jar fr.lri.swingstates.gestures.Training

Training application.

It contains five main panels:

• Top panel is dedicated to change distance thresholds used for rejecting gesture errors. See below.
• Middle left panel is used to enter names of gesture classes. To create a new class, type a class name in top left text entry field and click 'Add' button or hit 'Enter' key. To delete a class, select it and click 'Remove' button. To rename a class, right click on it to invoke a contextual menu.
• Middle center panel is used to enter gesture examples in a class of gestures. To enter examples for a gesture class, select this class in Left panel and draw examples in the training area.

  Input an example in a gesture class.

• Middle right panel is used to test the recognizer. To test your recognizer, draw a gesture in the testing area. Once mouse button is released a window pops up to display the recognition result for both algorithms:
  • the name of the recognized class for this gesture,
  • the distance between this gesture and the closest class. This distance value is used to reject errors, i.e., gestures that are too different from the gestures used for training. Within a classifier, if the distance for a given gesture is greater than its distance threshold, the classify method returns null. Note that these distances are absolute indicators that can be compared within an algorithm but that can not be compared between algorithms. For Rubine's algorithm, it is the distance between the feature vector of this gesture and the average feature vector of the recognized class. For $1, it is the distance "point to point" between this gesture and the best candidate example in the recognized class.
  • the complete list of gesture classes sorted from the best candidate to the worst candidate to classify this gesture.
  Accuracy for recognition for these algorithms can vary according to vocabulary properties. Comparing their results helps to choose the best recognizer for a given vocabulary. To use Rubine's recognizer in a program, use method newClassifier in fr.lri.swingstates.gestures.rubine. To use $1 recognizer in a program, use method newClassifier in fr.lri.swingstates.gestures.dollar1. In both cases, distance threshold for rejection can be set using method setMaximumDistance.

  Input a gesture to recognize.	Recognition results.

• Bottom panel is used to show examples already input in a class. To delete an example, click on it and hit 'Backspace' key.

Once a classifier is created, it can be saved. (item save in menu File...). Note that You can also load an existing classifier and modify it. (item open in menu File...). To test this application on an existing classifier, you can download the classifier used in this example.