Back to Homepage / Tutorial

State Machines in SwingStates


SwingStates uses Java inner classes to offer a natural syntax to program state machines. State machines can typically be used to program interaction. SwingStates proposes several classes of state machine including ones to handle Java mouse and keyboard events. Combining several state machines provide a rich control structure and allows to minimize the total number of states.


General introduction to SwingStates state machines

State machine: definitions and principle

State machines are used to easily describe interaction. A state machine consists of a set of states and a set of transitions labeled with events. Each transition connects an input state to an output state. Conversely, each state has a set of output transitions and a set of input transitions.

Only one state is active at a time, named the active state. Initially, the active state is the initial state. When an event that matches one of the output transitions of the active state, this transition is fired making the active state being updated.

Each transition can be associated to:

Each state can be associated to:

State machine: algorithm

A state machine works as follows:

(*) So the declaration order of output transitions of a state matters. Typically, transitions must be sorted from the most specific to the least specific.

A state machine can be represented graphically. It is a labeled directed graph. States are ellipses and transitions are arrows oriented from their input state to their output state. A transition is labeled by the event that triggers it.

Let us consider the state machine that describes the interaction to move a graphical shape by a mouse drag initiated on this shape and to move all shapes by a mouse drag initiated on the background. This machine has two states:

The following applet allows to test this interaction and to see the underlying state machine running in real time.

Failed to load applet. No Java 2 SDK, Standard Edition v 1.5.0 support for applet.

smClasses.jpg

See applet sources.

State machine: programming

In order to use a state machine in a program, we must be able to describe it and provide a way to implement the code to make it run. One solution consists in translating the graphical representation in a Java program but this solution requires having a hybrid editor to draw the state machine and to write the code implementing actions. Another solution, easier to implement, consists in describing it in a textual format and translating the description in Java. For the state machine of the last example, this textual form could be:

    
    State start {
    
        Transition Press Button1 on a shape s  => drag {
            lastPoint <- cursor.location
            toMove <- {s}
        }

        Transition Press Button1 on background  => drag {
            lastPoint <- cursor.location
            toMove <- {canvas.shapes}
        }
        
    }

    State drag {

        enter {
            highlight toMove
        }

        Transition Drag  => drag {
            translate toMove from lastPoint to cursor.location
        }

        Transition Release  => start

        leave {
            unhighlight toMove
        }

    }
    

Thanks to Java inner classes, implementing a state machine with SwingStates is very similar to this textual format:

The code of the state machine seen above is:

    // CStateMachine is a subclass of StateMachine that provides 
    // events to monitor a Canvas widget 
    sm = new CStateMachine(canvas) {
			
        CElement toMove = null;
        Point2D lastPoint = null;
			
        public State start = new State() {
				
            Transition moveShape = new PressOnShape(BUTTON1, ">> drag") {
                public void action() {
                    toMove = getShape();
                    lastPoint = getPoint();
                }
            };
				
            Transition moveAll = new Press(BUTTON1, ">> drag") {
                public void action() {
                    toMove = canvas;
                    lastPoint = getPoint();
                }
            };
				
        };
			
        public State drag = new State() {

            public void enter() {
                toMove.setOutlinePaint(Color.RED);
            }
				
            Transition stop = new Release(BUTTON1, ">> start") {
                public void action(){
                    toMove.translateBy(getPoint().getX() - lastPoint.getX(), getPoint().getY() - lastPoint.getY());
                }
            };
				
            Transition move = new Drag(BUTTON1) {
                public void action() {
                    toMove.translateBy(getPoint().getX() - lastPoint.getX(), getPoint().getY() - lastPoint.getY());
                    lastPoint = getPoint();
                }
            };
				
            public void leave() {
                toMove.setOutlinePaint(Color.BLACK);
            }
        };
			
    };	

Let us take a closer look:

    sm = new CStateMachine(canvas) {
        // declarations of the local fields and methods of the machine
        ...

        public State start = new State() {
            // output transitions of the state ''start''
            ...
        };
        
        public State drag = new State("drag") {
            // output transitions of the state ''drag''
            ...
        };
    };

Using anonymous classes allows to declare the states of the machine inside the declaration of the machine itself. The same principle is used to declare transitions inside a state:

    public State start = new State() {
        // declaration of the transition
        Transition moveShape = new PressOnShape(BUTTON1, ">> drag") {
            ...
        };

        // declaration of the transition
        Transition moveAll = new Press(BUTTON1, ">> drag") {
            ...
        };
    };

Let us now explain in further details declaration of transitions. For instance:

    Transition moveShape = new PressOnShape(BUTTON1, ">> drag") {
        public void action() {
            toMove = getShape();
            lastPoint = getPoint();
        }
    };


State machines available in SwingStates

SwingStates contains four state machine classes:

The top class, StateMachine, provides only three classes of transition: Event, to handle only high-level virtual events and TimeOut/TaggedTimeOut events to handle events fired when a timer expires.

BasicInputStateMachine is the direct subclass of StateMachine. It contains transitions to handle mouse events and key events (e.g. Click, KeyPress, etc.).

BasicInputStateMachine has itself two subclasses: CStateMachine and JStateMachine to handle a large set of transitions respectively on the SwingStates' Canvas widget and on existing Swing widgets (subclasses of JComponent).

smClasses.jpg

Debugging state machines

Graphical

SwingStates allows to visualize state machines while they are running by using a StateMachineVisualization object.

StateMachineVisualization smv = new StateMachineVisualization(sm);

A StateMachineVisualization is a subclass of JComponent so it can be packed as any other Swing widget.

Textual

SwingStates also proposes a textual form for debugging state machines. StateMachineEventListener interface and StateMachineEventAdapter abstract class work as any Java listener. Any object listener implementing StateMachineEventListener interface can be attached to a state machine sm using the method sm.addStateMachineListener(listener). The listener can then define callbacks which will be invoked when an internal change occurs in the machine: when a transition is triggered in the machine, when the machine is suspended, resumed, reset or inited. StateMachineEventAdapter abstract class implements StateMachineEventListener and define empty callbacks so one can override only the methods he is interested in.

Example using StateMachineEventListener:

StateMachine sm;

...

sm.addStateMachineListener(new StateMachineEventListener(){
    public void smStateChanged(StateMachineEvent e){
        System.out.println("State changed from "+e.getPreviousState().getName()+" to "+e.getCurrentState().getName()+"\n");
    }
    
    public void smStateLooped(StateMachineEvent e) {
        System.out.println("State looped on \n"+e.getCurrentState().getName());
    }

    public void smSuspended(StateMachineEvent e) {
        System.out.println("State machine suspended\n"+e);
    }

    public void smResumed(StateMachineEvent e) {
        System.out.println("State machine resumed\n"+e);
    }

    public void smReset(StateMachineEvent e) {
        System.out.println("State machine reset\n"+e);
    }

    public void smInited(StateMachineEvent e) {
        System.out.println("State machine inited\n"+e);
    }

});

Example using StateMachineEventAdapter:


StateMachine sm;

...

sm.addStateMachineListener(new StateMachineEventAdapter() {
    public void smStateChanged(StateMachineEvent e) {
        System.out.println("State changed from "+e.getPreviousState().getName()+" to "+e.getCurrentState().getName()+"\n");
    }
});

Communication between machines

Communication between state machines is illustrated all along this tutorial. In this section, we expose guidelines for using several state machines and refer/complete other examples of this tutorial.

State machines in parallel

Key point: avoid the problem of explosion the number of states in a machine.

The simple way of using several state machines is to make them run in parallel. Using a parallel is recommended to implement only one time a behavior that is available in many states. Consider the example with circular and linear menus an interface that uses both a linear menu and a circular menu, and that highlights the background of the menu items when the mouse cursor is over it. Rather than trying to combine all three interactions into a single state machine, it is much easier to use three state machines. Two state machines implement menu selection for respectively linear menu and marking menu on the right, while a third state machine implements highlighting of any menu item.

Dialog between machines

Key point: modular programming of interaction.

Between machines in general

A state machine can fire an event to a set of listeners (fr.lri.swingstates.sm.StateMachineListener) by calling the method fireEvent(EventObject). Firing an event for a machine means invoking eventOccured(EventObject) method of all its attached listeners (StateMachineListener). Attaching a listener to a machine sm is done using method addStateMachineListener(StateMachineListener).

In particular, a state machine itself is a StateMachineListener so one state machine can trigger transitions of another machine. The instruction below makes a state machine sm2 listen events incoming from a state machine sm1.

sm1.addStateMachineListener(sm2);

Thus, when sm1 fires an event:

sm1.fireEvent(new VirtualEvent("anEvent"));

This event event triggers the following transition in sm2:

sm2 = new StateMachine() {
    public State start = new State() {
        Transition t = new Event("anEvent") {
            public void action() { ... }
        };
    };
};

Note that, for any event of type VirtualEvent, we can use denote this event in a transition using the event key string, i.e. "anEvent" in the example above. In general, an event is a java.util.EventObject and transitions take the class of event as parameter. In the following example, sm2 listens custom color events of type ColorEvent:

// class of custom color events
class ColorEvent extends EventObject {

	Color color;
	
	public ColorEvent(StateMachine sm, Color c) {
		super(c);
		color = c;
	}
	
	public Color getColor() {
		return color;
	}
	
}
...
sm2 = new StateMachine() {
    public State start = new State() {
        // This transition is triggered by any event which is a ColorEvent
        Transition t = new Event(ColorEvent.class) {
            public void action() { ... }
        };
    };
};

We use this mechanism to make smMenu and smWidgets communicate in the pie menu example of this example. smWidgets is attached as a listener of smMenu:

smMenu.addStateMachineListener(smWidgets);

smMenu fires "cancel" and color events when a menu item has been selected:

smMenu =  new CStateMachine() {
    
    ...
    
    public State menuOn = new State() {
    
        ...
        
        Transition select = new Release(BUTTON3, ">> start") {
            public void action() {
                // smMenu fires color and cancel events
                if(color != null) fireEvent(new ColorEvent(smMenu, color));
                else fireEvent(new VirtualEvent("cancel"));
            }
        };
        
    };

};

These events trigger transitions in smWidgets:

smWidgets = new JStateMachine() {

    ...
    
    // listen for cancel and color events incoming from smMenu
    public State componentSelected = new State() {
    
        Transition select = new Event("cancel", ">> start") { };
        Transition color = new Event(ColorEvent.class, ">> start") {
            public void action() {
                selected.setBackground(((ColorEvent)getEvent()).color);
            }
        };
				
    };
};

Between machines attached to a common canvas

Suppose we want to propose several interactions for activating commands of a graphical editor to create new shapes, copy, cut and paste them. We want to be able activing it by using menus of this example or by using gestures of the classifier defined in this example. We represent the graphical editor as a canvas associated with a state machine that handles the following high-level events: new(Point), cut(CShape), copy(CShape) and paste(Point).

CStateMachine smEdition = new CStateMachine() {

    public State idle = new State() {
        
        Transition newShape = new EventOnPosition("new") {
            public void action() {
                newShape(getPoint());
            }
        };
        Transition copy = new EventOnShape("copy") {
            public void action() {
                copy(getShape());
            }
        };
        
        Transition cut = new EventOnShape("cut"){
            public void action() {
                cut(getShape());
            }
        };
        
        Transition paste = new EventOnPosition("paste"){
            public void action() {
                paste(getPoint());
            }
        };
    };
};

smEdition.attachTo(canvas);	

These high-level events are fired on the canvas by the state machines that implement the different interaction techniques (menu linear/circular and gestures). For example, when the state machine that implements gesture recognition has recognized a given class of gestures ("new", "cut", "copy" or "paste"), it processes an event on the canvas on the gesture location:

// for gesture machine
String recognizedClass = classifier.classify(gesture);
canvas.processEvent(recognizedClass, pInit);

In a similar way, machines that implement item selection for both fire events corresponding to the selected item:

// for menu machines (circular and linear)
String item = ((MenuItem)getTag()).getName();
canvas.processEvent(item, pInit);

It can be useful to handle richer events than a string on a location. The class VirtualCanvasEvent serves this need. For example, if we want to fire events on a canvas that carry a color to apply. We can create our class of color events by extending VirtualCanvasEvent:

public class ColorEvent extends VirtualCanvasEvent {
    Color color;
    public ColorEvent(Point2D pt, Color c) {
		super(pt);
		color = c;
	}
}

When a canvas receives an event of type VirtualCanvasEvent, it picks the shape under the location of this event so the following instruction

canvas.fireEvent(new ColorEvent(new Point2D.Double(25, 50), Color.BLUE));

can trigger transitions EventOnShape(ColorEvent.class) (or EventOnTag(ColorEvent.class)). For example, the event that has been fired will color the shape under location (25, 50) in blue thanks to the following machine:

CStateMachine colorShapes = new CStateMachine(canvas) {
    State start = new State("start") {
        Transition color = new EventOnShape(ColorEvent.class) {
            public void action(){ 
                getShape().setFillPaint(((ColorEvent)getEvent()).getColor());
            }
        };
    };
};
...
colorShapes.attachTo(canvas);

Suspend/Resume a machine

Key point: improve performance by making a state machine active only when necessary.

In the previous example, we used a modal interaction since we used the left mouse button for both gesture recognition and item selection in the circular menu. Changing mode is achieved by pressing the Shift key. To implement this modal interaction, we have simply used the following machine. It contains two states, one per mode, and activates the right machine (gesture or circular menu) according to its current state. Its current state changes each time the Shift key is pressed:

CStateMachine modes = new CStateMachine(editor) {
    public State markingMenuMode = new State() {
        Transition gesture = new KeyPress(KeyEvent.VK_SHIFT, ">> gestureMode") {
            public void action() {
                markingMenuStateMachine.suspend();
                gestureStateMachine.reset().resume();
            }
        };
    };
    public State gestureMode = new State() {
        Transition markingMenu = new KeyPress(KeyEvent.VK_SHIFT, ">> markingMenuMode") {
            public void action() {
                gestureStateMachine.suspend();
                markingMenuStateMachine.reset().resume();
            }
        };
    };
};

Failed to load applet. No Java 2 SDK, Standard Edition v 1.5.0 support for applet.

smClasses.jpg

See applet sources.