PlayBook: Top-Swipe Menu


Posted:   |  More posts about PlayBook AIR

Warning: the approach described here for SWIPE_START was carefully designed based on the observed performance of the simulator as it was at the time. Unfortunately, at least one app built around this approach does not work correctly on the "1.0" software running on the real hardware. If you've used this code in your app, you may encounter the same issue, in which case for now, switching to the more primitive SWIPE_DOWN mechanism.

In the BlackBerry PlayBook Developer Forum (Tablet OS SDK for Adobe AIR) there have been several threads about the SWIPE_START event and SWIPE_DOWN event and how to use them to implement an application menu. None of these have yet given a complete picture of how those events really work. But don't worry, I've analyzed it all for you!

SWIPE_DOWN Event

I'll cover this one first, because with it you can implement a simple "top-swipe" app menu with only a few lines of code.

SWIPE_DOWN is one of two events dispatched by the QNXApplication.qnxApplication singleton instance to signal user swiping activity in the top bezel area.

To top-swipe, the user begins swiping in the top bezel in the appropriate region. This region lies between X (horizontal) coordinates 50 and 975, leaving a 50-pixel-wide margin at either side for the top-left and top-right swipes (see image). These "corner" swipes are used to "peek" at the system status bar (as long as you hold your finger down) or to expose it completely if you raise your finger. (Note how the status bar "pushes" the app stage down, rather than sliding over.)

/static/files/top_swipe_regions.jpg

If the user continues swiping down far enough into the screen area (apparently to a Y value of around 21) then the SWIPE_DOWN event is issued immediately. That's all there is to it.

Given that your app has no advance knowledge of the imminent arrival of SWIPE_DOWN, it can't do anything before receiving it. Upon receipt, it could either tween a menu panel down into place, or even just make it appear instantly (e.g. with addChild()). That's about as simple as you can get. The following few lines add this to any app, as a starting point. Imports are left out to simplify the example.

QNXApplication.qnxApplication.addEventListener(QNXApplicationEvent.SWIPE_DOWN, onSwipeDown);
...

private const MENU_HEIGHT:int = 50;
private function onSwipeDown(e:Event):void {
    if (!menu) {
        menu = new Sprite();
        menu.graphics.beginFill(0x654321);
        menu.graphics.drawRect(0, 0, stage.stageWidth, MENU_HEIGHT);
        menu.addEventListener(MouseEvent.CLICK, closeMenu);
    }
    addChild(menu);
    // include the next two lines to animate the menu opening
    menu.y = -MENU_HEIGHT;
    Tweener.addTween(menu, {y: 0, time: 1});
}

private var menu:Sprite;
private function closeMenu(e:Event):void {
    removeChild(menu);
}

Note, however, how the animation completes regardless of what you do. It doesn't "track" the position of your finger, and you can't change your mind and return to the top: the menu will open no matter what. This approach is simple, but a bit crude.

SWIPE_START Event

In contrast to SWIPE_DOWN, there's not much you can do with SWIPE_START that doesn't involve a lot more lines of code. SWIPE_START is sent immediately whenever your finger touches the top bezel area in the top-swipe region. There are several interesting behaviours associated with it.

One is that you don't actually have to respond to any MouseEvents or touch events... you can watch for SWIPE_START and take some action based on that, even opening a menu as shown above for SWIPE_DOWN. That's neat, but possibly unfriendly to users who are used to the more conventional swiping behaviour.

If you want to respond to the actual swiping gesture you'll need to monitor several MouseEvents (on the stage), which is the reason this one is more complex than SWIPE_DOWN. The second interesting thing about SWIPE_START is that if you are listening for it, and the user's finger moves (as opposed to just being raised again), you will receive MouseEvents associated with the top-swipe. You won't see these MouseEvents if you don't listen for SWIPE_START.

But wait, there's more! You'll get the MouseEvents even if the swiping finger stays in the bezel area, and even if it moves across the top and down along the sides to the bottom! That should suggest a few interesting (but again, unconventional and possibly annoying) ways of responding to user actions.

The actual sequence of events for a real swipe will be this:

  1. QNXApplicationEvent.SWIPE_START [also known as swipeStart]
  2. MouseEvent.MOVE (a single one) [mouseMove]
  3. MouseEvent.DOWN [mouseDown]
  4. MouseEvent.MOVE (zero or more)
  5. MouseEvent.UP [mouseUp]
  6. MouseEvent.CLICK [click]

The mouseDown event will be seen immediately after the first mouseMove. If the finger keeps moving, you'll get lots more mouseMove events, followed eventually by a mouseUp and click (even if the finger never leaves the bezel region).

If the user simply touches and lifts the finger without moving it, you'll see just this:

  1. QNXApplicationEvent.SWIPE_START [swipeStart]
  2. MouseEvent.MOVE (a single one) [mouseMove]

You don't need to handle all the events shown, but you'll likely want at least the mouseDown event along with either mouseMove or an enterFrame handler, followed by the mouseUp event. (That's assuming you are still interested in implementing a conventional menu, and not heading right off to play with odd top-bezel gestures like a kid with a new toy.)

It's your choice which event to use to track the mouse motion, but you'll get way more mouseMove events than you need, so to reduce CPU usage you're probably better off with an enterFrame handler for the tracking.

The reason you should listen for the mouseDown event is to avoid taking action when the user merely brushes the bezel ("touch and release"). A naive implementation would use swipeStart to set up the position tracking, but then you'd do the wrong thing if the user tapped the bezel, then started a real swipe gesture inside the screen area.

Faking a Top-Swipe Gesture

Actually, you may have noticed there's a loophole in RIM's current design. If you touch-and-release in the bezel, generating a swipeStart then mouseMove, then touch and drag inside the screen, generating a mouseDown and more mouseMove events, the sequence is the same as a real top-swipe. (If you try this with a mouse, you'll see more mouseMove events as soon as the pointer moves into the screen area, but that's cheating. Without a mouse, there's no pointer and no mouseMove events unless you're touching the screen.)

The problem with ignoring this is that the user could abort a top-swipe but you'd respond to the next actual drag inside the app stage as if the user were still swiping. My original top-swipe implementation in the support forums suffers from this problem. Try it: click above the app (but don't swipe), then click and drag inside the screen area... you should see the menu open even though this isn't a valid top-swipe gesture.

There are probably several workarounds. The simplest would be to check whether the mouseDown event has coordinates very close to the swipeStart event, cancelling the top-swipe if it does not. The problem with this is that the coordinates reported for stage.mouseX (and mouseY) seem to correspond to the last-seen position (from a previous MouseEvent), not where the finger actually is at swipeStart.

A more complex approach involves listening for mouseMove following swipeStart, and noting the coordinates. Then filter the next mouseDown event and check that it's coordinates are the same. (I assume they should be exactly the same, especially as the mouseDown seems to be generated, not the real one that presumably triggered the swipeStart.) If the next mouse event is not mouseDown, or if the mouseDown position is different, then the user cancelled the swipe and is doing a non-top-swipe somewhere, even if it's right at the edge of the screen.

The Implementation

The code for this is a bit long, so I'm not including it directly here. If you want to peek at it, look at ca.microcode.menu.TopSwipeMenu and the demo program that uses it in MenuTest. (By putting it there, I can also make changes more easily, and you can submit issues to the tracker, if you like.)

The code implements a 7-state finite state machine that cleanly steps through the various stages possible. The design ensures that only the necessary listeners are active at any given time, so you don't wastefully watch for (especially) mouseMove or enterFrame events when it's not required. See the figure below.

/static/files/TopSwipeStates.png

As implemented, it requires you to specify a width and height for the menu, and optionally a background colour for the panel. You can adjust the "swipe_region" (probably not a good term for it, but it defines how far you have to swipe into the screen before you can release and have the menu continue opening on its own) and "slide_time" (duration in seconds for the complete menu slide) as class properties.

To close the menu you can either click below it, add a Done button as in the MenuTest example code, or touch the top bezel again.

I hope it proves useful to someone. It's taken quite a few hours to get it to this stage; I would appreciate a small credit somewhere in your app or docs if you choose to use it. At least let me know where it's been used, for my own satisfaction. (Legally you don't have to do either, as I've released it under the MIT License, but you must preserve my copyright notice and the license even if you change the code.)

Addendum

I wrote a follow-up with a suggestion for a simplified version that doesn't require listening for mouseMove at all.

Comments powered by Disqus