Category: Air

Air for mobile’s weird touch implementation

Let me begin by saying that after playing with Adobe AIR for mobile for the last couple of weeks, I have been really pleasantly surprised by general performance. I have done quite a bit of mobile device development off late with both iOS and Android, but AIR might actually be a contender for my next project.

That said, I am kinda surprised by the touch event implementation. I am hoping someone will correct me if I am missing something but here is what I am seeing so far.

In AIR, you can now choose to set a Multitouch Input mode to either intercept raw touch events (MultitouchInputMode.Touch_Point) or have the touch events mimic mouse events and get separate gesture events from the flash player when gestures are performed (MultitouchInputMode.Gesture). Note: when gestures begin, you stop getting mouse events till the gesture has completed.

My first problem is that you can pick one or the other, but not both. The former will give you raw touch events and so you can see when more than one touch is on the stage, but then you have to write your own code to define what a gesture is. The latter gives you gestures but you’ll never know when more than one finger are on the stage. But what if you want to track touch points independently till a gesture begins? You are on your own there.

Additionally, There is no information on touch positions on the Gesture event. There is a localX and localY which I presume is the point between the 2 touch events, but they don’t change much as the gesture occurs (I was trying to read these values while panning halfway into pinching, the change in values isnt representative of how much my fingers moved) which seem to be the x and y points of where the gesture began and do not change as the gesture events change.

Also, Gestures like Zoom and Pan do work independently but not together. So If you are zooming (pinching) into an image using 2 fingers and then start moving the 2 fingers in a particular direction without changing the distance between the touch points (Pan), you dont get the pan gesture events. This is unlike the behavior on most apps that allow zoom and pan.

At this point I started looking to read raw touch point data. Here is another implementation gap. AIR requires the developer to keep track of every touch point (identified by toucheventIds). There is no data model that you can query on the AIR Player that gives you an Array of touchPoints. This is irritating and smells of poor API design decisions. Compare this with iOS’s api for touches where I get the set of all touch objects every time the time touches begin or change:


- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event;

Anyway, thats as far as I have gotten so far. Maybe I am missing something but it seems that if you are really building something significant with AIR for mobile, you might need a custom non-Adobe gestures library.

Anyone know of a good one?

AIR Utility code: Window Management for applications that run in the background

This is something I thought would take me a couple of hours to implement for the application I am working on but ended up taking way too long validating behavior and the gotchas between the PC and Mac OS’s so its definitely good utility code to keep in your toolbox. The behavior I was trying to mimic was that of apps that run in the background when the main window is closed and then can be re-invoked by clicking on the icon in the dock on the Mac or the system tray in Windows.

First the API, which I am pretty proud of. To get the above behavior, once you have my class included in the classpath, all you need to do is:


AirUtils.runInBackgroundOnClose(window)

Doesn’t get much simpler than that ;). Optionally you can pass arguments to handle the invoke interaction and conditionally cancel it, an array of Bitmaps to choose from for the icon in the dock or system tray and a NativeMenu object that displays the menu that displays when the user right clicks on the icon. If the Array of Bitmaps is not passed in as a parameter, the code parses the applicationDescriptor XML file and uses the icons defined there (why doesn’t AIR do this by default?)

You can check the code at my AIRUtils Github repo.

Some important points to note:

1) InvokeEvent Gotcha
There were a few gotchas in the implementation. My first pass was something that looked like this:

For some bizarre reason this didnt work at all and debugging it showed that the InvokeEvent was being called multiple times. Turns out the InvokeEvent is unlike any other Event as (as per the docs): “When an event listener registers for an invoke event, it will also receive all invoke events that occurred before the registration”. The fix for that was to check NativeWindow visibility when activating the NativeWindow.

2) Overriding NativeWindow’s Close Event:
Another gotcha here: I wanted to call preventDefault() on the NativeWindow’s Close event, but that meant there was no way to exit the application on a Mac. Why? Because calling NativeApplication.exit called the close action on the NativeWindow which was being preventDefault-ed. So the app would just never exit. To fix this, I had to check if the NativeApplication was exiting when the close event was called and if so, not prevent its default behavior.

3) Loading the icon image from the appDescriptor.xml:
All the icon used in the dock/system tray is pulled from the appDescriptor.xml. The parsing/loading and capturing Bitmap of the icon is probably more work than just using a bitmap embedded in the swf but it made for a cleaner API. Another gotcha when implementing this was that using Loader.load() with a File’s nativePath worked on Windows but not on a Mac. Apparently the right way to load a file into a loader from the filesystem is by using the file’s url property and not the nativePath.

My FlashCampPhilly presentation: Some interesting libraries to consider for your next app

Last Saturday, Philadelphia was host to its first ever FlashCamp event. The event brought together some of the biggest names in Flash and Flex in the neighborhood and beyond (special kudos to Rob Hall for organizing the event from vision to a lot of very hard work). I did have one session at the event as well, as I walked through some of the libraries I have started using in a lot of my projects, including the soon to be alpha-ed Espresso Reader. These libraries included:

Is CSS the best metaphor for styling for a UIToolkit? Maybe not

Writing your own UI Toolkit is, if nothing else, educational. As the OpenPyro framework continues to evolve, every time a new feature is added, I look around at prior art in the domain and evaluate the choices made by different frameworks which has been fascinating. Of course, the first framework I look at (since I have had most experience with that) has always been Flex (both Flex 3 and 4 separatly since they are so different), but often I do compare that with choices made by the IPhone’s UIKit, HTML/CSS and (rarely) Swing or any of the Java toolkits.

One of the features that I know I will need to add in a future version will be a simpler way to style/skin the controls. OpenPyro controls are based around composition so creating new UI assets and passing them in as skin elements is simple enough but there is always a need to do something simpler. A blue scrollbar for example shouldn’t need blue assets, more the ability to tell the scrollbar to draw itself with a stroke/fill of blue color.

Flex’s choice in this domain has been CSS and thats not surprising. A lot of the way Flex was developed was to enable it to be a gateway drug to ActionScript for developers from other technology domains and who doesn’t know CSS? Flex 3’s CSS capabilities were very basic and I keep reading how Flex 4’s CSS engine is so much more complete. And considering how much easier it is to create a Flex 3 theme compared to a Look and Feel in Swing, that seems to have been a decent choice. Someone recently asked when OpenPyro would support that functionality which led to this post.

I have often found writing CSS for Flex a chore, for most of the reasons I find writing CSS rules boring: complete loss of any logic system. Deep in ActionScript land while developing an application, switching to CSS is irritating. Suddenly I am in a file where I cannot really code any logic or rules. One of the first libraries of ActionScript I wrote over 3 years back had explicitly defined style objects. For example, ScrollBars would have a corresponding ScrollBarStyle class that listed all style properties. Being a pure AS class, I could also code logic in that, so conditionally apply style properties because I could inspect the properties of the UI element the style was being applied to. Heck, I could even get code hinting of relevant styles for a particular component.

What spurred this post today was seeing how the engineering team I work with at Comcast Interactive Media adopting the Less CSS engine. I have personally not worked with this yet but I might in a coming sprint. By accounts so far, its awesome to work with and I completely don’t doubt that. Most of the logic here seems something that you can write pretty easily in pure AS3 classes and pass them around as something equivalent to Flex/Javascript’s CSSStyleDeclaration object (link to JS CSSStyleDeclaration object)

Anyway, the point of this post was, while CSS may be the most familiar styling metaphor, there may be better ones. I hope OpenPyro can probably adopt something more powerful like lessCSS rather than pure CSS.

Update: Another flavor of a *better CSS* system: SASS

Update 2: Smashing Magazine has a pretty good writeup on CSS ideas and debates. More food for thought as I spec out a styling language for OpenPyro.

When things go wrong: Communicating error and gathering information for client side code

Having developed client side applications for the last few years in a variety of languages (Java, Flash, AIR, Javascript and recent forays into Objective C for the IPhone), its funny that some discussions appear cyclically and are discussed as though for the first time in every new project. Communicating errors in your application seems to be one of them, and I wanted to pen a few thoughts on that.

Serverside developers have it easy!
Well, no ;). But there are paradigms that have become the norm now. If something bad happens and you get a 404 or a 500 error on a webpage, its pretty likely that its been logged and if its on a major product, alarms have been raised. Client side code, especially in the languages I have worked in the last couple of years, have been less…communicative. I would like to focus on AIR and Flash here since those are the technologies I work with most often. The problem becomes more critical in AIR where you have much more things that can go wrong since you are interacting with items on the user’s machine which is not a controlled environment.

Why communicate the error?
Very few client side web apps I have worked with, including my own, have ever seemed to have handled error correctly. Most of the time, the app has been scoped with time allocated against the features and error states have never been considered. Occasionally, a developer will put in an Alert window or something similar just as a precaution (I know I have). In Flash/AIR there doesnt seem to be a good library that I can integrate into my application that encapsulates best practicess. Heck, I dont even know what the best practices are, which is what prompted me to write this post.

What would you include in the error window?
While you may want to include app specific information in the error window, I think there are a few things that make most sense:

  • An error description: What happened ? Corrupt file? Cannot connect to server? Something readable. For bonus points, can the user do anything about this, like restart the application?
  • An error code: This makes sense and if you include a support system, like a bug tracker or a 1-800 number, it would make the conversation a lot more meaningful
  • The detailed stack trace: A good paradigm I have seen here is a toggleable details state that shows/hides the detailed error stack trace, would go a long way in figuring out the source of the error
  • Extra points: email API integration, or some other way for the user to send the error description with as little pain as possible

The last point is kind of interesting, cause you might want to send more than just the stack trace of the error. You might want to set the whole application state. If you have application logging enabled, say with something like the CIM logging framework, the whole log file might be of interest. Again, the developer must ensure that he is only logging non-personal information there.

Of course there are situations when the error might cause the entire app to blow up, without giving it enough time to react to the error. Would it make sense to log and end of session event and allow users to send the application log to the server if the end of session was not successfully logged for a prior session?

Prior Art:

I started looking at some Google results for Error dialogs and crash reporters. Here are some screenshots:

First is the JXErrorPane, a Java class part of SwingX project. This seems to be the most familiar UI for reporting errors(link to API docs)

jxerrorpane

The usage is pretty simple:

try {
//do stuff.... something throws an exception in here
} catch (Exception e) {
JXErrorPane.showDialog(e);
}

A step up would be the UI from crash reporters which would perhaps be overkill for a web app, but would make a lot of sense for desktop apps, especially in the beta stages. Here is a screenshot from something called Bug Buddy.

Bug Buddy

Whats kind of interesting here is that it adds a field that allows user to add information about the context of the crash, as well as an email address that may be used to contact him or something.

The last screenshot is from an open source library called JRFeedbackProvider someone presented at the Philadelphia Cocoa Users Group.

JRFeedbackProvider. This has the features of the previous crash reporters but also adds a “request a feature” tab which is great if you are running an alpha version of your application and are looking for more than bug reports.

If you have seen any other error UI that seem to do a good job, please add a comment here.

An app for myself: Facebook EventSync

Off late I have become pretty involved in a local (Philadelphia) user group called RefreshPhilly. The group meets the first Monday of every month and hosts a couple of talks by some local new-media professionals, designers or developers. The event is hosted at the new Comcast building where I work.
However like a lot of big corporate buildings, the Comcast building only allows registered guests into the building. This meant that all attendees had to be registered with building security to allow them to enter. Since our attendee list often reaches 100 odd people, I needed to get the names to security in chunks, so that I didnt have to get all of them at the last second. Unfortunately the Facebook events page only shows the total number of people who have confirmed their attendence. What I really needed was “How many new people have confirmed since the last time I looked”. So I whipped this little app that uses the Facebook API to take snapshots of the attendee lists from any Facebook event (using a valid event id) at any time and show the number of new people who have confirmed attending or maybe attending status by comparing the snapshot to the previous snapshot saved on a local file.

Facebook Event Sync

I am almost sure this app will only ever have one user (me) but hey, if you ever need it, you can download it from here. Feel free to comment if you would like to see any additional functionality here, or if you have an idea for this being a more practical app 🙂

LogBook and Application Performance

Its exciting to see LogBook, Comcast Interactive Media’s first open source project get some traction. There is talk of LogBook being included in other open source projects as well as people have started submitting patches and suggestions to the project.

One issue that was submitted a few days back was around the Logging implementation dispatching all logging events out of the application across the LocalConnection. Here is a quote from the issue:

Logging could add significant overhead to an application, due to
log message construction, and sending messages to LogBook.

It is good practice to minimize logging overhead, with code like:

if (Log.isInfo()) {
LOG.info(“version: {0}”, version);
}

The current implementation of class LocalConnectionTarget always
sets the log level to ALL and the log filter to “*”. This means
that all logging methods are executed, which is undesirable.

You can see the details of the issue here.

I just wanted to elaborate a little on why the LocalConnectionTarget class does not use any kind of filtering to reduce the number of LogMessages being constructed.

One of the reason we went with LogBook as an AIR application as opposed to a web application was because as far as I know, the Flash Player process on the web is independent from that in AIR. This means that LogBook could become a very elaborate application on its own but it will not impact the performance of the application using LogBook based logging. Also when we pass objects across LocalConnection, these objects do not accumulate in the application’s memory. Here is a screenshot from a quick experiment where I ran an endless loop that created a custom LogDispatcher object, had it send a log message across the LocalConnection and then get garbage collected. The memory profile of the application can be seen below:

LogBook App Memory Profile

Here is also the live objects screen capture:

LogBook Enabled App Live Objects Screen Grab

As you can see, over 600 objects were constructed but all of them could be garbage collected with no overhead in terms of memory used.

So your take away from this discussion is: using LogBook has virtually no memory overhead in your application. The idea of broadcasting all messages to LogBook is to allow you to filter the data on the client at the moment of debugging or after, rather than configuring it at some pre-launch time.

Now LogBook itself is another story. Another bug submitted by the same developer who reported this also talked about LogBook’s performance when over 1000 messages were collected, and yes, I have verified that at that point, the application becomes pretty sluggish. So for the next version of the LogBook, I am considering leveraging the SQLite database that AIR comes packaged with.

More on LogBook 2.0 soon. We are already on it and it will be good ;). If you would like to participate in defining the functionality in 2.0 or better yet, help us develop it, join the conversation on the LogBook user group here.

Released: LogBook, An Open Source AIR based Flex Logging Application

126399-logbook_1_4_0_1_alpha_development_other

As promised, today we are formally releasing LogBook, our internally developed tool for logging events from Flex (and coming very soon, pure Flash AS2/AS3) applications. For the uninitiated, logging is the process of recording and storing information about events in your application. The functionality is similar to the trace command, but a lot more robust. Logging combined with a strong log viewer like LogBook gives you a much deeper understanding of the state of your application.

Some quick notes on LogBook:

1) LogBook is being released under the MIT open source license.
2) A detailed post on its features and how to use it is here at the CIM Flash Team Blog.
3) The code for LogBook is being hosted at http://cimlogbook.googlecode.com/.
4) We have also set up a Google group for logbook that we can use to discuss features that we should add in upcoming releases and bugfixes/patches.
5) The installer can be found in the downloads section there.
6) LogBook runs on the AIR beta 3 runtime.

At CIM we already have a list of features we are going to release in the next build. But of course if you have any feature you would like to request, please drop us a mail at cimlogbook@googlegroups.com. We are really hoping LogBook 1.2 is the first step towards a robust Flex/Flash development/debugging toolset.

Happy debugging :).

Releasing tomorrow: LogBook, an open source AIR based Flex logging application

The team at Comcast Interactive Media is putting the final touches to an internally developed tool we use for logging events from Flex applications. The application is built on top of the Flex framework and runs as an installable AIR application. Best of all, we will be releasing it under the MIT open source license. A much more detailed blog entry about logbook’s features will be published soon but here is a quick screengrab of the application (click to open in a new window):

Stay tuned ;).

Update: LogBook has been released and can now be downloaded at http://code.google.com/p/cimlogbook/

[Video] DiggGraphr Desktop 1.0: My entry for the Air Derby

DiggGraphr is finally mature enough for me to give it 1.0 status. I finally got around to implementing the features I have been meaning to for a while, including auto update and inline browsing using the HTML control in AIR. The code for it wasnt difficult but the change in user experience is pretty phenomenal. You can read more about it and grab the latest version from http://www.arpitonline.com/blog/?page_id=63.

I captured a quick video of the application in action so take a look. Its grainy but you get the idea. You’ll have to download the app to check it out.