Rethinking GUI Programming Paradigms

I have to say the current state of programming and application development do bother me. I feel after so many years of programming, making things like desktop applications and web applications should be a lot easier.  Over the last few years I have done a lot of application development in a bunch of technologies like Java, SWT, Flash, HTML/JS and more recently Objective C. And yet our development environments (or IDEs) are pretty much the same as 20 years ago with vi or emacs. I have a lot of friends who have really good ideas and yet they are pretty daunted by the idea that they could build their applications themselves, and I dont blame them. Most of the advice programmers give them is to read a couple of books and download Textmate.

Their have been a lot of “GUI builder” applications over the years and pretty much all of them have been panned by “real” developers. The usual (and often justified) reason has been that they generate crappy code. Its shocking to me why GUI development remains such an afterthought in todays world.

Did Flash get it right?

Like it or hate it, Flash was probably one of the first tools that got it somewhat right. Its success and now the reason it gets panned, is because it made it very approachable for someone who was not a programmer to get something in their mind out on the internet. The ability to draw a shape and then attach some behavior to it in code was huge.

This was my path to it too. I came to the US persuing my graduate degree in VLSI Engineering (chip design). Somehow I got a job at a University Learning Center with a very loose mandate: the director of the center had a lot of ideas on e-learning with video and had just seen some Flash video sites and was fascinated (remember Flash video only came around in version 6 of Flash). My role was to “make something cool for e-learning and video”. I had never programmed before but had some experience with Flash animation, mostly making small animated movies hoping to get on Atom Films someday.

In about 6 months I had gotten really addicted to Flash since for the first time, my animations could respond to the user. Because of Flash, I quit VLSI and moved to Software Engineering, with my final thesis in creating cross language bridges between GUI and non GUI parts of an applicaiton (using Flash and Java). The approachability of Flash back then was my gateway drug to programming. Flash has since “matured” to a real programming language but I feel at the cost of the approachability the earlier versions had.

Ugh Java

Moving to the Software Engineering team was a bizarre experience. The Graduate School was a Java school and I got started working in Swing. It was crazy! I could do any kind of UI in Flash in a heartbeat but getting Java to render a custom designed button was virtually impossible (so we always stuck to the default lookAndFeel). Actually back in the day, I don’t think there was even a Java IDE like Netbeans that we used. Everything was written in VI and then compiled by using the command line.

Interface Builder: A good idea that was never completed ?

Working with Interface Builder, the tool used to develop User Interfaces for Mac apps,  for a while, I see the thought process that must have been going on when it was being developed. The concept of laying out the UI and tying it to the application logic components is a good one. But IB feels like the team was disbanded halfway into the project. The tool as it stands right now is such a pain to understand that most new developers who get into Objective C programming try to avoid it as much as they can.

The 280 Atlas IDE for Cappuccino seems to have taken the IB concept and extended it to the web. I like this better than IB if for nothing else than gathering all application windows in one. The floating canvases on IB get annoying really fast.

Drag and drop application development:

I love this video. I am sure to scale this to “real” GUI development would be a significant amount of work, but that would be the holy grail (at least for me)

3d physics simulation with Wiimote from Balazs Serenyi on Vimeo.

The same idea is also echoed in MIT’s Scratch project. Scratch declares its mandate as making it easy to “create your own interactive stories, animations, games, music, and art”. While the focus seems to be towards animation and getting kids to learn programming concepts, can it be extended to real world projects ?

Another interesting project that surfaced recently was Sikuli. Check the video on that below. My favorite part of this was that the “script” actually references the images that behavior is assigned to. You dont need to understand variables at this point (You may need to to write business logic if you have to, but thats a bridge you can cross later)

Human-readable “code” with Ruby and Cucumber:

Just getting started with the Ruby programming language, I have been really impressed by its focus on readable code. An open source project called Cucumber is also fascinating. I haven’t done any work with it but have attended a few sessions on it in different user group meetings. Cucumber takes the readability of code even further with a goal of having non technical folk be able to define a site’s functionality very granularly.

Final thoughts: Enough with the text editor already

Using a pure fast text editor for developing applications seems to be the badge of a real developer. But writing lines of  text to express ideas in my head seems to be a paradigm that should have evolved a while back but hasn’t. Am I overly simplifying the problem? Possibly. But after so many decades since the inception of programming there is very little to justify the high barrier to entry it presents to most newbies. To me the current state of programming feels like the state of mobile phones before the iphone and the ipad. Whatever anyone’s feeling on Apple are, its hard to deny how much they humanized computing.

Its time to do that for programming too don’t you think?

Some other related interesting stuff

  • The Adobe design team recently posted some thoughts on improving the programming experience in Flash.
  • Some good thoughts on XCode UI enhancements

Author: Arpit Mathur

Arpit Mathur is a Principal Engineer at Comcast Labs where he is currently working on a variety of topics including Machine Learning, Affective Computing, and Blockchain applications. Arpit has also worked extensively on Android and iOS applications, Virtual Reality apps as well as with web technologies like JavaScript, HTML and Ruby on Rails. He also spent a couple of years in the User Experience team as a Creative Technologist.

13 thoughts on “Rethinking GUI Programming Paradigms”

  1. I don’t have nearly the breadth of experience with other languages and IDEs that you do but I’d agree that the environment for creating custom UIs is primitive compared to the advances in other areas. I have the feeling that we are on the verge of some radical changes in UI development but that could just be wishful thinking on my part.

    If you have a second check out – just watch the short intro video. That is something I came up with after building my 50th custom video player.


  2. 1 for making application developments more powerful.

    I often think programmers are usually so busy making new and better products that they don’t get enough time to focus their skills on their own environments and processes. Or even help teach new developers.

    Not sure how to solve that, but I guess the first step is awareness, so nice work there, buddy!


  3. Our product, Storyboard Suite, tries to bridge the gap between graphic designers (who want to drive a products user interface and behaviour) and embedded engineers (who often just want to worry about the guts of the system and not the look of the data).

    It is specifically targeted at embedded products and can simulate an application in a manner similar to the way Flash is used to build a demo and decouples user interface and system logic with an MVC model and Lua scripting.

    You can download a free evaluation and try it out.



  4. I like that programming a good user experience is rather hard. I’ve worked hard to get very good at it and now, finally, get paid for all of my efforts. If it became easy . . . well I would kind of be out of a job. For that same reason I don’t want a nifty do-it-all machine that builds amazing furniture for anybody with an idea. I want a gifted craftsman to build something special with his hands . . . and get paid accordingly. The reason I love programming is because it is a craft and while every good craftsman needs good tools nothing beats hard work and experience.

    I guess that even if it becomes easy to build “good” user experiences there will always be a place for us who craft “great” user experiences.


  5. I’ve been using REBOL for years. It’s practically unknown, but I’ve never found any other language or development tool so well conceived.


  6. Pingback: paradox1x

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: