Planet Maemo: category "feed:0d3ed10193588edfe41285c9c902da78"

Portable apps for iPhone, Android, Pre, n900 - Can Processing.js help?

Tue, 09 Mar 2010 02:02:00 +0000

I don't know. But if you want to find out, then I have something for you.

Processing.alt

Visit this website from your desktop browser. You will see an IDE. Write some processing code to create a sketch and save it when you are done.

Visit the same website from your iPhone/Android/n900/Pre browser. You will get a simple viewer interface. Find your sketch under My Sketchbook and run it.

Processing is a well known language to create sketches, animations, games. It is a simplified dialect of java. The resulting sketches are in the form of bytecode and can be run on desktop or in java applets. In Summer of 2008, John Resig (jQuery fame) ported Processing to javascript. Processing.js uses the HTML5 canvas element to render the sketches.

The modern browsers that ship with today's smart phones support HTML5 element. So it is very much plausible to get your Processing sketch/game working on all the modern smart phones without any knowledge of native app development for each platform. Processing.js has a potential to become an uber-portable library for mobile development.

That was the idea I started out with about a month ago. After several iterations, I ended up with a website that you see now. It is an IDE when opened from desktop browser. You can write Processing code, run it, save it, share it for others to see in the Gallery. There are also some samples you can find in Gallery right now. Try them out.

The same website acts as a viewer for your processing sketch when opened from a mobile phone. You can view the shared sketches from the Gallery or you can login and find your saved sketches in My Sketchbook.

One great advantage of Processing.js is, it can intermix javascript with Processing code. So in the samples you will find how javascript is used to automatically find the kind of browser and adjust the dimensions of the sketch accordingly.

I have tested the site on Android HTC Magic and Nokia n900. A friend told me it worked as expected on his iPhone too. Also I am treating Palm Pre's user agent as mobile platform. So on these four platforms you should see the mobile interface of the website. But in case if you don't, let me know and I'll see what is wrong.

As you will see that running Processing code on smartphones is indeed possible, but is fairly sluggish. However I believe the future is not far when the phones will become more powerful, browsers will become more optimized and Processing.js can also be tuned for speed. We already have two gadgets with 1GHz processors - Nexus One and iPad.

The IDE also has a lot of scope for innovation. But before jumping ahead of myself, I decided to open it as beta. I would like to hear your thoughts and suggestions on this idea.

So try it out and leave your opinion in the comments.

P.S. For logging in to save your sketches, use your GMail account. You don't need to login to browse through the Gallery.

Ads:
* Processing: A Programming Handbook for Visual Designers and Artists
* Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction (Morgan Kaufmann Series in Computer Graphics)
* Processing: Creative Coding and Computational Art (Foundation)6 0

Frets on Fire for Maemo - Update

Wed, 28 Oct 2009 11:05:00 +0000

Back in March, when Fremantle SDK was released, I had ported Frets on Fire to it. Subsequent discussion made it clear that modifications were required to actually make it run on the device. I created a stripped down version which could run without the OpenGL code. This made it suitable for n900 (and even n810). But due to lack of actual device, I couldn't test it.

Fast forward 7 months. Now I have access to actual n900, so recently I went back to the FoF code. After couple weeks of getting re-acquainted with the code, I have it satisfactorily running on the n900.

Frets-On-Fire code primarily has three parts -

playing the song (loading song, computing notes, etc. uses pygame for audio),
showing the visuals (PyOpenGL+pygame code) and
handling input.

All these components are pretty involved and hard to rewrite - especially so with the audio part (I hardly have any experience in musical notes and midi files). So my priority was to get almost all of the audio code as it is. It was easy to tweak the input code - to make it suite the n900 keyboard. I bypassed the visual part completely - replaced OpenGL with simple pygame sprites. (You may get disappointed by the stark looks).

I have created a project on googlecode to host the code - maemofof. It shows all the changes I had to make, starting from the base Frets-on-Fire code from v1.3.110.

The next step is to package it nicely and push to extras-devel. There are few problems with packaging though.

I couldn't get ogg files to play through pygame. I installed the ogg-support package, but for pygame/sdl to work with it, something more is required - which wouldn't install due to broken dependencies (libsdl-mixeroggwav1.2-dev). So for now I converted all ogg files to wav. But the wav files are huge (~80MB if a single song is included as sample). That makes them inconvenient to package and they also take a long time to load (~90 seconds).

But good news is, the ball got rolling again. I hope to make some progress in coming days. If you have any suggestions, feel free to chime in comments.8 0

FoF - PyOpenGL = miniFoF (possibly for n810)

Tue, 24 Mar 2009 11:23:00 +0000

After spending some time on the Frets-on-Fire code I have managed to liberate it from PyOpenGL dependency.

In first step, I bypassed all the GUI code, without making any changes to the audio and input processing components. So I ended up with a completely blank window with a default song playing and me pressing the keyboard frets blindly.

In second step, I wrote only few lines of code to draw the notes on the screen using basic pygame.draw functions.

Here is a screen shot of the minimal UI.

Then I realized that, in theory this minimal version can also run on n810. I tested it in Diablo SDK and it runs. I couldn't hear the sound, but that's an issue with my scratchbox settings. I am sure that this will run on n810, however I am not sure if it will have real time response times. Unfortunately, I don't have a n810 to try this on (mine must be changing hands in black market as I write this :( ). Nevertheless I made necessary changes so that someone can try it on real n810 if they wish to.

If you have played FoF on keyboard, you would know F1-F5 keys are used to control frets and RETURN key is used to pluck the string. One possible way to achieve this with n810 is to use the keys on the keypad.

Following mockup should give you an idea.

F1 = Quote, F2 = K, F3 = H, F4 = F, F5 = S

In place of RETURN we can use the square key as shown above, but I couldn't find what its keycode is. So for now I have assigned that functionality to Q. If someone can provide with that keycode in pygame, I will fix it.

The gameplay is for single song only. I have bypassed all the menus for now. As the song starts the goal is to hit the note when it reaches the white line. You can play different songs by specifying their name as --play command line option.

This is not (yet) a single click install app, however.

I have put together the steps that you will need to follow, to get it working on actual n810. Those should work to the best of my knowledge, but you may run into errors. If you do, please report them in comments and I can help with them.

If you are interested in the changes I had to make, then here is the list of changes starting from the baseline FoF code (r1165 - r1176)

Give it a try. If it runs slow, let me know how slow. There still must be bunch of code paths that we can eliminate to improve the response times.10 0

FoF update

Wed, 18 Mar 2009 22:16:00 +0000

After the Frets-on-Fire-on-Fremantle post, the feasibility of it on real actual hardware was discussed on maemo mailing list and in comments. It became clear that it's not sufficient to get FoF running in SDK to prove that it will also run on final hardware. We will have to work on the OpenGL GUI to make it happen.

So I dived into the internals of FoF. And this is what I've come up with so far. (Click on image for detailed view)

It is far from a formal UML diagram, but it tries to capture the elaborate architecture of FoF code.

I am experimenting with some parts of the above diagram. Let's see how it goes.

That's all for now... BTW, I posted inkface v0.2.3 yesterday (highlights - all tests running in Diablo SDK, basic Clutter support). Check the detailed changelog here.9 0

Twitter client with inkface-pygame v0.2.2

Thu, 12 Mar 2009 10:00:00 +0000

Here is an update on the Inkface library. But before that let me give a background of the project for the benefit of new readers that will be reading this post via Planet Maemo.

Inkface is an SVG based GUI framework. Unlike the desktops - where GUI components need to be keyboard/mouse friendly; the handheld GUIs need to be finger friendly. Therefore the handheld GUI components should be naturally manipulatable - like parts of an image. Therefore inkface provides a framework in which, GUI is composed in an image editor like Inkscape, instead of rigidly coded in the program. The various elements of SVG image are presented as python objects to the programmer who can then write his program logic using these elements as widgets.

The current version of inkface uses pycairo for vector graphics rendering and uses pygame as backend surface to draw on. A clutter backend is in the plans.

With that background, let me show a demo of an app that I designed (in Inkscape) and coded (in python) using inkface-pygame library v0.2.2. It's a twitter client. The demo shows how the GUI can be changed vastly by merely changing the SVG files and doing no change in the code at all. (the --theme option tells the app to just use a different set of SVG files) The first one is the default theme and the later has a vintage look.

The whole GUI consists of only 2 SVG images (corresponding to 2 screens - login and main twits page). So to create a new theme one only needs to create/change these two SVG files in Inkscape. Compare this to the traditional approach where a theme consists of tens of PNG images of specific sizes.

I was aiming to release it as an app for diablo, but I had to postpone the plan. For improving the performance during animation, I used pygame's features that are only available in v1.8.x and I later found that Diablo ships with pygame v1.7.x. So I need to work around this incompatibility before I release it for n8x0 devices.

For more information on the project check out the wiki. You can download v0.2.2 tarball from here. It is pure python code and can be tried on desktop. For details on the changelog of v0.2.2 check my post on the mailing list here.10 0

Frets on Fire on Maemo 5 (Fremantle)

Wed, 04 Mar 2009 05:33:00 +0000

Couple days ago I played Frets on Fire (FoF) for the first time. I was totally hooked on to it. After going through its hilarious tutorial and starting with a pathetic performance, I managed to score 37K points with 87% accuracy on the "Defy the Machine" song a few moments ago.

Couple days ago Nokia released the alpha version of Maemo 5 SDK (Fremantle) too.

I woke up next morning with an inevitable thought of porting the former on the later.

After a day's work here is a video demo of "Frets on Fire" running in the Maemo scratchbox SDK. And find below a list of things I had to do to get it working.

[If you don't know about FoF, it's an open source Guitar Hero clone that runs on many desktop platforms. It is written completely in python. While playing it you hold your keyboard in an unusual fashion so that it simulates a rockstar's guitar. (wikipedia)]

In future we can tweak FoF's UI so that the strings could be pressed against the frets using the touch screen itself.

There is no sound in the video, but it's just because I couldn't channel the sound through two layers of virtualization and into the video recorder software. I could hear the songs alright though after transporting them over wire using pulseaudio.

[Note that scratchbox is running on Ubuntu which itself is a KVM virtual machine. That should put the poor framerate into perspective.]

Here are the things I had to do:

Fremantle SDK doesn't have python distutils. In diablo it comes from extras repo, so I borrowed it from there as well. (Added diablo extras repo in /etc/apt/sources.list)
Fremantle SDK has a bug that I had discussed on mailing list during pre-alpha release, due to which the SDL/pygame applications cannot use the OpenGL support of the SDK. Due to same bug, FoF would also not work. After some googling I figured out the root cause and a work around. I built a private version of libsdl with --enable-video-opengl option. That did the trick. I have filed a bug to get this fixed in Fremantle.
I also had to build custom packages of PIL (python imaging library) and numpy. I believe these libraries are available in diablo through extras repository, so I guess they will also be available in fremantle in future.
Next I had to modify src/Input.py to comment out pygame's joystick initialization code.
The really tough part was getting sound working. I found an old post to get sound working in scratchbox. Although it was a correct procedure, it didn't work for me. After a while I remembered from the Maemo summit that Fremantle runs pulseaudio (as opposed to esound in diablo). After that I did some googling and figured how to transport the sound over network using pulseaudio server and client. Here are couple useful links. [1] [2]
The job on sound wasn't over yet. FoF runs all its music from ogg files. pygame won't play ogg files on fremantle. I dabbled into recompiling SDL_mixer with ogg support, but that didn't help. Then another discussion from Maemo summit came to mind, and I remembered for some (non-technical ?!) reason Maemo doesn't play ogg. So to get around it, I converted all .ogg files into .wav using ffmpeg. Changed the code wherever necessary to replace the file name extensions from ogg to wav. That did the trick and finally I could hear the sound.
The game by default used 600x480 resolution. After some analysis of the code, I found fretsonfire.ini file in my home directory. I changed it to set the resolution to 800x480 to fit into the Xephyr window.
There is one thing that I couldn't get to work. Arrow keys won't work. I found that the scan codes sent through Xephyr to the pygame were different from what are put in pygame constants.

... so when Nokia releases that next device, we will have an electric guitar in our pocket. :)12 0

Inkface+pygame v0.2.0

Wed, 25 Feb 2009 07:23:00 +0000

As per the plan for v0.2, I am ready with a working implementation of inkface. I also made few more design decisions after my last design post. Here are the highlights:

1. All code is in python
2. For X11 surfaces, pygame is used.

I found that pygame (python bindings for SDL) is a mature library for drawing on X11 surfaces. It also has some optimizations which help in 2D animations. There is also OpenGL support, but I do not plan to use it inside inkface library. It is also well supported on Maemo.

I have written my favorite twitter client app with the new inkface-pygame library. The following video demo shows, how I could quickly code 2D scrolling animation with the help of pygame.

The v0.2.0 is delivered as a tarball. I have also updated the inkface wiki with some getting-started tips.0 0

Smooth scrolling with Inkface+pygame

Sun, 22 Feb 2009 08:02:00 +0000

Preview of what's coming out later in the week. A v0.2 drop of inkface-pygame package for maemo and others should be ready in coming few days.

0 0

Inkface v0.1.2, Twitter-inkface client

Sat, 10 Jan 2009 01:25:00 +0000

Finally it is ready. Actually it has been ready for more than couple of weeks now, but I wanted to make sure the bugs in my demo app were not due to problems in core library.

The demo app is a client I wrote for the Twitter service. The intention was to demonstrate how an intuitive GUI can be designed using an Image editor. With the GUI designed using Inkscape and an off-the-shelf python library to talk to Twitter service (Thanks to DeWitt for python-twitter), I only had to write simple glue code to get the whole app working. Check out the video.

Good news is I have setup a debian repo to host packages for this app.

In the Maemo Application Manager, add following Catalog:

Name: Altcanvas
Web Address: http://repo.altcanvas.com
Distribution: testing
Components: main

Refresh the application list. You should see 'twitter-inkface' app in the list. All the dependencies will be automatically pulled from the repo.

Since the repository implementation is currently experimental (No it's not a standard web server! I will save it for a separate post), you may face a problem while installing or I may have to take it offline for modifications. In that case, you can download following four packages and install them with dpkg ("dpkg -i *.deb")

libaltsvg,inkface-python, inklib, twitter-inkface.

For more info on Inkface project, visit the project page.0 0

Inkface - SVG based GUI design

Sun, 07 Sep 2008 22:50:00 +0000

I have always wondered why are there so many advanced technologies to learn to create the GUI for a software application. GUI is what we see, then if we want to design or customize it, then it should be as simple as editing an image in an Image editor. This becomes more important with the next-gen handheld devices, where the GUIs will be handled more intuitively - by fingers - rather than keyboards or mice. However the problem and its solution wasn't very apparent to me.

... until I read this post by Andre Schmidt on Clutter mailing list few months back. In his simsui project, he has designed a slider widget from an SVG image and used clutter to render its differnt parts. This little project defined the problem and solution I was looking for. I played with the simsui source code. Unfortunately, that code is written for very specific widget and had some duplicate implementation of some rendering logic.

So I decided to give a try to a generic SVG (vector graphics) based GUI framework. I looked for open source SVG rendering libraries. I chose librsvg (which is now part of the Gnome desktop) for this purpose. It took me a month to understand its internals and to extend them to fulfill my needs. I eventually got it working. I now have created a fork of that library - dubbed 'libaltsvg' - under my altcanvas project.

What I have achieved so far is complete isolation of GUI from the application logic code. I have coded two samples apps - A music player and A virtual keyboard for handhelds - using this new GUI framework. As you can see in the following demo video, the GUI look-n-fill can be changed by merely changing the input SVG filename. In this framework the app logic gets simplified as well. The code for the two apps can be found in player.c and keyboard.c.

high-res version on Blip.tv

The complete source code can be found here in altcanvas repository.

The demo above is prepared on my desktop, however I have tested these on my n810 as well. Of course, the animation effects are too much for it to handle. But it works decent at minimum frame rates. I will post packages for n810 diablo in next couple of days.

To put this in perspective: this framework lets you do on your desktop/mobile, what Adobe flash lets you do on the web (and on mobiles too) - with an added benefit of coding in C (and python or other popular languages, once I get their bindings ready). There are some closed source projects for doing vector graphics based GUIs for mobile devices (ikivo, bitflash). But I haven't found any open source initiatives so far. Let me know for sure if you have.

If you are attending OSiM or Maemo summit next week in Berlin, then look for me. I will be happy to give a demo.

Update (Sep 12):
A package for trying the Inkface demo on Nokia n810 is available now. Download the .deb file.
[Note that the keyboard/music player are not fully functional apps. They are proof-of-concept apps for showing Inkface GUI framework benefits.]
I will be flying tommorrow afternoon for Germany. If any of you guys are in OSiM or Maemo summit, I will be glad to meet. After I come back from the trip, I will work on further improvements to this project. Till then... have fun!!!0 0

Flickr uploads from n810

Tue, 22 Apr 2008 00:36:00 +0000

Finally I am done with new version of publishr for Nokia n810. Check out the demo video.

You can install from here. Check out the project page.0 0

OSiM USA 2008 - Day 2

Thu, 13 Mar 2008 22:47:00 +0000

The second day of the conference was awesome.

As I said at the end of my last post, I was expecting more interesting sessions on second day. But I had even more fun than expected. Most enjoyable sessions to me were the 2 demos - one by Soonr and another by Matthew from OpenedHand.

I asked many presenters about what sort of business model do they envision for open source application developers, and I didn't get any conclusive answer from them. However I found partial answer to my question in the demo from Soonr (and also from the talk with funambol yesterday). In short, Soonr allows users to save their data (docs, contacts, etc.) in the cloud (an account with them) and lets them access it using their cell phone or desktops. It customizes this data to suite the browser's capabilities (downsize powerpoint slides etc.). Basically it has the same benefits as having mobile versions of all Google/Yahoo services. However Soonr makes the process seamless. I liked the idea of how they handle the stolen cell phone use-case. If the user looses his cell phone, he can configure his account to disallow access from that cell phone.

I particularly liked the OpenedHand presentation because it was the only hardcore technical presentation in the entire conference. Matthew Allum (CEO, founder of OpenedHand), demonstrated the Clutter GUI his company is working on. The presentation was impressive. I have almost never attended a presentation where the slide show hasn't been done using PowerPoint. I suppose I might have seen OpenOffice presentations, but those were clearly not flashy. Matthew however gave presentation on his Ubuntu laptop and his slides had more fascinating effects than any PowerPoint slides I had ever seen. When I asked him at the end of the session about which software he used, he said that it's one his own little scripts. I know now where to look for, while preparing my next presentation... Back to "Clutter". If the phone has hardware support for graphics acceleration then Clutter gives all the snappy effects that you see in modern iPhone-like phones. See it for yourself. I have dowloaded and built the Clutter from o-hand's svn repository and played with many examples myself. It's very well structured source code and easy to build. Matthew's demo was done on an iPod-touch, but I heard it will also be able to run in upcoming GTA02 version of Openmoko phone.

Among all the people I met at conference, I enjoyed talking to Quim Gil the most. He is the product manager of Maemo project and I have read him on mailing lists from time to time. The discussions we had were very educational for me. I had many questions on Maemo (some of them naive and impractical), but Quim was nice enough to answer all of them with enthusiasm. I am a software engineer by profession and have worked with bunch of smart people; but never have I had such a frank discussion with as senior a person as a product manager. I think that is why we call it "Open" source. Thanks to Quim, Matthew and Hal (Funambol), I left the conference with an inspired spirit.

I was very excited about hearing Sean Moss-Pultz, but I got little disappointed when he couldn't make it to the conference. However Michael Shiloh gave very informative session on Openmoko. I got to handle the GTA02 that he passed among the audience. It was nice to see it, before I actually buy it in next few months.

There were couple of panel discussions on how web 2.0 is different from or similar to mobile 2.0. Among a lot of arguments and opinions I found Fabrizio's (Funambol CEO) comment very curious. It's well known that mobile browsers are lagging in AJAX and flash support as compared to desktop, which impedes the access to web 2.0 from mobile phones. With iPhone's browser interface and n810's microb engine that goal is looking within reach. However according to Fabrizio, mobile 2.0 will be made up with native apps connected to the web, rather than a fancy browser. I have always liked this idea personally even on desktops. But only time will tell how the web will unfold on the phone in our hands.

Ari Jaaski's presentation in the morning was as per expectations. Carolyn Lewko did very good job of hosting today's sessions. Andreas Constantinou also had many insightful comments during various sessions. Dan Morril's Android presentation was also nice. I would have liked to ask him about difficulties in porting android to other platforms because of the closed source binaries at the core of android, but then I changed my mind. My hunch is, Google will at least provide pre-compiled binaries for various platforms in future, even if they won't open source everything. Let's see. I hope someday I will be able to run android on OpenMoko and Nokia tablets.

Overall, this was a successful beginning of the US chapter of OSiM. Congratulations to the organizers. I hope there will be increased participation from the developer community next time. After all, the Open source phenomenon is nothing without the Geeks in the Garage. If big corporations are to involve more in open source, they should get to meet face to face with the real innovators. Conferences like this can play key role in that.0 0