With a direct flight from London (LHR) to Seoul (ICN), I was lucky enough to fly on Korean Air. Only Quatar Airways comes close in terms of service, though I still like their aubergine-colored uniforms more. Incheon Airport won a "best airport" award several times in a row, and I think it's well-earned. It starts with the gorgeous view when flying in, but the best argument going for the airport is probably A'REX, an affordable, fast and reliable railway line that connects Icheon with the most important spots in the city. Taxi drivers will try to convince you otherwise but don't be fooled, there's no competition to that line.

I stayed near Hongik University station, which is serviced by A'REX. It's a great location, especially at night. Tristan, our famous GTK+/Glade (and now also EDS) hacker, decided to stay nearby so we could go out for drinks and cure my jetlag with great food and a hangover. The hangover didn't happen (too much food?) and neither did the jetlag. As I arrived a couple of days early, I had enough time to discover the city a bit and prepare for the conference (read: slides and stuff). My talk about Wayland input methods seemed to be well received. It lead to a couple of good discussions about text input in general.

The venue was difficult to find in this maze of office buildings. It was good to see old & new friends at the summit though I had hoped for a lot more attendees. Perhaps (local?) advertising the event at universites would have helped. At least in my perception, the target audience for F/OSS development have always been students with enough spare time on their hands to try out stupid things (such as writing free software). I have come to prefer 3-day conferences over 2-day conferences. Two days just doesn't allow for enough hallway discussions as your time to attend talks is very limited. Spreading talks out over three days gives me a better opportunity to balance between talks and hallway track. I wish for instance that FOSDEM wasn't crammed into two days. The other extreme are probably Akademy and GUADEC that go over a week if you count BoFs and workshops as part of the confernence. But I am pretty sure that two days are too short ;-)

Thanks to the GNOME Foundation for 1) accepting my membership \o/ 2) approving my travel sponsorship. This would not have happened without my talk being accepted, which as I understand from talking to others who submitted session proposals was a quite lucky circumstance on its own.

2 0

This is a feature that I personally wanted for a long time. Interrupted by FOSDEM, some Wayland research and many other things, I finally managed to get word prediction and error correction beyond prototype quality. The video shows just how amazingly good the Presage word prediction can be, even without extensive training (in fact, for the video we used the minimal language model training that comes with a regular Presage installation). The second part of the video shows how combining Presage with a spellchecker such as Hunspell further improves the provided word candidates.

Presage uses a very scalable approach called text n-grams. There is a lot of research in that area, but language models of contemporary language usage are either well guarded or cannot be freely distributed. Luckily, Presage comes with training tools such as text2ngram. Users can feed arbitrary language corpora to it, though one should be careful to perhaps not mix different languages too much.

Matteo Vescovi, the author of Presage (formerly known as Soothsayer), started the work as part of his master thesis a couple of years ago. The heart of Presage are the different predictors. They can be queried in parallel and the result lists are merged, using probability analysis.

Presage certainly has a lot of potential. It comes with an easy to use C++ API but also provides bindings for C and Python. In fact, it even provides a D-Bus API, which would make it possible to run it as a system service. The user could then benefit from (and train!) the same language models from different applications.

Hunspell probably doesn’t need much introduction. It is used in many Linux desktops. It’s a fine library and comes with many dictionaries. However, one should be aware that Hunspell itself cannot provide word prediction, which is why it wasn’t enough for mobile text input. As a fallback for Presage, it works very well though.

Of course there are a couple of things we could do from here. As Jon mentioned in the video, the virtual keyboard’s word ribbon UI could host word suggestions from other applications, such as the Google search in the browser. For Unity’s dasher input or Gnome Shell’s search, the application names could be shown instead. Or we could hook it up to Bash completion.

PS: Anyone up to package Presage for Fremantle or Nemo? It could be pretty interesting to see the next release of Maliit running on the N900.

Permalink | Leave a comment  »

5 1

There is seemingly a trend (in ideas) to use your mobile device as a remote control for stationary devices such as your TV.

We thought that experimental support for remote text input should be easy to add to Maliit, as it already comes with a server-client architecture. As soon as client and server can run on different host machines, we’d have network transparency and it should basically just work. Luckily, we use D-Bus as our IPC, and it turns out that D-Bus understands remote host addresses. If you check the README in maliit-framework you will recognize that – with the latest release (0.90.0) – it only takes an additional environment variable, MALIIT_SERVER_ADDRESS, to connect a client (that is, an application that uses the Maliit input context) to the input method server. The scary part is perhaps that one has to disable D-Bus authentication. Therefore it's a good idea to only use this inside a trusted network.

Jon created a video which demonstrates the new feature in a better way, using a laptop that is connected to a projector as a TV replacement. The great thing about this feature is that it requires no changes to the applications, nor to the Maliit input method plug-ins, it’s the framework that handles this internally (or not so internally, now that Qt 5 turned our D-Bus API into a public one).

However, to make Maliit truly useful as a remote control, at least two additional features are required:

• Have a proxy text editor widget on mobile device that allows text interactions such as copy and paste, cursor positioning and rich text formatting.
• Emulate and transmit mouse or touch events, possibly through a dedicated touch area on the touch screen of the mobile device.

The second feature requires an extension to the Maliit protocol, whereas the first is already filed as bug MALIIT#84. After that, I could use my N9 (or my N900, as Maliit also runs there) to remote control my Gnome or Ubuntu Unity desktop, which would work perfectly for my movie watching habits. Kudos goes to Krzesimir and Jon for their work on this feature.

Permalink | Leave a comment  »

3 1

It was nice to see Jarno and Esko at the conference, too. We even stayed in the same hotel. I hope they enjoyed the Ethiopian lunch as much as I did. And perhaps they're not too angry any more that we lead them to drink Absinthe ;-)

Jon and I gave two talks. Jon's talk (slides) was about Maliit as a project, explaining what Maliit is (and what it is not), combined with a short history lesson about the project. I tried to outline the difficulties of mobile text input in general (slides), picking some use-cases that are known from the desktop world and showing why simply copying the use-cases and their known interaction models does not work very well. I honestly liked Jon's talk more though.

Neither of us two actually managed to visit other talks, even though we wanted to. We had to ask Jarno, Esko and others about what great talks we missed. Apparently there were quite a few :-(

Our Maliit T-Shirts were well received, though we usually only handed them out when someone listened to our Maliit ramblings long enough.

We were asked about accessibility several times, which is currently not within the scope of Maliit but perhaps something to think about in the future.

We also got to talk with the people working on (text) input in Redhat and Intel, mostly in the context of Wayland. There are some interesting opportunities to get things (more) right this time around.

Thanks to our employer, Openismus, for sending us there!

9 0

After that we tried to become independent of MeeGo, but neither freedesktop.org nor the GNOME project could give us a suitable home. So we had to go with our own infrastructure in the end, which probably was the best we could do, in any case. We now enjoy our own website (mostly a wiki, for which we can also analyze the traffic), our own IRC channel, our own public bugtracker, our own mailing lists and a build bot. We also make use of other services such as launchpad.org and the openSUSE Build Service, both for packaging but also as part of our continouous integration setup. Both services provide nightly builds for Maliit, for example (though we still lack packages for ARM).

But there was always one thing missing: T-Shirts. Now that this is solved, too, we can finally call Maliit a real opensource project ;-) Hopefully we'll soon have another group photo of the people who've been involved in the project over the years. I'll make sure to bring a couple of T-Shirts to FOSDEM, so make sure grab Jon or me if you want one.

5 1

This video highlights one of Maliit's key features: pluggable input methods which come with their very own user interfaces. The Chinese input methods show how Maliit offers support for composed characters. The video is proof that 3rd party development for Maliit (open-source and proprietary) is not only possible but also happening.

maliit.org states that "it should be easy to customize existing input methods or develop powerful new input methods, whether for profit, research or fun", we actually mean it.

The harder question is of course how to motivate others to actually get started on input method development with Maliit. For that, we have a multipronged strategy:

1. Provide sufficiently polished reference plugins that can show off Maliit capabilities but also serve as inspiration for new plugins (hence the BSD license for reference plugins). Our reference plugins are currently using Qt/C++ (Maliit Keyboard) and QML (Nemo Keyboard). We also have PySide support, but no one contributed a reference plugin yet. This gives choice to interested input method developers, and we think that's important. The reference plugins serve another role when it comes to designing new API: They become our testbed, allowing us to verify our API proposals.

2. Ship Maliit with a bunch of example plugins and example applications. None of them try to be complete. They are all self-contained though and usually show one feature at a time. This can be tedious to maintain, but we believe that examples need to stay small and focused, otherwise developers won't look at them.

3. Documentation that is easy to consume. Our documentation is not as concise and clear as we'd like it to be, but it's slowly improving. We also experiment with videos that can serve as an introduction to more in-depth (text) documentation.

4. Packages for most common Linux distributions. This one seems obvious, but sadly, it's quite a lot of work for us to keep up with it (and we already use automated services such as Launchpad and OpenSuse Build Service). In the hope to attract dedicated packagers we wrote down some packaging guidelines

5. An architecture that had 3rd party plugins and multiple toolkit support in mind from the start. The plugin developer facing API needs to be easy to use and clearly documented. This will be the focus of the upcoming 0.9x series.

We will demo Maliit @ FOSDEM 2012, hope to see you there!

4 1

This means that Maliit has a near complete project infrastructure, all available under *.maliit.org, and all that thanks to Karsten, the always professional and very experienced hostmaster here at Openismus.

There is one more thing that we need (as demonstrated by me when I made my first broken release), and that's a simple build bot setup for continuous integration. Right now, we still rely on Nokia's infrastructure. I am confident that buildbot will fit all of our requirements, as long as it is trivial to maintain.

I am also happy that we improved our documentation significantly, thanks to Dave. He translated the important documentation bits into proper English and made it more accessible, demonstrating his doxygen skills. As a bonus, he also updated the project's README files, something we had neglected for a long time.

Future development

Regarding the Maliit development, I think we have simplified things a lot. D-Bus activation for the Maliit server (which finally means one server instance per user session) and the new support for plain QML plugins makes it almost trivial to get started with Maliit. We also let go of the critically acclaimed MeeGo Keyboard in Maliit upstream, which made me a bit sad of course.

Still, it probably was the right decision: MeeGo Keyboard is heavily fine-tuned for Nokia's N9 and it has some dependencies that are hard to satisfy outside of Harmattan. Over time, with ever changing requirements, the code has naturally evolved into a rather complex design. The result, however, is a very polished product, and ultimately, that's the only thing that matters (even though many opensource developers will disagree, strangely enough). Everyone in the team is proud of what we have achieved.

At the same time, I can understand how new contributors will be put off by all the complexity. So Maliit upstream will instead focus on the very basic but almost trivial to build Maliit Keyboard. For new contributors, that's a good thing. For us, it means the possibility to fix shortcomings in the plugin API. This is important, as one of our main goals has always been to enable others to write great input method plugins for Maliit, which will then run on any platform that Maliit supports. The Swype VKB plugin and the Japanese VKB plugin for example both demonstrate that we are on a good way, but I think we can do better.

Maliit itself still needs a good reference plugin, of course, even if only as a showcase (though I want it to be more). All this doesn't mean that MeeGo Keyboard goes away; its development will continue in the MeeGo Touch repositories, just as before (effectively degraded as just another Maliit plugin). But what we can take over, hopefully, is our experience when it comes to creating one of the best virtual keyboards currently available.

5 1

Miniature 0.5 is being released for MeeGo Harmattan (Nokia N9 & N950) and Maemo (Nokia N900). Thanks to everybody involved in the initial Maemo attempts and the experimental version that was made available after the Miniature 0.4 release."

We also improved usability, compared to the previous release, but there's still a ton of work left.

A bit of history

I started working on Miniature – a chess client for freechess.org – in November 2009, after reading the Call for Contributors. Even though we had a pretty cool P2P feature (based on Telepathy and developed mostly by Dariusz Mikulski), it never quite reached the original goal: playing chess online. Back then I was learning how to create UI's with Qt Graphics View, which was all the rage at the time. Well, we now know that writing real UI's with that technology is a major PITA, but for my pet project, it was just too much. I got lost in the struggle.

For the next 18 months, Miniature was basically dead. Another failed project that started so promising. Quim did not want to give up though. After the N9 announcement, he launched a second Call for Contributors.

Perhaps I responded to his mail because I was embarrased at the idea of people wasting time trying to salvage the working parts of Miniature; there simply wasn't much to salvage! So I started again, this time with a very clear goal: online chess, and online chess only. Let others create the actual UI and whatnot. Focusing on one prominent feature and not having to worry about the UI worked well for me, even though I had to iterate over some architecture ideas until I felt comfortable. Quim in the meantime started to prototype the UI with QML. It was impressive to see his results, a level of polish I could have never achieved with my Qt Graphics View approach. At some point the backend was good enough to be sewn together with the frontend and suddenly we had achieved where I failed before: A touch enabled chess client for the N9 that can play chess online.

Having my own useful application available on the N9, published through OVI store, means a lot to me. I hope others will enjoy Miniature as much as we enjoyed re-creating it the second time around.

5 0

Maliit is such a system package, but its flexible architecture allows for a creative way to replace the MeeGo Keyboard with a more recent version. This can be useful if you want to testdrive new features and to … nah whom am I kidding, it's purely for fun!

Be warned though, the following hack requires you to enable developer mode on your N9. Don't ever activate it unless you're absolutely sure what you're doing to your N9. It would be unforgivable to brick this beauty because of some misguided hack the planet attitude.

First we need to find a MeeGo Keyboard tag that will be compatible with the installed Maliit framework version on your device. Check that the output of

$ apt-cache showpkg meego-keyboard

matches the dependencies mentioned in the tag's Debian control file and the packages installed in your scratchbox ARMEL target.

Apply the community patch on top of the chosen tag. It renames the package to meego-keyboard-community and only installs the plug-in's .so file, together with a renamed CSS file (libmeegotouch requires that CSS file names match with library names).

This mean that we won't uninstall the regular package, as we still depend on most the other files that meego-keyboard installs.

Now build the Debian package. Copy it over and login to the device, then gain root access via devel-su. It's recommended to make a backup of /usr/lib/meego-im-plugins before installing the package.

After installing libmeego-keyboard-community, remove libmeego-keyboard.so from /usr/lib/meego-im-plugins, to avoid in-fights between the two plug-ins. Use

$ gconftool-2 -s /meegotouch/inputmethods/onscreen/enabled -t list --list-type string [libmeego-keyboard-community.so, en_gb.xml]
$ gconftool-2 -s /meegotouch/inputmethods/onscreen/active -t list --list-type string [libmeego-keyboard-community.so, en_gb.xml]

to activate the community plug-in. The language settings applets will most likely get confused, so be prepared that enabling new language layouts might only work directly via GConf from now on.

Gain user access and kill meego-im-uiserver. It should now load the new community plug-in. If you want to get the original MeeGo Keyboard back, uninstall the community package and copy the .so back from your backup. Alternately, you can try to reinstall it:

$ apt-get install --reinstall meego-keyboard

Have fun!

4 2

Krzesimir Nowak joined Openismus in the beginning of August this year. Not only is he a very talented developer (or, as I once said, the first one to actually fill the gap Daniel left), but he's also fun to work with.

Just before I went on my vacations in September, I left him with some nice tasks to improve one of the cool features in Maliit: Dynamic key overrides. The way I had set up the tasks I knew he would trip over bits of hideous code and seeing how he could deal with it was part of the exercise.

For now, Krzesimir's work on Maliit is finished. With 25 commits, two of them being bug fixes in said ugly code, Krzesimir did an outstanding job while thoroughly understanding and fixing one of Maliit's core features in a rather short time span. Of course he will tell you that it took him way too long ;-)

Thanks for your contribution, Krzesimir, and well done!

5 0

We released Maliit 0.80.7 on Friday. Over these last days, I am doubly proud about our project. Not only did the N9's virtual keyboard get astonishing reviews across the board, but what's even better: We managed to keep this software open-source. In our communities, there will always be those who focus too much on technical aspects. I remember the technical struggles we had even within MeeGo! But now we get feedback from real users who couldn't care less about what Qt or MeeGo Touch is, and to be honest, that's a refreshing change.

Being here at Qt's Developer Days 2011, it feels great to get such feedback directly, from first-time users of the Nokia N9. Especially the fine haptic feedback and the keyboard's accuracy gets noticed.

I also had the possibility to see a Japanese input method — running on the N9 and powered by Maliit. Seeing how well this plugin already integrates with the platform, I feel that our architecture yet again has been justified. I am looking forward to see more Maliit plugins, and more platforms using Maliit!

7 1

If you happen to create enums inside a QObject, then it will be exported to QML via the Q_ENUMS helper:

SomeEnumsWrapper
    : public QObject
{
    Q_OBJECT
    Q_ENUMS(SomeState)
public:
    enum SomeState {
        BeginState,        // Remember that in QML, enum values must start
        IntermediateState, // with a capital letter!
        EndState
    };
};

You will still need to declare this class as an abstract type for QML to be able to use enums from it (put in your main function for example):

qmlRegisterUncreatableType<SomeEnumsWrapper>("com.mydomain.myproject", 1, 0,
                                             "SomeEnums", "This exports SomeState enums to QML");

Now in QML, the enums can be accessed as '''SomeEnums.BeginState'''. Note how the enum is accessed through the exported type name, not an instance.

But what if you've put your enums into a dedicated C++ namespace? Then the same mechanism can be used. Let's start with the namespace:

namespace SomeEnums {
    enum SomeState {
        BeginState,
        IntermediateState,
        EndState
    };
}

We can re-use the idea of wrapping enums in a QObject type, with one tiny change:

SomeEnumsWrapper
    : public QObject
{
    Q_OBJECT
    Q_ENUMS(SomeState)
public:
    enum SomeState {
        BeginState = SomeEnums::BeginState, // Keeps enum values in sync!
        IntermediateState = SomeEnums::IntermediateState,
        EndState = SomeEnums::EndState
    };
};

The process of forwarding all your enums through this mechanism can be tedious, but being able to use enums properly in QML properly will improve the readability and maintainability of your QML code.

For a fully working example check Maliit's Qt Quick support

6 0

But at the same time, Nokia's N9 is one of those devices that the MeeGo community has always been waiting for.

If "This is not MeeGo!" is the only thing that comes to your mind whilst reading about all the N9 excitement then you still haven't realized MeeGo's biggest problem: No. Compelling. Devices. And if - at the same time - you are one of the MeeGo project leaders, then you should do yourself and everyone else involved with MeeGo a favor and simply resign.

We need more visionary leaders than you.

3 1

During that cleanup, Jan Arne did a tremendous job improving our build infrastructure. Instead of Qt's .prf files, Maliit now uses pkg-config everywhere. Plugin developers don't have to think about install paths - they can simply read it from the .pc files, for instance:

pkg-config --variable pluginsdir maliit-plugins-0.80

The version number at the end indicates another nice improvement: versioned libraries and plugin interfaces, which means several Maliit versions can be installed in parallel (although you never want to run more than one server per session).

To install the Maliit framework into a custom directory (say, $HOME/install), simply use:

qmake -r M_IM_PREFIX=$HOME/install

Afterwards, set your PKG_CONFIG_PATH to $HOME/install/lib/pkgconfig and compile the Maliit plugins. Quite honestly, it has never been easier to develop input methods for Maliit than with this release.

5 0

After some initial discussions with Kristian Høgsberg (Wayland, of course) and Jørgen Lind (who works on Qt Lighhouse), I also addressed Wayland. It became clear that one needs some kind of input method interface directly in Wayland. Kristian immediately started with a small prototype, in order to explain better how a Wayland compositor can provide a much better window management policy than what we currently have with Maliit and X11.

I think the session itself was really successful. I was surprised at the strong interest in this topic.

It became apparent that we should do something about Qt's input context API. For instance, add more input methods hints, come up with a better interface that describes the focus widget, preedit handling, orientation support and so on.

Now we only need to agree on how to make it happen :-)

5 0

I am planning to use Jon's example material for an input method workshop at one of the upcoming MeeGo Freedays (German only) here in Berlin, just to see how useful it is for newcomers.

Maliit starts to feel more and more like a real open-source project, and I am proud of that. Just compare our wiki from end of February with the current, information-packed version, or take a look at the steady traffic on our own mailing list (started only in March this year). Or perhaps just try googling it!

I am really happy to eventually see real contributions and input method plugins from others, which makes me think that we are on a good way.

But this is still only the beginning, and more interesting news will follow soon.

6 0

We had more people going to the talk in Dublin than to this one, but at least the slides and the recording are available online now.

4 2