While developing MiTubo I've recently felt the need of parsing HTML pages: the first problem I wanted to solve was implementing proper RSS feed detection when the user entered a website URL into MiTubo's search box, so that MiTubo would parse the site's HTML, look for <link rel="alternate"...> URLs in the HEAD section, and let the user subscribe to any video feeds found there.

A quick search in the internet did not provide a clear answer: I found a Qt HTML parser in (stalled) development, and a few other C++ or C parsers (among the latters, lexbor is the most inspiring), but all of them seem to take the approach of parsing the HTML file into a DOM tree, while I was hoping to find a lightweight SAX-like parser. Pretty much like Python's html.parser.

Anyway, I don't remember how it happened, but at a certain point I found myself looking at html.parser source code, and I was surprised to see how compact it was (apart, of course, for the long list of character references for the HTML entities!). Upon a closer look, it also appeared that the code was not making much use of Python's dynamic typing, so, I thought, maybe I could give it a try to rewrite that into a Qt class. And a few hours later QHtmlParser was born.

As this post's title suggests, the process of rewriting html.parser with Qt was quite straightforward, and the nice thing about it is that I didn't have to spend any time reading the HTML standard or trying to figure out how to implement the parser: I just had to translate Python code into C++ code, and thanks to the nice API of QString (which in many ways resembles Python's — or vice versa) this was not too hard. I even left most of the original code comments untouched, and reused quite a few tests from the test suite.

It was time well spent. :-)

If you think you might need an HTML parser for your Qt application, you are welcome to give it a try. It's not a library, just a set of files that you can import into your project; for the time being I only have a build file for QBS, but I'll happily accept contributions to make it easier to use QHtmlParser with projects built using other build systems. You can see here the changes I made in MiTubo to start using it and detect RSS feed in a webpage's HEAD.

That's all for now. And in case you missed the link before, you can find QHtmlParser here.

I was a happy user of Del.icio.us for many years until the service closed. Then I moved my links to Google Bookmarks, which offered basically the same functionality (at least for my needs): link storage with title, tags and comments. I’ve carefully tagged and filed more than 2500 links since I started, and I’ve learnt to appreciate the usefulness of searching by tag to find again some precious information that was valuable to me in the past.

As part of a summer clean-up of the desk drawers, I pulled out the N800 and N9 and ported my game Loonies 8192 to these devices. Since those are "proper" Linux devices, one can compile things directly on-device (just install gcc from the SDK repos), and with SSH, it's easy to type on a real keyboard.

Anyway, you can install the game via the landing pages:

For the N800, make sure "maemo Extras" is enabled so it will find libsdl1.2 if it's not already installed. Head over to https://loonies.thp.io/n800/ on the device and download the deb, it will be installed by Application manager.

For the N9, make sure you have n9repomirror installed (again, so libsdl1.2 can be installed if necessary). Enable third party applications in Settings, Applications, Installations. Then head over to https://loonies.thp.io/n9/ on the device and download the deb, selecting after the download is finished will ask you to install it.

The N9 version is also available on openrepos.net.

And don't forget that the game is also available for DOS, various consoles and handheld consoles as well as on Windows. All of the builds are available on itch.io.

This is the last post of the series showing interesting debugging tools, I hope you have found it useful. Don’t miss the custom scripts at the bottom to process GStreamer logs, help you highlight the interesting parts and find the root cause of difficult bugs. Here are also the previous posts of the series:

In this new post series, I’ll show you how both existing and ad-hoc tools can be helpful to find the root cause of some problems. Here are also the older posts of this series in case you find them useful:

• GStreamer WebKit debugging tricks using GDB (1/2)
• GStreamer WebKit debugging tricks using GDB (2/2)
• GStreamer WebKit debugging by instrumenting source code (1/3)
• GStreamer WebKit debugging by instrumenting source code (2/3)
• GStreamer WebKit debugging by instrumenting source code (3/3)

Use strace to know which config/library files are used by a program

If you’re becoming crazy supposing that the program should use some config and it seems to ignore it, just use strace to check what config files, libraries or other kind of files is the program actually using. Use the grep rules you need to refine the search:

$ strace -f -e trace=%file nano 2> >(grep 'nanorc')
access("/etc/nanorc", R_OK)             = 0
access("/usr/share/nano/javascript.nanorc", R_OK) = 0
access("/usr/share/nano/gentoo.nanorc", R_OK) = 0
...

Know which process is killing another one

First, try to strace -e trace=signal -p 1234 the killed process.

If that doesn’t work (eg: because it’s being killed with the uncatchable SIGKILL signal), then you can resort to modifying the kernel source code (signal.c) to log the calls to kill():

SYSCALL_DEFINE2(kill, pid_t, pid, int, sig)
{
    struct task_struct *tsk_p;
    ...
    /* Log SIGKILL */
    if (sig & 0x1F == 9) {
        tsk_p = find_task_by_vpid(pid);

        if (tsk_p) {
            printk(KERN_DEBUG "Sig: %d from pid: %d (%s) to pid: %d (%s)\n",
                sig, current->pid, current->comm, pid, tsk_p->comm);
        } else {
            printk(KERN_DEBUG "Sig: %d from pid: %d (%s) to pid: %d\n",
                sig, current->pid, current->comm, pid);
        }
    }
    ...
}

Wrap gcc/ld/make to tweak build parameters

If you ever find yourself with little time in front of a stubborn build system and, no matter what you try, you can’t get the right flags to the compiler, think about putting something (a wrapper) between the build system and the compiler. Example for g++:

#!/bin/bash
main() {
    # Build up arg[] array with all options to be passed
    # to subcommand.
    i=0
    for opt in "$@"; do
        case "$opt" in
        -O2) ;; # Removes this option
        *)
            arg[i]="$opt" # Keeps the others
            i=$((i+1))
            ;;
        esac
    done
    EXTRA_FLAGS="-O0" # Adds extra option
    echo "g++ ${EXTRA_FLAGS} ${arg[@]}" # >> /tmp/build.log # Logs the command
    /usr/bin/ccache g++ ${EXTRA_FLAGS} "${arg[@]}" # Runs the command
}
main "$@"

Make sure that the wrappers appear earlier than the real commands in your PATH.

The make wrapper can also call remake instead. Remake is fully compatible with make but has features to help debugging compilation and makefile errors.

Analyze the structure of MP4 data

The ISOBMFF Box Structure Viewer online tool allows you to upload an MP4 file and explore its structure.

I got myself the MouldKing 13106 Forklift, which is based on the MOC 3681 by KevinMoo and wanted to share my impressions with you.

This is the last post on the instrumenting source code series. I hope you to find the tricks below as useful as the previous ones.

Lets say, you want to reduce the water carbonate hardness because you got a shiny coffee machine and descaling that is a time-consuming mess.

If you dont happen to run a coffee-shop, using a water-jug is totally sufficient for this. Unfortunately, while the jug itself is quite cheap, the filters you need will cost you an arm and a leg – similar to how the printer-ink business works.

The setup

Here, we want to look at the different filter options and compare their performance. The contenders are

NamePricingBrita Classic~15.19 €PearlCo Classic12.90 €PearlCo Protect+15.90 €

As said initially, the primary goal of using these filters is to reduce the water carbonate – any other changes, like pH mythology, will not be considered.

To measure the performance in this regard, I am using a digital total dissolved solid meter – just like the one used in the Wikipedia article. To make the measurement robust against environmental variations, I am not only measuring the PPM in the filtered water, but also in the tap water before filtering. The main indicator is then the reduction factor.

Also, you are not using the filter only once, so I repeat the measuring over the course of 37 days. Why 37? Well, most filters are specified for 30 days of usage – but I want to see how much cushion we got there.

So – without further ado – the results:

Results

NameØ PPM reductionØ absolute PPMBrita Classic31%206PearlCo Classic24%218PearlCo Protect+32%191

As motivated above, the difference in absolute PPM can be explained by environmental variation – after all the measurements took place over the course of more than 3 months.

However, we see that the pricing difference is indeed reflected by filtering performance. By paying ~20% more, you get a ~30% higher PPM reduction.

The only thing missing, is the time-series to see beyond 30 days:

As you can see, the filtering performance is continuously declining after a peak at about 10-15 days of use.

And for completeness, the absolute PPM values:

In this post I show some more useful debugging tricks. Check also the other posts of the series:

This is the continuation of the GStreamer WebKit debugging tricks post series. In the next three posts, I’ll focus on what we can get by doing some little changes to the source code for debugging purposes (known as “instrumenting”), but before, you might want to check the previous posts of the series: