Planet maemo: category "feed:db85272b5cc4c48c874836930690af4e"
Over recent years, libferris has been using Qt to mount some Web stuff as a filesystem. I have a subclass of QIODevice which acts as an intermediary to allow one to write to a std::ostream and stream that data to the Web, over a POST for example. For those interested, that code is in Ferris/FerrisQt.cpp of the tarball. It's a bit of a shame that Qt heavy web code isn't in KIO or that the two virtual filesystems are not closer linked, but I digress.
Libferris has been gaining some KDE love in recent times. There is now a KIO slave to allow you to see libferris from KDE, also the ability to get at libferris from plasma.
I've been meaning to update the mounting of some Web services like vimeo for quite some time. I'd also like to expand to allow mounting google+ as a filesystem and add other new Web services.
In order to manage time so that this can happen quicker, I thought I'd try the waters with a pledgie. I've left this open ended rather than sticking an exact "bounty" on things. I had the idea of trying a pledgie with my recent investigation into the libferris indexing plugins on a small form factor ARM machine. I'd like to be able to spend more time on libferris, and also pay the rent while doing that, so I thought I'd throw the idea out into the public.
If you've enjoyed the old tricks of mounting XML, Berkeley DB, SQLite, PostgreSQL and other relational databases, flickr, google docs, identica, and others and want to see more then please support the pledgie to speed up continued development. Enjoy libferris!
I've been meaning to update the mounting of some Web services like vimeo for quite some time. I'd also like to expand to allow mounting google+ as a filesystem and add other new Web services.
In order to manage time so that this can happen quicker, I thought I'd try the waters with a pledgie. I've left this open ended rather than sticking an exact "bounty" on things. I had the idea of trying a pledgie with my recent investigation into the libferris indexing plugins on a small form factor ARM machine. I'd like to be able to spend more time on libferris, and also pay the rent while doing that, so I thought I'd throw the idea out into the public.
If you've enjoyed the old tricks of mounting XML, Berkeley DB, SQLite, PostgreSQL and other relational databases, flickr, google docs, identica, and others and want to see more then please support the pledgie to speed up continued development. Enjoy libferris!
Sorry for the chatter post, but if anybody has recommendations for a tool that can build for win, osx, and lin that would be great. The project is an autofools one, mainly coded in 100s of kloc of C++. Builds like a treat on a Fedora machine, can be beaten unto submission to build on osx, and I assume on a suitably tainted windows machine it will gcc into binaries too. At least it has built on those other platforms in the past.
I've had great success with OBS, but that was mainly for Linux packages. It seems OBS can do mingw too, but I've not walked the valley of darkness into building for the more closed platforms on OBS before.
The saucelabs looks pretty cool, but it seems targeted to web code if I am reading it correctly.
The initial plan is to get 24hr rolling packages for all platforms and have feedback as to which day a github commit has broken the package build. It might be nice to have it for each github commit, but I think it would be easy enough to bisect a break given a 24 hour window unless an armada of contributors rushes at the ship.
A separate build issue I've been tinkering with in my mind for a while is grabbing from a github repo and creating android packages. Different code base for this though, mainly some of my n9 apps, as such, preferably for a mixed C++/QML app. But I think for that project I'll wind up taking my chisel and hammer and coming back with a cron job.
I've had great success with OBS, but that was mainly for Linux packages. It seems OBS can do mingw too, but I've not walked the valley of darkness into building for the more closed platforms on OBS before.
The saucelabs looks pretty cool, but it seems targeted to web code if I am reading it correctly.
The initial plan is to get 24hr rolling packages for all platforms and have feedback as to which day a github commit has broken the package build. It might be nice to have it for each github commit, but I think it would be easy enough to bisect a break given a 24 hour window unless an armada of contributors rushes at the ship.
A separate build issue I've been tinkering with in my mind for a while is grabbing from a github repo and creating android packages. Different code base for this though, mainly some of my n9 apps, as such, preferably for a mixed C++/QML app. But I think for that project I'll wind up taking my chisel and hammer and coming back with a cron job.
OK, a rather long post about effective digital communication. Hopefully an interesting read to folks who would like to add some code to protect communications but haven't gotten around to that TODO item just yet.
After doing some power benchmarking recently (1.2ghz kirkwood with gb net up = 5watts) I decided to work out how fast these arms can do useful work^TM. In the running is a Synology ds212j, DreamPlug running the Freedombox, Nokia n9 mobile phone, and an Intel 2600k just for working out where those relative numbers sit when compared to a desktop machine.
The above image shows the cipher performance of "openssl speed" across many machines. The 2600k is only single threaded, so could be many times faster in real world use by taking advantage of the cores. One interesting point right off the bat is that the 1.2Ghz kirkwood in the synology NAS is bested by the 1.0ghz CPU of the Nokia n9. Arms is not arms.
Removing the overload from the Intel i7 2600K from the graph we see that the Dreamplug is very close to the ds212j in terms of performance.
On the other hand, the digests show a distinct advantage to the Dreamplug setup. Again the n9 has a nice little lead on the others. Since a mobile phone can perform some useful work, one should perhaps also be demanding the NAS also offer handy features as well as just serving data blocks.
The RSA sign and verify graphs both show the same large slump for the ds212j unit. So for connection heavy workloads there would seem to be a large difference to the throughput you might get for the ARM you choose. On the other hand, the Dreamplug and ds212j both have similar performance on steam ciphers. So if connections are longer term then the difference will be less.
I would love to add benchmarks for the CuBox and QNAP 2ghz (TS-219PII) NAS units. It would also be interesting to use after market software on the ds212j and see the difference.
The above image shows the cipher performance of "openssl speed" across many machines. The 2600k is only single threaded, so could be many times faster in real world use by taking advantage of the cores. One interesting point right off the bat is that the 1.2Ghz kirkwood in the synology NAS is bested by the 1.0ghz CPU of the Nokia n9. Arms is not arms.
Removing the overload from the Intel i7 2600K from the graph we see that the Dreamplug is very close to the ds212j in terms of performance.
On the other hand, the digests show a distinct advantage to the Dreamplug setup. Again the n9 has a nice little lead on the others. Since a mobile phone can perform some useful work, one should perhaps also be demanding the NAS also offer handy features as well as just serving data blocks.
The RSA sign and verify graphs both show the same large slump for the ds212j unit. So for connection heavy workloads there would seem to be a large difference to the throughput you might get for the ARM you choose. On the other hand, the Dreamplug and ds212j both have similar performance on steam ciphers. So if connections are longer term then the difference will be less.
I would love to add benchmarks for the CuBox and QNAP 2ghz (TS-219PII) NAS units. It would also be interesting to use after market software on the ds212j and see the difference.
I have in mind an app to use RFID over the NFC hardware on the n9 to allow the phone to be unlocked by an RFID tag. This way the password can be nice and long for locking the device, but I don't have to interact with the virtual keyboard in order to use the phone. If the phone locks, I just tap the RFID tag to it, it unlocks, and I'm away. Yes, two phase would be a logical next step, but that assumes that the "lock" on the phone is very hard to break.
Reading the RFID on the n9 is the simple part, I have that working fine with physical keys. Unlocking the phone is proving not so simple. I was hoping that MeeGo::QmLocks::setState would do the trick, but it is not to be so as the API docs explain.
Anyone stumbled on a way to unlock the n9 from a program?
Reading the RFID on the n9 is the simple part, I have that working fine with physical keys. Unlocking the phone is proving not so simple. I was hoping that MeeGo::QmLocks::setState would do the trick, but it is not to be so as the API docs explain.
Anyone stumbled on a way to unlock the n9 from a program?
I thought I'd see how well some pledgie tasks would do in the wild. Some of this code is stuff I've been wishing to write for a while but unable to make the time.
If you want to get at some of the funky filesystems offered by libferris from your KDE desktop, you might like to support my "KIO Slave for libferris" pledgie:
If you've always wanted to add support for Off-the-Record messaging to your project, or help entice somebody else in that direction, I have a tutorial article on offer here:
If you want a REST interface for libferris, listing directories and getting at files over HTTP/HTTPS, you can help make that happen here:
And finally, last for today but not least, if you want to get at jpeg images on your Zoneminder server using nice normal command lines like the following:
Then you might like to throw a little loose change at the zoneminder plugin pledgie:
If you want to get at some of the funky filesystems offered by libferris from your KDE desktop, you might like to support my "KIO Slave for libferris" pledgie:
If you've always wanted to add support for Off-the-Record messaging to your project, or help entice somebody else in that direction, I have a tutorial article on offer here:
If you want a REST interface for libferris, listing directories and getting at files over HTTP/HTTPS, you can help make that happen here:
And finally, last for today but not least, if you want to get at jpeg images on your Zoneminder server using nice normal command lines like the following:
$ fcat zoneminder://server/monitor | okular -
Then you might like to throw a little loose change at the zoneminder plugin pledgie:
I always tend to lean more towards SSH for VPN stuff than other tools like openvpn. With this mindset I was working out how one might get access to servers on the Intranet while on the move. Having created a bunch of nice little apps that do things on my network is all well and good, but I wanted to also use them while on the move.
My current solution is to use two socks servers on the phone. It might seem strange to need two, but this allows routing depending on source address and destination address. The first SOCKs server is dante which routes all "normal" traffic directly to the chosen Internet connection on the phone. Traffic which wants to go to selected addresses on my Intranet is instead forwarded to a second socks proxy on the phone. This second socks proxy is implemented with ssh "DynamicForward" to send traffic over the SSH link to the Intranet.
The glory here is that Internet traffic goes directly, and Intranet traffic is sent over an encrypted pipe to the Intranet. If on the other hand I connect to any router on the Intranet, the phone gets a local IP address and dante is instructed to "route" traffic from that local address directly to the target instead of to the second local ssh socks server. Thus I don't have to concern myself with the details of which side of firewalls I am on, the phone automatically does that for me and chooses the most efficient route for packets.
Having a phone that runs so much like a proper Linux distro makes this sort of thing fairly easy to get going. No need for funky NDKs, NDAs, or TLAs.
If you want to play around with this stuff I have dante debs in my repo:
http://fuuko.libferris.com/maemo/n9/
Using non privlidged ports on the phone means you can run danted as a normal user. I have an icon on the n9 home screen that makes sure dante is running instead of using init.d at the moment. Also, an icon that runs a bash encapsulated with tsocks can be handy for tinkering:
Desktop file segment:
Using a similar style Exec in a desktop file one can start the SSH client in "-N" mode and use the passphrase to connect and setup an SSH connection with a socks server on the phone. It is quick and dirty without shiny QML interfaces, but it works^TM.
My current solution is to use two socks servers on the phone. It might seem strange to need two, but this allows routing depending on source address and destination address. The first SOCKs server is dante which routes all "normal" traffic directly to the chosen Internet connection on the phone. Traffic which wants to go to selected addresses on my Intranet is instead forwarded to a second socks proxy on the phone. This second socks proxy is implemented with ssh "DynamicForward" to send traffic over the SSH link to the Intranet.
The glory here is that Internet traffic goes directly, and Intranet traffic is sent over an encrypted pipe to the Intranet. If on the other hand I connect to any router on the Intranet, the phone gets a local IP address and dante is instructed to "route" traffic from that local address directly to the target instead of to the second local ssh socks server. Thus I don't have to concern myself with the details of which side of firewalls I am on, the phone automatically does that for me and chooses the most efficient route for packets.
Having a phone that runs so much like a proper Linux distro makes this sort of thing fairly easy to get going. No need for funky NDKs, NDAs, or TLAs.
If you want to play around with this stuff I have dante debs in my repo:
http://fuuko.libferris.com/maemo/n9/
Using non privlidged ports on the phone means you can run danted as a normal user. I have an icon on the n9 home screen that makes sure dante is running instead of using init.d at the moment. Also, an icon that runs a bash encapsulated with tsocks can be handy for tinkering:
Desktop file segment:
Exec=meego-terminal -n -e /bin/sh /home/user/bin/create-tsocks-env
...
$ cat /home/user/bin/create-tsocks-env
export http_proxy=http://1.2.3.4:1313/
tsocks bash
Using a similar style Exec in a desktop file one can start the SSH client in "-N" mode and use the passphrase to connect and setup an SSH connection with a socks server on the phone. It is quick and dirty without shiny QML interfaces, but it works^TM.
My growing collection of Qt/QML applications has now hit 6 apps with the inclusion of a Zoneminder client for the n9. As I still have ideas for more apps I wonder how long until I fill half a screen with my own apps on the device ;)
Anyways, the Zoneminder app uses server side scaling and jpeg generation to keep bandwidth down and CPU/RAM very low on the device. I have modified some of the images for example purposes. The app is basically as shown but will display your real camera data instead of Koalas... Unless you are lucky enough to have some furry little friends. The first camera starts a live view right off the bat when the app starts, so there is as little finger use needed to get some useful action from running the app.
Only active cameras are listed. The play button changes the image on the right to be a 1 second rolling "live view" of that camera. The little watch moves to the recorded events for that camera which is shown below. The config is quite simple; the Zoneminder server URL, and the user/password to login to that server.
Alarm events are shown on the left side and the play button rolls through that event with the progress bar showing how far through the alarm you are. Naturally both the lists scroll to show more cameras or events if there are more than 5. By default only 25 of the most recent alarm events are shown in the list. A future expansion might allow paging back to see older ones.
Anyways, the Zoneminder app uses server side scaling and jpeg generation to keep bandwidth down and CPU/RAM very low on the device. I have modified some of the images for example purposes. The app is basically as shown but will display your real camera data instead of Koalas... Unless you are lucky enough to have some furry little friends. The first camera starts a live view right off the bat when the app starts, so there is as little finger use needed to get some useful action from running the app.
Only active cameras are listed. The play button changes the image on the right to be a 1 second rolling "live view" of that camera. The little watch moves to the recorded events for that camera which is shown below. The config is quite simple; the Zoneminder server URL, and the user/password to login to that server.
Alarm events are shown on the left side and the play button rolls through that event with the progress bar showing how far through the alarm you are. Naturally both the lists scroll to show more cameras or events if there are more than 5. By default only 25 of the most recent alarm events are shown in the list. A future expansion might allow paging back to see older ones.
The n9 libferris app now allows you to search by the URL of the file as before, but also now by the file text content and it's modification time. Query type is selected by a button in the top right corner which unfortunately isn't nearly as easy to read in the video file as it is in real life.
As libferris handles extraction, update, and storage of metadata from disparate locations I have also added a sprinkling of what that means into the video. Notice that the first search by URL shows a comment "REST interface to libferris" in bold. This is simply the "annotation" metadata of the file but is much more interesting to the searcher than it's URL. Likewise in the second query, which finds a Gutenburg text file by searhing on text content, the annotation offers the name of the book that the file contains. Again much more interesting content to the human who is at the helm.
The third query is on the modification time of the files. There are three ways offered to perform a time search, "more recently than this" or "than last" which can have month or day as options for example, and modified >= X months ago which obviously wants a number as the query text. When querying by time like this, libferris happily accepts some human readable terms like "begin this month" as the time you specify. This makes it just as convenient to use search in scripts as in the dedicated front ends like the n9 app.
The plan is as always, to push metadata from being an after thought to being a first class citizen: able to be created, read, written, indexed, and searched on. Any file with metadata should be able to expose that as simply as its mtime or size, which is all currently done via a key-value "Extended Attribute" interface at the lowest level.
If you like libferris or the recent updates (status.net mounting, these indexing tweaks etc) then please consider making a donation. If you want to use this technology at a corporate level, please feel free to contact me.
As libferris handles extraction, update, and storage of metadata from disparate locations I have also added a sprinkling of what that means into the video. Notice that the first search by URL shows a comment "REST interface to libferris" in bold. This is simply the "annotation" metadata of the file but is much more interesting to the searcher than it's URL. Likewise in the second query, which finds a Gutenburg text file by searhing on text content, the annotation offers the name of the book that the file contains. Again much more interesting content to the human who is at the helm.
The third query is on the modification time of the files. There are three ways offered to perform a time search, "more recently than this" or "than last" which can have month or day as options for example, and modified >= X months ago which obviously wants a number as the query text. When querying by time like this, libferris happily accepts some human readable terms like "begin this month" as the time you specify. This makes it just as convenient to use search in scripts as in the dedicated front ends like the n9 app.
The plan is as always, to push metadata from being an after thought to being a first class citizen: able to be created, read, written, indexed, and searched on. Any file with metadata should be able to expose that as simply as its mtime or size, which is all currently done via a key-value "Extended Attribute" interface at the lowest level.
If you like libferris or the recent updates (status.net mounting, these indexing tweaks etc) then please consider making a donation. If you want to use this technology at a corporate level, please feel free to contact me.
Continuing my n9 apps I now have an index search app that uses libferris. Currently I have only exposed URL regular expression search. This is searching an index maintained and stored in a PostgreSQL database. Solving regex query in a timely manner is a fairly complex problem with one useful solution outlined in one of my previous blog entries.
Note that data is downloaded over normal http(s) from the server using the n9 app, so no NFS or other network mounts are needed. This might be handy for grabbing a PDF, image or text file off an Intranet file server while on the move.
I have also been updating the clucene libferris module to allow more effective index use during regex/infix queries. This libferris engine works quite nicely on older ARM machines. Benchmarks on that to follow, once I reindex the arm and produce the comparative figures.
Note that data is downloaded over normal http(s) from the server using the n9 app, so no NFS or other network mounts are needed. This might be handy for grabbing a PDF, image or text file off an Intranet file server while on the move.
I have also been updating the clucene libferris module to allow more effective index use during regex/infix queries. This libferris engine works quite nicely on older ARM machines. Benchmarks on that to follow, once I reindex the arm and produce the comparative figures.
As part of my foray into n9 apps, a new one that displays information from the Australian Bureau of Meteorology on the n9. Being pure QML + Javascript it also runs on the desktop. The forecasts include min and max temps, and for the Europeans reading this, the middle line "10..." is the UV prediction and roasting window for the day... all your sunscreen are belong to us.
Apart from saving a little on downloads for the topology and background images when on the move, you get to customize those as they are just png files living in the images subdirectory. So I have the local rocket launch site indicated. I'd like to make a LOTR style topology overlay, perhaps mt Glorious becomes Mordor?
Code available at https://github.com/monkeyiq/qmlbom
Apart from saving a little on downloads for the topology and background images when on the move, you get to customize those as they are just png files living in the images subdirectory. So I have the local rocket launch site indicated. I'd like to make a LOTR style topology overlay, perhaps mt Glorious becomes Mordor?
Code available at https://github.com/monkeyiq/qmlbom