Planet Maemo: category "feed:db85272b5cc4c48c874836930690af4e"

Like a Bird on a Wire(shark)...

Sat, 20 Jul 2013 17:10:00 +0000

Over recent years, libferris has been using Qt to mount some Web stuff as a filesystem. I have a subclass of QIODevice which acts as an intermediary to allow one to write to a std::ostream and stream that data to the Web, over a POST for example. For those interested, that code is in Ferris/FerrisQt.cpp of the tarball. It's a bit of a shame that Qt heavy web code isn't in KIO or that the two virtual filesystems are not closer linked, but I digress.

I noticed a little while ago that cp to vimeo://upload didn't work anymore. I had earmarked that for fixing and recently got around to making that happen. It's always fun interacting with these Web APIs. Over the time I've found that Flickr sets the bar for well documented APIs that you can start to use if you have any clue about making GET and POST etc. At one stage google had documented their API in a way that you could never use it. I guess they have fixed that by now, but it did sort out the pretenders from those two could at least sniff HTTP and were determined to win. The vimeo documentation IIRC wasn't too bad when I added support to upload, but the docs have taken a turn for the worst it seems. Oh, one fun tip for the young players, when one API call says "great, thanks, well done, I've accepted your call" and then a subsequent one says "oh, a strange error has happened", you might like to assume that the previous call might not have been so great after all.

So I started tinkering around, adding oauth to the vimeo signup, and getting the getTicket call to work. Having getTicket working meant that my oauth signed call was accepted too. I then was then faced with the upload of the core data (which is normally done with a rather complex streaming POST), and the final I'm done, make it available call. On vimeo that last call seems to be two calls now, first a VerifyChunks call and then a Complete call.

So, first things first. To upload you call getTicket which gives you an endpoint that is an HTTP URL to send the actual video data to, as well as an upload ticket to identify the session. If you try to post to that endpoint URL and the POST converts the CGI parameters using multipart/form-data with boundaries into individual Content-Disposition: form-data elements, you loose. You have to have the ticket_id in the URL after the POST text in order to upload. One little trap.

So then I found that verifyChunks was returning Error 709 Access to the chunk list failed. And that was after the upload had been replied to with "OK. Thanks for the upload.". Oddly, I also noticed that the upload of video data would hang from time to time. So I let the shark out of the pen again, and found that vimeo would return it's "yep were done, all is well" response to the HTTP POST call at about 38-42kb into the data. Not so great.

Mangling the vimeo.php test they supply to upload with my oauth and libferris credentials I found that the POST had a header Expect: 100-continue. Right after the headers were sent vimeo gave the nod to continue, and then the POST body was sent. I assume that just ploughing through and giving the headers followed by the body confused the server end and thus it just said "yep, ok, thanks for the upload" and dropped the line. Then of course forgot the ticket_id because there was no data for it, so the verifyChunks got no chunk list and returned the strange error it did. mmm, hindsight!

So I ended up converting from the POST the newly available PUT method for upload. They call that their "streaming API" even though you can of course stream to a POST endpoint. You just need to frame the parameters and add the MIME tailer to the POST if you want to stream a large file that way. Using PUT I was then able to verify my chunks (or the one single chunk in fact) and the upload complete method worked again.

In the end I've added oauth to my vimeo mounting, many thanks to the creators of the QOAuth library!
2 0

Save Ferris: Show some love for libferris...

Wed, 08 May 2013 01:57:00 +0000

Libferris has been gaining some KDE love in recent times. There is now a KIO slave to allow you to see libferris from KDE, also the ability to get at libferris from plasma.

I've been meaning to update the mounting of some Web services like vimeo for quite some time. I'd also like to expand to allow mounting google+ as a filesystem and add other new Web services.

In order to manage time so that this can happen quicker, I thought I'd try the waters with a pledgie. I've left this open ended rather than sticking an exact "bounty" on things. I had the idea of trying a pledgie with my recent investigation into the libferris indexing plugins on a small form factor ARM machine. I'd like to be able to spend more time on libferris, and also pay the rent while doing that, so I thought I'd throw the idea out into the public.

If you've enjoyed the old tricks of mounting XML, Berkeley DB, SQLite, PostgreSQL and other relational databases, flickr, google docs, identica, and others and want to see more then please support the pledgie to speed up continued development. Enjoy libferris!

0 0

Cross platform package building..?

Fri, 14 Dec 2012 15:53:00 +0000

Sorry for the chatter post, but if anybody has recommendations for a tool that can build for win, osx, and lin that would be great. The project is an autofools one, mainly coded in 100s of kloc of C++. Builds like a treat on a Fedora machine, can be beaten unto submission to build on osx, and I assume on a suitably tainted windows machine it will gcc into binaries too. At least it has built on those other platforms in the past.

I've had great success with OBS, but that was mainly for Linux packages. It seems OBS can do mingw too, but I've not walked the valley of darkness into building for the more closed platforms on OBS before.

The saucelabs looks pretty cool, but it seems targeted to web code if I am reading it correctly.

The initial plan is to get 24hr rolling packages for all platforms and have feedback as to which day a github commit has broken the package build. It might be nice to have it for each github commit, but I think it would be easy enough to bisect a break given a 24 hour window unless an armada of contributors rushes at the ship.

A separate build issue I've been tinkering with in my mind for a while is grabbing from a github repo and creating android packages. Different code base for this though, mainly some of my n9 apps, as such, preferably for a mixed C++/QML app. But I think for that project I'll wind up taking my chisel and hammer and coming back with a cron job.

1 1

Off the Record Messaging: A Tutorial

Mon, 30 Jul 2012 06:54:00 +0000

OK, a rather long post about effective digital communication. Hopefully an interesting read to folks who would like to add some code to protect communications but haven't gotten around to that TODO item just yet.

A commonly used method for sending messages to others when you need authentication and privacy is to use an OpenPGP tool such as GNU Privacy Guard (GnuPG). For real time communications such as instant messaging, IRC, and socket IO, using Off The Record (OTR) messaging provides Perfect Forward Secrecy and secure identification of the remote party without the need for a web of trust.

In order to operate without a web of trust, libotr implements the Socialist Millionaires' Protocol (SMP). The SMP allows two parties to verify that they both know the same secret. The secret might be a passphrase or answer to a private joke that two people will easily know. The SMP operates fine in the presence of eaves droppers (who don't get to learn the secret). Active communications tampering is not a problem, though of course it might cause the protocol not to complete successfully.

Because the SMP doesn't rely on the fingerprint of the user's private key for authentication, the private key becomes almost an implementation detail. Once generated, the user generally doesn't need to know about the key or it's fingerprint. The only time a user really cares to know is when a key is created because a bit of entropy has to go into that process. Of course, an application should avoid regenerating keys for no reason because each time the key is replaced the user has to use the SMP again to allow remote parties to authenticate them.

In this article I'll show you how to use the current release, libotr 3.2.0+, to provide OTR messaging. I'll present two examples which are both in C++ and use the boost library for socket IO. I have gone this was so we can focus on the OTR action and not the details of sockets.

The first example does not use the Socialist Millionaires' Protocol (SMP). So the new_fingerprint() callback is essential to establishing a secure session. When not using the SMP, authentication is performed by comparing the sent fingerprints of those you are wishing to communicate with against known good values. These known values must be sent beforehand through a secure secondary channel, such as a face to face meeting. Once fingerprints have been accepted, subsequent OTR communications with the same party can be performed without explicit fingerprint verification.

The second example makes things simpler for the user by using the SMP for authentication of the remote party. This way, the information exchanged beforehand becomes shared experiences you and the other party have had such that a question can be raised that only you and they can easily answer.
A central abstraction in using the libotr library is the struct s_OtrlMessageAppOps vtable. This is used by libotr to callback into your code when something happens such as a cryptographic fingerprint being received, or libotr wanting to send a message to the other end. The later happens frequently during OTR session establishment.

If a program monitors it's socket IO using select() or some other mainloop abstraction, then having these internal protocol messages being sent is not so much of an issue. Alas, for the simple echo server I present one must remember that there might be one or more internal OTR protocol messages sent from what seems like outside of the normal program flow. I'll get back to this point while describing the relevant section of the first example.

Many of the callback functions in s_OtrlMessageAppOps might be simple stubs, but you should be aware of inject_message() which will be called when libotr itself wants to send something, notify and display_otr_message can both provide feedback to the user, the new_fingerprint() method is called when a remote key is discovered in order to allow you to inform the user and possibly abort the session. The gone_secure() method is called to allow you to inform the user that they are off the record. When you call libotr functions you supply both a pointer to a s_OtrlMessageAppOps structure uiops and a void* opdata. When libotr calls a method in uiops it will pass opdata back to you.

Another common three parameters you will pass to libotr functions are the accountname, protocol and sender or receiver name. The protocol string can be anything as long as both ends of the system use the same protocol string. The state data that libotr uses is stored in an OtrlUserState object which is created with otrl_userstate_create() and passed to many of the libotr functions along the way.

The code below loads a private key or creates a new one if none already exists. Because creating a new key is an entropy heavy operation, the setupKey() function warns the user that if they are erratic it the process might move along a bit quicker. Note that the uiops has a callback create_privkey to generate a key if needed. I just prefer to make this codepath explicit and out of the main callback logic.

 bool ok( gcry_error_t et )
{
    return gcry_err_code(et) == GPG_ERR_NO_ERROR;
}

void setupKey( const std::string& filename )
{
    gcry_error_t et;
    
    et = otrl_privkey_read( userstate, filename.c_str() );
    if( !ok(et) )
    {
        cerr << "can't find existing key, generating a new one!" << endl;
        cerr << "this needs a bunch of entropy from your machine... so please" << endl;
        cerr << "move the mouse around and slap some keys mindlessly for a while" << endl;
        cerr << "a message will be printed when keys have been made..." << endl;
        et = otrl_privkey_generate( userstate, filename.c_str(),
                                    accountname, protocol );
        if( !ok(et) )
        {
            cerr << "failed to write new key file at:" << filename << endl;
        }
        cerr << "Have keys!" << endl;
    }
}

The main.cpp program implements both the client and server. The server mode is selected by passing -s at startup. Firstly, a userstate is created, some variables set depending on if we are a client or server, and the correct private key is loaded or created.

    OTRL_INIT;
    userstate = otrl_userstate_create();

    keyfile = "client.key";
    accountname = "client";
    recipientname = "server";
    if( ServerMode )
    {
        keyfile = "server.key";
        accountname = "server";
        recipientname = "client";
    }
    setupKey( keyfile );

The core logic for the echo client is to read a string from the user, send it to the server, grab a reply from the server and show it to the user. The start of the client code connects to a given port on localhost and reads a string from the user.

        VMSG << "client mode..." << endl;
        stringstream portss;
        portss << Port;
        iosockstream stream( "127.0.0.1", portss.str() );
        if (!stream)
        {
            cerr << "can't connect to server!" << endl;
            exit(1);
        }

        string s;
        while( true )
        {
            getline(cin,s);
            cerr << "your raw message:" << s << endl;
            cerr << "send plaintext:" << colorsend(s) << endl;

We certainly do not want to send the raw string s over the wire to the server though. That would very much be "on the record". So the next fragment of the client gets libotr to encrypt the string s so we can send it off the record to the server. The userstate is the value created during program initialization using otrl_userstate_create(). The ui_ops is the vtable s_OtrlMessageAppOps structure described above, and opdata is the value we want libotr to pass back to our methods in ui_ops when it uses them. In this case, we use the address of the iostream for the socket as the opdata so callbacks can send and receive data on the socket if they so desire. The newmessage will point to an off-the-record message that the server can decrypt to read the string s. The tests on the return value for message_sending() ensure that we have a new, encrypted off the record message to send instead of the plaintext s.

 void* opdata = &stream;
OtrlTLV* tlvs = 0;
gcry_error_t et;
char* newmessage;

void* opdata = &stream;
OtrlTLV* tlvs = 0;
gcry_error_t et;
char* newmessage;

et = otrl_message_sending( userstate, &ui_ops, opdata,
                           accountname, protocol, recipientname,
                           s.c_str(), tlvs, &newmessage,
                           myotr_add_appdata, &ui_ops );
cerr << "encoded... ok:" << ok(et) << endl;
if( !ok(et) )
{
    cerr << "OTR message_sending() failed!" << endl;
}
if( ok(et) && !newmessage )
{
    cerr << "There was no error, but an OTR message could not be made." << endl;
    cerr << "perhaps you need to run some key authentication first..." << endl;
}
if( newmessage )
{
    VMSG << "have new OTR message:" << newmessage << endl;
    s = newmessage;
}

Since we have replaced the plaintext s with the off the record version, we send that to the server using the socket iostream and then wait a moment before reading a response. The while loop is slightly hairy in that it will block for new messages if we are not secure. As I mentioned above, libotr can call the inject_message() callback to write a new off the record message to the socket. Outgoing messages will be generated and injected during session establishment. There is no incoming version of inject_message() so the client needs to keep reading these injected messages before it tries to send another off the record message. One will find that there are many messages exchanged between libotr at each end when the string s is written to the socket. This only happens the first time through to setup the OTR protocol.

When reading messages from the server, the encrypted string is read and passed to otrl_message_receiving(). If the recevied message was an OTR message that was sent from the other end by libotr using inject_message() then otrl_message_receiving() will indicate to the client that it should simply ignore this message. Otherwise a real message was encrypted and sent by the server and so the client will show the user the decrypted newmessage.

 cerr << "WRITE:" << s << endl;
stream << s << endl;
usleep( 200 * 1000 );
while( !secure && stream.peek() != std::iostream::traits_type::eof()
       || secure && stream.rdbuf()->available() )
{
    s = "junk";
    VMSG << "reading data from server" << endl;
    getline(stream,s);
    VMSG << "READ:" << s << endl;

    int ignore_message = otrl_message_receiving(
        userstate, &ui_ops, opdata,
        accountname, protocol, recipientname,
        s.c_str(),
        &newmessage,
        &tlvs,
        myotr_add_appdata, &ui_ops );

    VMSG << "ignore:" << ignore_message << " newmsg:" << maybenull(newmessage) << endl;
    if( ignore_message )
    {
        VMSG << "libotr told us to ignore this message..." << endl;
        VMSG << "available:" << stream.rdbuf()->available() << endl;
        VMSG << " in_avail:" << stream.rdbuf()->in_avail() << endl;
        
        continue;
    }
    if( newmessage )
        s = newmessage;
    otrl_message_free( newmessage );

    cout << color( s ) << endl;
}

Server mode is handled by a thread which executes server_session() using the std::iostream for the new socket.

if( ServerMode )
{
    VMSG << "server mode..." << endl;

    boost::asio::io_service io_service;
    tcp::acceptor a( io_service, tcp::endpoint( tcp::v4(), Port ));
    for (;;)
    {
        h_iosockstream stream(new iosockstream());
        a.accept( *(stream->rdbuf()) );
        boost::thread t(boost::bind(server_session, stream));
    }
}

The server implementation would look like the below if OTR messaging was not being used.

void server_session( h_iosockstream streamptr )
{
    iosockstream& stream = *(streamptr.get());
    while( stream )
    {
       std::string s;
       getline( stream,s );
       cout << "server got:" << s << endl;
       stream << s << endl;
    }
}

The OTR server implementation starts out the same way, reading a string from the socket. Then our old friend otrl_message_receiving() is called to decrypt that message. If ignore_message is set then there is nothing to be done and we simply continue to the top of the loop to read another string from the client. Also, if we are not yet secure, there is no point in trying to send a new OTR message back to the client, so we simply continue at the top of the while loop again. This way we avoid writing replies to the client when session establishment messages are sent by libotr on the client side.

This might seem a little strange at first, how will we ever become secure and start replying to the client if all we do is read from them and throw away the messages. The thing to keep in mind is that messages sent with inject_message() on the client will be seen by libotr when we call otrl_message_receiving() which in turn might cause libotr on the server to inject_message() with a reply to this session establishment message. Eventually libotr will call the gone_secure() OtrlMessageAppOps callback in which we set the global variable secure to true, this allowing the server to start replying to the client as it normally would.

void server_session( h_iosockstream streamptr )
{
    iosockstream& stream = *(streamptr.get());
    while( stream )
    {
        gcry_error_t et;
 std::string s;
 VMSG << "getting more data from the client..." << endl;
 getline( stream,s );
 VMSG << "READ:" << s << endl;
                    
 void* opdata = &stream;
 OtrlTLV* tlvs = 0;
 char *newmessage = NULL;
 int ignore_message = otrl_message_receiving(
    userstate, &ui_ops, opdata,
    accountname, protocol, recipientname,
    s.c_str(),
    &newmessage,
    &tlvs,
    myotr_add_appdata, &ui_ops );

 VMSG << "ignore:" << ignore_message << " newmsg:" << maybenull(newmessage) << endl;
 if( newmessage )
        s = newmessage;
 otrl_message_free( newmessage );
 if( ignore_message )
 {
     VMSG << "libotr told us to ignore this message..." << endl;
     continue;
 }
                
 cout << "ignore:" << ignore_message << " server got:" << s << endl;
 cout << "message from client:" << color(s) << endl;

 // do not echo back messages when we are establishing the session
 if( !secure )
     continue;

The remainder of server_session() creates the echo reply message, encrypts it with otrl_message_sending() and sends the OTR message over the socket.

  static int count = 0;
  stringstream zz;
  zz << "back to you s:" << s << " count:" << count++;
  s = zz.str();
  cout << "writing...s:" << s << endl;
  cerr << "send plaintext:" << colorsend(s) << endl;

  et = otrl_message_sending( userstate, &ui_ops, opdata,
     accountname, protocol, recipientname,
     s.c_str(), tlvs, &newmessage,
     myotr_add_appdata, &ui_ops );
  if( !ok(et) )
  {
     cerr << "OTR message_sending() failed!" << endl;
  }
  if( ok(et) && !newmessage )
  {
     cerr << "There was no error, but an OTR message could not be made." << endl;
     cerr << "perhaps you need to run some key authentication first..." << endl;
  }
  if( newmessage )
  {
     VMSG << "have new OTR message:" << newmessage << endl;
     s = newmessage;
  }
                
  VMSG << "writing otr...s:" << s << endl;
  stream << s << endl;

As the security of the OTR messaging relies on fingerprints in the first example, the new_fingerprint callback presents our fingerprint and the remote fingerprint and asks the user if they want to continue to establish the session or not. Unforuntately this means the user has to eyeball scan the remote fingerprint against an expected value they have obtained from the remote party at some other time in a secure channel.

static void myotr_new_fingerprint( void *opdata, OtrlUserState us,
                                   const char *accountname, const char *protocol,
                                   const char *username, unsigned char fingerprint[20])
{
    cerr << "myotr_new_fingerprint(top)" << endl;

    char our_fingerprint[45];
    if( otrl_privkey_fingerprint( us, our_fingerprint, accountname, protocol) )
    {
        cerr << "myotr_new_fingerprint() our   human fingerprint:" << embold( our_fingerprint ) << endl;
    }
    
    cerr << "myotr_new_fingerprint() their human fingerprint:"
         << embold( fingerprint_hash_to_human( fingerprint )) << endl;
    cerr << "do the fingerprints match at the remote end (enter YES to proceed)" << endl;
    std::string reply;
    getline( cin, reply );
    if( reply != "YES" )
    {
        cerr << "You have chosen not to continue to talk to these people... good bye." << endl;
        exit(0);
    }
}

Simpler authentication with SMP

The second example uses the SMP to avoid having to verify fingerprints. For good measure, the fingerprints established are saved and loaded to/from disk so that subsequent conversations do not need any SMP or user fingerprint verification.

During process startup, fingerprints are read from file if they exist;

 std::stringstream fn;
 fn << "fingerprints-" << accountname;
 gcry_error_t e = otrl_privkey_read_fingerprints( userstate, fn.str().c_str(), 0, 0 );

The otrl_message_sending() and otrl_message_receiving() functions both have a parameter OtrlTLV *tlvs. The tlvs allow data to be sent and received as sideband information that does not effect what you send with libotr. The SMP uses the tlvs to communicate the information that it needs in order to authenticate.
In server_session() the main change is a check on the tlvs variable after calling otrl_message_receiving().

 if( tlvs )
 {
    handle_smp( stream, tlvs, userstate, &extended_ui_ops, opdata );
 }

The client initiates the SMP and has heavier changes to it's code. After creating a iosockstream to localhost, the client calls run_smp_client() to setup the OTR session and run the SMP to authenticate. Apart from the call to run_smp_client() the client mainloop while(true) doesn't need to change. This makes sense because the SMP is normally only used at session establishment when we do not know about the remote key (fingerprint) already.

In the run_smp_client function, the first while( !secure... loop will establish an OTR session using fingerprints just like the first example. This time we do not stop to ask the user to verify the fingerprints, we simply record that a new fingerprint was seen. This is done by setting runSMP=true to force the SMP if we are using a fingerprint that we didn't already have on disk.

If runSMP is set then we read a secret from the user and call otrl_message_initiate_smp() to get the SMP ball rolling with libotr. This leads to the second while( !secure loop which will stop when we are secure again.

void run_smp_client( iosockstream& stream )
{
    void* opdata = &stream;
    OtrlTLV* tlvs = 0;

    // establish session using fingerprints
    stream << "?OTR?v2?" << endl;
    usleep( 200 * 1000 );
    while( !secure && stream.peek() != std::iostream::traits_type::eof() )
        client_read_msg_from_server( stream );

    if( !runSMP )
    {
        return;
    }
    
    VMSG << "Starting the Socialist Millionaires' Protocol " << endl
         << " to work out who the other guy is..." << endl
         << endl;
    VMSG << "please give me a secret that only you and the other guy know..." << endl;
    std::string s;
    getline( cin, s );
    int add_if_missing = true;
    int addedp = 0;
    ConnContext* smpcontext = otrl_context_find( userstate,
                                                 recipientname, accountname, protocol,
                                                 add_if_missing, &addedp,
                                                 myotr_add_appdata, &ui_ops );

    cerr << "addedp:" << addedp << " smpcontext:" << smpcontext << endl;
    if( !smpcontext )
        return;
    otrl_message_initiate_smp( userstate, &ui_ops, opdata, smpcontext,
                               (const unsigned char*)s.c_str(), s.length() );

    // we are only secure if the SMP succeeds
    secure = 0;
    while( !secure && stream.peek() != std::iostream::traits_type::eof() )
        client_read_msg_from_server( stream );
        
    cerr << "secure:" << secure << endl;
    if( secure == SMP_BAD )
    {
        cerr << "couldn't authenticate server, exiting..." << endl;
        exit(1);
    }
}

The client_read_msg_from_server() function calls otrl_message_receiving() and checks if tlvs is set and if so calls handle_smp() with that tlvs value.

As you see from the above, whenever a tlvs is set in the client or server then handle_smp() is called. If you look at the UPGRADING file in libotr 3.2.0+ you will see a skeleton code in "3.3.4. Control Flow and Errors" which the handle_smp() is based on. The handle_smp() function uses otrl_tlv_find() on tlvs to check for internal OTR messages sent from libotr itself which describe a stage in the SMP. handle_smp() is like a primitive state machine working through from SMP1 (the server asking for the secret to respond to the client's initial request), through to SMP3 and SMP4 which are called when the protocol completes with either success or failure (same or different secrets).

  if( tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP2))
  {
    if (nextMsg != OTRL_SMP_EXPECT2)
    {
       cerr << "smp: spurious SMP2 received, aborting" << endl;
       otrl_message_abort_smp( userstate, ui_ops, opdata, smpcontext);
       otrl_sm_state_free(smpcontext->smstate);
    }
    else
    {
       cerr << embold("SMP2 received, otrl_message_receiving will have sent SMP3") << endl;
       smpcontext->smstate->nextExpected = OTRL_SMP_EXPECT4;
    }
  }

If the secrets are proven to be the same when the SMP is used it is adventagious to save the fingerprints to disk so that future communications do not require user fingerprint verificaiton or the SMP.

if(  tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP4) 
  || tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP3))
 {
 if( smpcontext->smstate->sm_prog_state == OTRL_SMP_PROG_SUCCEEDED )
   {
     std::stringstream fn;
     fn << "fingerprints-" << accountname;
     gcry_error_t e = otrl_privkey_write_fingerprints( userstate, fn.str().c_str() );
   }
}

Hopefully you are now in a better position to add libotr support to your real time network programs. The full source code to these programs as well as the HTML for this post itself is up on my github page. Remeber, using off the record messaging doesn't nessesarily mean you have anything to hide, just that you have nothing to show.

6 0

An ARMs race, with a core i7 in there too for relativity

Wed, 06 Jun 2012 19:32:00 +0000

After doing some power benchmarking recently (1.2ghz kirkwood with gb net up = 5watts) I decided to work out how fast these arms can do useful work^TM. In the running is a Synology ds212j, DreamPlug running the Freedombox, Nokia n9 mobile phone, and an Intel 2600k just for working out where those relative numbers sit when compared to a desktop machine.

The above image shows the cipher performance of "openssl speed" across many machines. The 2600k is only single threaded, so could be many times faster in real world use by taking advantage of the cores. One interesting point right off the bat is that the 1.2Ghz kirkwood in the synology NAS is bested by the 1.0ghz CPU of the Nokia n9. Arms is not arms.

Removing the overload from the Intel i7 2600K from the graph we see that the Dreamplug is very close to the ds212j in terms of performance.

On the other hand, the digests show a distinct advantage to the Dreamplug setup. Again the n9 has a nice little lead on the others. Since a mobile phone can perform some useful work, one should perhaps also be demanding the NAS also offer handy features as well as just serving data blocks.

The RSA sign and verify graphs both show the same large slump for the ds212j unit. So for connection heavy workloads there would seem to be a large difference to the throughput you might get for the ARM you choose. On the other hand, the Dreamplug and ds212j both have similar performance on steam ciphers. So if connections are longer term then the difference will be less.

I would love to add benchmarks for the CuBox and QNAP 2ghz (TS-219PII) NAS units. It would also be interesting to use after market software on the ds212j and see the difference.

6 1

Pragmatically unlocking an n9, possible?

Wed, 18 Apr 2012 04:09:00 +0000

I have in mind an app to use RFID over the NFC hardware on the n9 to allow the phone to be unlocked by an RFID tag. This way the password can be nice and long for locking the device, but I don't have to interact with the virtual keyboard in order to use the phone. If the phone locks, I just tap the RFID tag to it, it unlocks, and I'm away. Yes, two phase would be a logical next step, but that assumes that the "lock" on the phone is very hard to break.

Reading the RFID on the n9 is the simple part, I have that working fine with physical keys. Unlocking the phone is proving not so simple. I was hoping that MeeGo::QmLocks::setState would do the trick, but it is not to be so as the API docs explain.

Anyone stumbled on a way to unlock the n9 from a program?2 2

Support libferris, get at ferris through KIO, and read about OTR messaging

Mon, 02 Apr 2012 17:24:00 +0000

I thought I'd see how well some pledgie tasks would do in the wild. Some of this code is stuff I've been wishing to write for a while but unable to make the time.

If you want to get at some of the funky filesystems offered by libferris from your KDE desktop, you might like to support my "KIO Slave for libferris" pledgie:

If you've always wanted to add support for Off-the-Record messaging to your project, or help entice somebody else in that direction, I have a tutorial article on offer here:

If you want a REST interface for libferris, listing directories and getting at files over HTTP/HTTPS, you can help make that happen here:

And finally, last for today but not least, if you want to get at jpeg images on your Zoneminder server using nice normal command lines like the following:


$ fcat zoneminder://server/monitor | okular -

Then you might like to throw a little loose change at the zoneminder plugin pledgie:

0 4

N9: Two socks VPN solution

Sat, 31 Mar 2012 00:51:00 +0000

I always tend to lean more towards SSH for VPN stuff than other tools like openvpn. With this mindset I was working out how one might get access to servers on the Intranet while on the move. Having created a bunch of nice little apps that do things on my network is all well and good, but I wanted to also use them while on the move.

My current solution is to use two socks servers on the phone. It might seem strange to need two, but this allows routing depending on source address and destination address. The first SOCKs server is dante which routes all "normal" traffic directly to the chosen Internet connection on the phone. Traffic which wants to go to selected addresses on my Intranet is instead forwarded to a second socks proxy on the phone. This second socks proxy is implemented with ssh "DynamicForward" to send traffic over the SSH link to the Intranet.

The glory here is that Internet traffic goes directly, and Intranet traffic is sent over an encrypted pipe to the Intranet. If on the other hand I connect to any router on the Intranet, the phone gets a local IP address and dante is instructed to "route" traffic from that local address directly to the target instead of to the second local ssh socks server. Thus I don't have to concern myself with the details of which side of firewalls I am on, the phone automatically does that for me and chooses the most efficient route for packets.

Having a phone that runs so much like a proper Linux distro makes this sort of thing fairly easy to get going. No need for funky NDKs, NDAs, or TLAs.

If you want to play around with this stuff I have dante debs in my repo:
http://fuuko.libferris.com/maemo/n9/
Using non privlidged ports on the phone means you can run danted as a normal user. I have an icon on the n9 home screen that makes sure dante is running instead of using init.d at the moment. Also, an icon that runs a bash encapsulated with tsocks can be handy for tinkering:
Desktop file segment:


Exec=meego-terminal -n -e /bin/sh /home/user/bin/create-tsocks-env
...

$ cat /home/user/bin/create-tsocks-env
export http_proxy=http://1.2.3.4:1313/
tsocks bash

Using a similar style Exec in a desktop file one can start the SSH client in "-N" mode and use the passphrase to connect and setup an SSH connection with a socks server on the phone. It is quick and dirty without shiny QML interfaces, but it works^TM.5 1

Zoneminder on the n9. Maybe n900 too?

Tue, 27 Mar 2012 01:16:00 +0000

My growing collection of Qt/QML applications has now hit 6 apps with the inclusion of a Zoneminder client for the n9. As I still have ideas for more apps I wonder how long until I fill half a screen with my own apps on the device ;)

Anyways, the Zoneminder app uses server side scaling and jpeg generation to keep bandwidth down and CPU/RAM very low on the device. I have modified some of the images for example purposes. The app is basically as shown but will display your real camera data instead of Koalas... Unless you are lucky enough to have some furry little friends. The first camera starts a live view right off the bat when the app starts, so there is as little finger use needed to get some useful action from running the app.

Only active cameras are listed. The play button changes the image on the right to be a 1 second rolling "live view" of that camera. The little watch moves to the recorded events for that camera which is shown below. The config is quite simple; the Zoneminder server URL, and the user/password to login to that server.

Alarm events are shown on the left side and the play button rolls through that event with the progress bar showing how far through the alarm you are. Naturally both the lists scroll to show more cameras or events if there are more than 5. By default only 25 of the most recent alarm events are shown in the list. A future expansion might allow paging back to see older ones.5 1

Ferris on the n9: Search by URL, content, and mtime

Sat, 24 Mar 2012 18:10:00 +0000

The n9 libferris app now allows you to search by the URL of the file as before, but also now by the file text content and it's modification time. Query type is selected by a button in the top right corner which unfortunately isn't nearly as easy to read in the video file as it is in real life.

As libferris handles extraction, update, and storage of metadata from disparate locations I have also added a sprinkling of what that means into the video. Notice that the first search by URL shows a comment "REST interface to libferris" in bold. This is simply the "annotation" metadata of the file but is much more interesting to the searcher than it's URL. Likewise in the second query, which finds a Gutenburg text file by searhing on text content, the annotation offers the name of the book that the file contains. Again much more interesting content to the human who is at the helm.

The third query is on the modification time of the files. There are three ways offered to perform a time search, "more recently than this" or "than last" which can have month or day as options for example, and modified >= X months ago which obviously wants a number as the query text. When querying by time like this, libferris happily accepts some human readable terms like "begin this month" as the time you specify. This makes it just as convenient to use search in scripts as in the dedicated front ends like the n9 app.

The plan is as always, to push metadata from being an after thought to being a first class citizen: able to be created, read, written, indexed, and searched on. Any file with metadata should be able to expose that as simply as its mtime or size, which is all currently done via a key-value "Extended Attribute" interface at the lowest level.

If you like libferris or the recent updates (status.net mounting, these indexing tweaks etc) then please consider making a donation. If you want to use this technology at a corporate level, please feel free to contact me.4 1

Libferris on the N9: JSON, REST, QML, Index and Search, VFS all together

Thu, 22 Mar 2012 07:55:00 +0000

Continuing my n9 apps I now have an index search app that uses libferris. Currently I have only exposed URL regular expression search. This is searching an index maintained and stored in a PostgreSQL database. Solving regex query in a timely manner is a fairly complex problem with one useful solution outlined in one of my previous blog entries.

Note that data is downloaded over normal http(s) from the server using the n9 app, so no NFS or other network mounts are needed. This might be handy for grabbing a PDF, image or text file off an Intranet file server while on the move.

I have also been updating the clucene libferris module to allow more effective index use during regex/infix queries. This libferris engine works quite nicely on older ARM machines. Benchmarks on that to follow, once I reindex the arm and produce the comparative figures.4 1

QML BOM: Weather on the go...

Wed, 14 Mar 2012 17:29:00 +0000

As part of my foray into n9 apps, a new one that displays information from the Australian Bureau of Meteorology on the n9. Being pure QML + Javascript it also runs on the desktop. The forecasts include min and max temps, and for the Europeans reading this, the middle line "10..." is the UV prediction and roasting window for the day... all your sunscreen are belong to us.

Apart from saving a little on downloads for the topology and background images when on the move, you get to customize those as they are just png files living in the images subdirectory. So I have the local rocket launch site indicated. I'd like to make a LOTR style topology overlay, perhaps mt Glorious becomes Mordor?

Code available at https://github.com/monkeyiq/qmlbom4 1

N9, QML, Javascript and differential update.

Mon, 12 Mar 2012 00:59:00 +0000

I had a server app that had both a desktop and web interface. Though I discovered that the web interface was less than monkey finger friendly on the n9 and required some authentication each time. I thought this was a job for QML and javascript on the device. The good news is that I now have a client using just those tools, so no cross compilation needed at all ;)

The core of the UI is updated via a timer which runs an ajax call, parses the JSON response and gives that to updateModel(). The first implementation of the later method was to clear the listModel and repopulate from scratch each time. Rather wasteful, even though it was only being called once every 2-3 seconds, it was eating up 8-9% CPU of the n9 in top. A better solution is to iterate the existing items, performing update, remove, or add as you go. This needs a "key" field for each tuple and a stable total order on the model JSON data, either from the server or through an explicit client side sort.

I'll probably put this up on github soon, but in the meantime, this is the updateModel() which has the smarts I mentioned above. It takes 3-4% of CPU instead, and should maintain view state better on the device. I use objid as the key, and the "state" is cloned to objstate because I noticed some issues there with QML interaction using state as the property. Interesting what one can do with QML+JS in very short time for the n9.


    function updateModel( model, data ) 
    {
 var existingIndex = 0;
 var existingIndexMax = model.count;
 var id;

 for( id in data ) 
 {
     var found = false;
     var name  = data[ id ];
     var col   = data[ id ];
     var k;
     var row = {};
     row[ "objid" ] = id;
     row[ "name" ] = "v";
     for( k in col ) 
     {
  if( k == "state" ) {
      row[ "objstate" ] = col[k];
  }
  else {
      row[ k ] = col[ k ];
  }
     }

      // find it if it exists...
     for( ; existingIndex < existingIndexMax; ++existingIndex )
     {
  if( model.get( existingIndex ).objid == id )
  {
      found = true;
       model.set( existingIndex, row );
      ++existingIndex;
      break;
  }
  else
  {
      // This element from the local model no longer exists on the server,
      // remove it. Do not advance the index as we have removed an element instead.
      model.remove( existingIndex );
      --existingIndex;
  }
     }

     // it didn't exist, add it as new.
     if( !found )
     {
  model.append( row );
     }
 }
    }

4 1

Libferris on both arms

Thu, 08 Mar 2012 04:07:00 +0000

I now have libferris on a 512mb ARM5t embedded device and a 1gb arm7 one (the Nokia n9). As part of the fun and games I updated the clucene and boostmmap index modules for libferris and created a QtSQL module which uses SQLite by default. Hey, when you can't choose which cake to have why not have a slice of them all... or a slice to fit the moment or device at hand. eg, desktop or server works nicely with the soprano or postgresql indexing modules, embedded is nicer on mmap or clucene.

I find it ironic that I never really thought much of "embedded" devices when hacking libferris. But devices with 512/1024mb of RAM are really not so much embedded I guess.

A few things I've tried to do in the design of libferris index+search is to design for many machines and also federations of them. It is possible to search a router, phone, and desktop's individual indexes as a federation from the laptop. Another thing that helps is that indexing is routed through the "findexadd" commands. So you can use find and split to break up indexing activity, and have it done from cron when you want.
The new --total-files-to-index-per-run option works in combination, causing findexadd to exit when it has indexed a given number of files. Note that if a file has not changed since it was last indexed it is not indexed again (no need), so that file does not count toward the
total-files-to-index-per-run tally.

The below is a little script to incrementally index just selected metadata from /usr and your home directory using clucene. The WHITELIST environment variable stops libferris from trying to sniff up metadata for files and has it only look for and add what metadata you want. If you have md5 in there then libferris will store the checksum for each file, at a commensurate cost in IO. Splitting into batches of 5000 prevents the process running too long and wanting too much RAM.

$ mkdir -p ~/.ferris/ea-index
$ cd ~/.ferris/ea-index
$ fcreate --create-type eaindexclucene .
$ vi update.sh
#!/bin/bash

TMPDIR=~/tmp
EAIDXPATH=~/.ferris/ea-index

cd $TMPDIR
find /usr | split -l 5000 - usr.split.
find ~ | split -l 5000 - home.split.

export LIBFERRIS_EAINDEX_EXPLICIT_WHITELIST=name,size,mtime,mtime-display,atime,ctime,user-owner-name,group-owner-name,user-owner-number,group-owner-number,inode
echo "whitelist: $LIBFERRIS_EAINDEX_EXPLICIT_WHITELIST"
cd $EAIDXPATH
rm -f write.lock
for if in $TMPDIR/*split.*
do
echo "Processing $if"
cat $if | feaindexadd -P `pwd` -1
done

Ironically the arm5 has given me much less trouble overall. One issue seems to be with gcc-4.4x on the n9. Charming little errors like my old friend the undefined __sync_val_compare_and_swap_4 which stops memory mapped boost data structures from working properly and also leaves the clucene-core-2.3.3.4 build laying on the side of the road bleeding. I've hacked the clucene code to get around the atomic errors, but then seem to have found that search results are not accurate. I guess my quick hack there was just bad^TM. Especially since the arm5 produces the right results using the virgin clucene codebase.

I've been trying to convince gcc 4.5 and 4.6 to build for me so I can use the updated compiler to generate a proper and working clucene for the device. I seem to run into little build issues after time consuming rebuilds. (uses VFP register arguments, yay). Once I stop compiling compilers then maybe I can get my favourite indexing code on the n9.1 3

The Nokia N9: Going for a spin

Sun, 26 Feb 2012 15:55:00 +0000

In short, I now have a Nokia n9, and setup a scratchbox for it, and compiled & installed abiword, clawmotia, unison, and libferris for and on the device. If that didn't enlarge your page down key, the longer version now follows ;-)

FWIW my impression of the device is very positive. Hardware wise the biggest issue I have is the inability to add more storage to it. A high gb microsd slotted in would make things a whole bunch nicer, mainly for using the device as a music box on the road. The speed of interaction is very nice, and the swipe interface is very easy to get used to. Just flick an app away to minimize it or flick from another screen edge to close it.

gstreamer works quite nicely for viewing network streams, though there is a deadzone bar on the logical "bottom" of the screen (left of screen in portrait mode). I noticed this bar in abiword too, so I assume it is an issue with going fullscreen for non Qt native apps.

There isn't much to show for unison, as I tend to drive that from the dekstop/laptop end of the connection. Getting it on the device is always a matter for first compiling ocaml and installing that into the scratchbox.

One very pleasant surprise is how snappy clawmotia runs on the n9. Startup time is significantly less than on the n900 and the screen operations like swiping the main controls left to get to the jog shuttle page is now at an acceptably smooth speed. I'll have to work up a video of this in action to show the speed of things...

I got libferris compiled for the n9 too. Using boost::spirit for some of the parsers has dropped one compile time dependency away. I did make both ferris and ferriscreate able to compile in glib2 only mode. You loose one or two apps by doing this, but it also makes the footprint smaller so that I can use it with QML apps without needing to pull in gtk2 from a third party repo just for those apps in ferris itself. Mainly some capplets, gfcreate, and the gfcp and other graphical coreutils reimplementations are lost in building this way.

And what is not complete without a screenshot of things on the device? Sorry for the quality of the picture, I need to work out lighting for some device shots a bit better than this adhoc quick shot. Just to tap into the ferris power a tiny bit I decided to mount an XML file as a filesystem and fcat directly into it.. A few of the fun things I plan are a QML client for index/search using the customized maemo index backend in libferris and using the mounted pulseaudio and mounted gstreamer to implement yet another audio player for the device. The index+search in both these apps should be blisteringly fast, given that I could index an NFS mount with 10x the data I could hold on an n810 and search as you type in real time...

Abiword proved to be the harder thing to compile for the n9. It seems GTK2/3 has fallen off the device so I got gtk2 from a third party repository. The downside is that the app doesn't run entirely fullscreen and has pretty much no finger friendlyness from gtk2 itself. Oh yeah, the virtual keyboard doesn't come up on text entries in gtk2 either. I might be compiling and installing gtk2 incorrectly on the device, so this complaint about device interaction might be my own fault?

I then got libwv and libwmf, following the dependency rabbit down the white hole. Of course my abiword compile has full ODF support in it too! The below screenshot is of one of my sample ODF+RDF documents with the content that links to RDF shown in purple. For more info on RDF in abiword see the abiword talk video from lca 2012.

More fun an games to follow, and I'll post a link to a repo with the debs once I work out where I'm going to host the 100mb odd of files. Not all those are for the device, some are mainly for scratchbox such as the dev packages and full ocaml compiler debs.4 1

Mounting statusnet/identica as a filesystem

Wed, 22 Feb 2012 17:13:00 +0000

With the 1.5.12 release of libferris hot off the presses, I thought I'd blog about how to use the new statusnet mounting. I tried to do some testing against identica at the start, and managed to get myself silenced there ;) I was resurrected within hours once I put forward my case as not-a-trollbot^tm. Anyway, that led me to setup my own local statusnet server for testing against, and now libferris can mount identi.ca and also one or more local servers if you so desire.... yay!

There is a little bit of setup required, web keys, secrets and oauth... the normal story.
The identica filesystem contains a directory for each server you have setup in ~/.ferris/identica. To update my public status I just write to the status file in my identica account:

$ echo "this is coming from the console via libferris, mounting the public statusnet ;-p" \
| ferris-redirect identica://identica/status

To see what has been going on I can list the timeline directory, using "-0" to get libferris to tell me what metadata is interesting for that directory instead of viewing the protection bits and other non interesting information.

$ fls -0 identica://identica/timeline
89868729 Welcome to Identi.ca, @monkeyiq! Welcome Bot 12 Feb 2 03:53
...

There is also support for direct messaging through the filesystem, seeing retweets, friends, and mentions.

Now for the details... Configuration happens through the virtual filesystem. Each statusnet server you want to mount has it's own directory inside ~/.ferris/identica. The configuration directory lists the base URL for performing REST requests on that server and the prefix of where the webkey and secret are to live. For the public identi.ca server I have the following:

$ pwd
~/.ferris/identica/identica
$ cat rest_base
https://identi.ca/api/
$ cat token_prefix
identica

And the webkeys go in a file in ~/.ferris prefixed with the token_prefix above:
$ cat ~/.ferris/identica-api-key.txt
5045332...

I do not distribute web keys, so you need to go to http://identi.ca/settings/oauthapps and create one yourself.

If you have a private statusnet server foobar then you create ~/.ferris/identica/foobar with the REST url for it, and foobar as the token_prefix. Then put the web API key into ~/.ferris/foobar-api-key.txt.

To setup the oauth part, we use our old friend capplet-auth. Again for our foobar custom server replace the auth-with-site with foobar instead. This will give you a URL to visit to authorize your webapp (libferris) on your account. You'll be given a PIN to copy back onto the console and then you should be setup.

$ ferris-capplet-auth --auth-service identica --auth-with-site identica6 0

Searching again (400ms -> 2ms)

Mon, 02 May 2011 03:12:00 +0000

The problem statement: you want to search for a file using any arbitrary substring contained in the file url. This is deceptively hard, if you think in a relational db way, you want something "like %foo%" where the leading percent will force indexes to be considered redundant and a sequential scan to ensue. If you try instead to break up the file's URL into words and use a full text indexing solution you will find your old friend the leading percent again, like here for Lucene.

One easy way to use full text indexing style and allow substrings is to cheat. First disable word stemming (ie, dont turn diving and dive into a stemmed "dive" word). Then for each word to be added to the full text index, shift it left from 1..10 times. So wonderful, onderful, nderful, derful, erful etc are all added as words for the same URL. This is most simply accomplished using a postgresql function to do the shifting.

Then when you search for a substring foo you want to find to_tsquery('simple,'foo:*') as a fulltext query on your url column. As prefix searches are allowed in fulltext index engines like Lucene and postgresql's TSearch2 engine, this evaluates quite quickly.

For 200,000 URLs when using the raw regex match "~" in postgresql you might see 400ms evaluation times, the same data using a gin index on to_tsvector('simple',fnshiftstring(url)) might return in 2-3ms. Of course the index can't always handle your regular expression, but if you can tease out substrings which must be present from your regular expression, a shifted gin index could drop evaluation times for you. eg, .*[Ff]oob([0-9]... could use a shifted search for "oob" as a prefilter to full regular expression evaluation.

Getting down to a few ms evaluation allows GUIs to return some sample results as you type in your regular expression. This speed up will be available in 1.5.6+ of libferris.

A while ago I released an engine for maemo with statistical spatial indexing for regular expression evaluation. It's an interesting problem IMHO, and its also a very common one.3 0

Change Tracking: Why?

Sun, 24 Apr 2011 17:40:00 +0000

As some of my past posts have mentioned, the OASIS group is currently working out how it wishes to extend support for change tracking in ODF. The change tracking feature allows you to have an office application remember what changes you have made and associate them with one or more revisions. There are many examples where governments might want such traceability, but small businesses are likely to find this functionality valuable too.

Late last year there was an ODF plugfest held in Brussels where it was decided that an Advanced Document Collaboration subcommittee should be formed to work out how to serialize tracked changes into ODF.

There are currently two proposals. One which is generic and tracks how the XML of the ODF is modified over time, and an extension to the limited change tracking that already exists in ODF 1.2.

As my previous posts have mentioned, I'm involved in making abiword able to produce and consume ODF files which contain tracked changes in the format of the generic proposal.
See my github for abiword and its change tracking test suite. Ganesh Paramasivam has been working to make Calligra and KOffice support the generic proposal too, and Jos van den Oever has done some hacking to enrich WebODF for change tracking.

As I have been going along implementing things in abiword, I've been clarifying some points on the oasis office-collab mailing list, and participating in some of the general conversation there too.

Some of the differences between the two proposals can have a large impact to the complexity of implementing change tracking in applications. Folks who are involved in office applications which use ODF might like to read up on the proposals and have their say on the future of this feature before a decision is made for you.

I think it is important for ODT to include a good, complete change tracking specification because it would offer all people the ability to collaborate on documents and businesses the change to know the exact extent of modifications made to documents by each party.5 0

From Canon CR2 Raw files to the Web

Thu, 20 Jan 2011 21:28:00 +0000

A potentially interesting artefact of me playing around with a Canon DSLR is that libferris now has some initial support for CR2 "raw" files. Complements of course go to the dcraw library and tools for the heavy lifting. The upshot is that ferriscp ~/cam/foo.CR2 flickr://me/upload now works as expected. Once I update my maemo build, this should also let me use an n900 with 3G to upload from RAW while on the move.

I need to include stuff for ICC profiles and the like but at least the Web services get what they expect and everything runs smoothly, and it will for you too next release (TM). I really should look into mounting the kipi plugins as a filesystem sometime, they offer some awesome Web import/export functionality, ripe for a filesystem I tell's ya.

I still need to add "rgba-32bpp" metadata support for CR2 files. As you can imagine from the name, if you read this metadata for a jpeg, png or other supported image file, libferris decodes the image to 32 bit RGBA byte values for you. It might also be interesting to create another "file as filesystem" design where a bunch of virtual files are offered to see the small thumbnail, the roughly 1080p sized thumbnail or the ICC corrected rendering of the CR2 and others. Such virtual files are really handy for drag and drop to other filesystems ;)

I might hunt around for some Nikon raw files to add support for those too at some point. Though there is nothing like having a camera generating files to make you want to make your software support those files.5 0

Sharing is Caring...

Mon, 15 Nov 2010 15:46:00 +0000

My repository now contains a steaming ferris-sharing-plugin_0.2_armel.deb which will hook up libferris to the Nokia sharing system on the n900. This lets you click the funky Brussels atom building "sharing" icon from many places and copy images to flickr, 23hq, facebook.

There isn't much feedback at the moment, that is planned for "the future". You might also like to install the ImageMagick from my repo. Simply expand it to /opt and make a link /usr/lib/ImageMagick-6.6.0 -> /opt/ImageMagick-6.6.0/lib/ImageMagick-6.6.0. This gives you proper handling of exif orientation during scaling prior to upload with libferris.

I also have some tweaks to libferris itself which will be in the upcoming 1.5.1 release to help with annotations etc during upload. The next step is to get vimeo and youtube upload support happening too, since libferris can already upload to those.3 0