Brainstorm

Developers should get karma based on the relevance of their software

Posted on 2009-09-10 19:35 UTC by Quim Gil. Status: Under consideration, Categories: maemo.org.

Currently the developers receive karma basically based in how many Garage projects are maintaining or involved with. This is actually a not very relevant statistic: one developer might have only one project making happy to 100.000 users (see Mplayer) while other might have opened 14 garage projects for 14 command-line direct ports of Debian that actually nobody uses or could care less about.

This is not easy to address as there are many possibilities to be unfair e.g. several developers in a project with different degree of involvement, apps downloaded by thousands that are "easy" ports of projects developed hardly by someone else. I wouldn't make big fuzz of this karma for developers, but it would be good to take it into account somehow.

Currently it's easier to get plenty of karma blogging about apps, talking about apps and commenting bugs in apps... but at the end are the dvelopers who carry with a lot of the hard work. Let's praise them!

Discussion thread: http://talk.maemo.org/showthread.php?t=31519

Solutions for this brainstorm

0
0
0

Solution #1: Take downloads into account

Posted on 2009-09-10 19:42 UTC by Quim Gil.

Say 100 downloads = 1 karma point

This is an amount small enough to try to tweak it with fake manual downloads (and if you play dirty with fake automatic downloads it will be vident at some point). See http://maemo.org/downloads/downloads/OS2008/25/ for a reference.

0
0
0

Solution #2: Take valoration into account

Posted on 2009-09-10 19:21 UTC by Quim Gil.

If [stars] => 3 then karma = [stars] * [votes]

Apps with 0 / 2,99 stars don't generate any karma since these ratings are considered low / bad.

0
0
0

Solution #3: Karma from applications is distributed among maintainers

Posted on 2009-09-10 19:57 UTC by Quim Gil.

The total amount of karma points received from an application through downloads, valoration, etc would be divided in equal parts among the maintainers of the application. Maintainers are identified at http://maemo.org/packages (the maemo.org admins might know a better way to find out who are maintainers).

The equal distribution among maintainers is probably the most fair and simple way. Maintainers have usually full permissions on packages and code base: usually these permissions are not given lightly just to anybody. It also balances efforts: a single developer of a medium-popular app would get more or less the same points than a maintainer of a larger team of a top popular app.

What about contributors not maintainers? Let's not complicate things too much, specially not in the beginning. No karma for them, contributing to a project should be gratifying in itself. This way we also avoid pressure to the existing members of a team from newcomers thirsty of karma points.

0
0
0

Solution #4: Karma from applications is distributed among maintainers based on their SCM adds/updates in the project

Posted on 2009-09-10 19:56 UTC by Kamen Bundev.

This seems like a more fair karma distribution, since there can be added developers with no contributions. Further the project admin should be able to additionally move some points between the developers up to a certain extent (not all of the points should be distributable).

Of course this a more complicated approach and probably should not be implemented from the start.

0
0
0

Solution #5: Combination of ratings and distribution among developers but per version

Posted on 2009-09-10 21:09 UTC by Reggie Suplido.

This proposed solution is combining solutions #2 and #3but factoring in versioning. The aim is for better apps, so if the developers themselves tell end-users to rate their apps as well as provide feedback, ratings and comments becomes then help improve the apps. When rating apps, it should be by version (major release only). A version 1.0 of an app can get 3 stars but on the next major version release, say 2.0, it can get 5 stars. As for distribution of karma, it should be by version (major release only) too. Version 1.0 karma can have 3 developers sharing, but for version 2.0, there can be 5 developers sharing.

Click on the screenshot to see a larger one!
0
0
0

Solution #6: Measure existing userbase via updates

Posted on 2009-09-14 16:38 UTC by Attila Csipa.

When measuring just downloads, there is no indication how many users liked the software and how many have uninstalled it. When a new version is released, however, there is a spike in downloads that shows exactly this. The volume of the spike is the approximate numbers who still have the application installed. This way we can measure an active userbase, even if in an indirect form and use that as part of the karma calculation. Yes,  some people never update, but the point is that the metric will be equally wrong on all projects, so you do get a realistic comparison of userbases.

0
0
0

Solution #7: Use a modifier based on level of 'Maemoness'

Posted on 2009-09-14 21:04 UTC by Attila Csipa.

A list of categories could be established that would modify karma value for a package. The categories could be 'Maemo compatible' (as something coming straight from Debian, for example), 'Maemo port' (something that has been hildonized and adapted for use in line with other Maemo apps), 'Designed for Maemo' (something that has been written exclusively or primarily for Maemo). Of course more categories could be introduced to make the scale more fine-grained.

0
0
0

Solution #8: work with ohloh

Posted on 2009-10-11 08:33 UTC by Josef Assad.

As the solution title says. There's several reasons why this might be a good idea, and a few why it might be bad.

Pro(s)

  1. Ohloh have been thinking developer karma for longer than maemo have or will. It's their expertise domain and developers acept the ranking system.
  2. A lot of the developers I know would consider ohloh competitive ranking a stronger incentive than a separate maemo community karma ranking simply because of the sheer number of people on ohloh.
  3. I can't speak for ohloh but I imagine considering their raison d'etre they would be happy to get maemo development statistics piped into their system.

Con(s)

  1. A quick skim of Ohloh's API documentation suggests that existing facilities are mainly for data retreival. Ideally, hooking the maemo development community into ohloh would be a more bidirectional data flow. This solution is probably going to require more than just the right code. Actual people will have to talk to actual other people (shock, horror!)

Implementation details

  1. It would be good if developer karma rankings could be seen on ohloh against the whole developer body and also against the subset which is the maemo developer body.

In similar vein, Stackoverflow.com and superuser.com could also be hooked into to provide facilities similar to, say, launchpad's answers.

There's probably more good and bad reasons around this proposed solution but it's a Sunday morning and I've only had 2 coffees.

0
0
0

Solution #9: User ratings

Posted on 2009-10-14 18:01 UTC by Joseph Pingenot.

The following Solution requires modification to the site software in addition to modifying the karma system.

The basic premise behind this Solution is that user effort to give feedback correlates directly with the impact of the software on them (good or bad!)

Currently, the maemo site allows users to rate softare and comment on it. The Ratings might be extended to allow users to fill out a survey on their use of the application. The survey might consist of the following points, on a 0-5 scale (5 is completely agree):

* I use this software on a [daily/weekly/monthly/yearly] basis. (this might also be a list box instead of rating)

* This software is critical to my work.

* This software is something that I enjoy using.

Other options are clearly possible (comments below). A scoring system from the survey results could be used to convert the scores into karma.

In addition, there could be a bot email address to send user testimonials, which will be prominantly displayed on the page (e.g. a [Testimonials] tab or button). Because this is out-of-channel and requires much more work on the part of the user, it should be worth much more than any other option, and perhaps scale with number of lines.

A possible second option (which again is assumed to correlate directly with user impact) is how much they're willing to pay to keep the software going. Allow users to donate to the project, providing a tips jar for the developer. Tips -> karma with some conversion factor. This would also help attract developers to the platform, and help developers buy devices. Win-win-win, as Michael Scott would say.

Latest activities to brainstorm Developers should get karma based on the relevance of their software