Os2007On770

OS 2007 on 770 - hacker edition
OS2007 on 770 Hacker Edition - Installation instructions
Hacker Edition Modifications against normal OS2007
Reproducing the Hacker Edition using official N800 and 770 images

Extract the root filesystem image
Bluetooth firmware
Patching init.d scripts
DSP modules and configuration
alsa.conf
ke-recv
osso-ias and osso-hss
osso-ic
Opera web browser
Metalayer-crawler
Device naming from N800 to 770
osso-filemanager
MCE
Hardware buttons configuration
Xomap
Multimedia and plug-ins
VOIP configuration

Creating and installing modified rootimage
Xomap weirdness
Current state with the multimedia
Progress since the first preview version
Archives

Things that cannot be shipped to 770 users

OS 2007 on 770 - hacker edition

The goal is to run as much as possible of OS 2007 on the 770. We're not targeting the end user or aiming at production quality, but instead focusing on providing the OS 2007 platform for hacking on a 770. We mostly used precompiled binaries and only recompiled something when we had to.

OS2007 on 770 Hacker Edition - Installation instructions

The latest Hacker Edition release is packaged as FIASCO-image, just like other 770/N800 software releases. You can download it from download page. The installation steps are the familiar ones:

flasher -F SU-18_2007HACKER_4.2007.36-2_PR_F5_MR0_ARM.bin -f

Notes:

This is a complete FIASCO. There is no need to flash the latest 2006 release before flashing this one.
Using R&D mode or disabling lifeguard reset are not strictly necessary anymore. You may occasionally experience unexpected reboots while trying OS 2007 hacker edition, but overall this release should be more stable than the previous one.
This release is now using the old 2.6.16 kernel and initfs. This is because we realized that using the old, well tested base components is more reliable solution than untested (on 770 platform) n800 kernel. This caused some problems with the ke-recv and connectivity, but solving them was easier than getting osso_hss (alone) to work ;)

Known issues:

When the desktop comes up, everything seems to be frozen for 20 seconds. I expected earlier that this is because of esd/DSP problems. However, more likely reason seems to be high CPU utilization. The system is starting many services in the background at the same time...
Pausing video playback in mediaplayer cleans the screen.
WMA is not supported
Macromedia Flash is not supported. But there are other ways to show flash with 770.
Probably something more as well...

Hacker Edition Modifications against normal OS2007

Hacker edition is based on normal OS2007 binary image. The following table describes which packages have been changed and how. Everything not listed in the table have remained the same:

Package	License	Modification	Patches
xserver-xomap	Open	B	#531
xkbdata	Open	T	#525
libgst0.10-dsp	Open	R
farsight0.1-rtp	Open	T	#1197
osso-af-startup	Open	T	#526
mini-rc	Open	T	#529
libasound2	Open	T	#996
telepathy-gabble	Open	B	#1198
telepathy-stream-engine	Open	B	#1199
osso-core-config	Open	T	#1469
osso-interaction-server	Closed	R
ke-recv	Closed	RT
restartke-recv init script	Open	T	#1470
mediaplayer-ui	Closed	B
osso-mediaengine	Closed	B
metalayer-crawler0	Closed	B
osso-hss-control	Closed	R
libmediaplayer0	Closed	B
osso-mvi	Closed	R
osso-browser-opera-adapter	Closed	B
osso-browser-opera-eal	Closed	B
osso-browser-opera-dynamic	Closed	R
osso-ic	Closed	B
osso-dsp-modules-rx-34	Closed	RT
bt-firmware	Closed	R
osso-status-bar-sound	Closed	R
osso-filemanager	Closed	B
osso-startup-wizard	Closed	B
osso-applet-device	Closed	B
mce	Closed	B
macromedia-flashplayer	Closed	R
gstreamer0.10-wma	Closed	R
gstreamer0.10-hantro	Closed	R

Where:

T stands for textual changes (script or configuration)
B means patching and recompiling binaries
R means removing/replacing binaries by old ones without recompilation

Reproducing the Hacker Edition using official N800 and 770 images

If you have both N800 and 770, you can try to reproduce hacker edition using official images. These steps also describe in detail what have been changed when compared to official N800 image. You cannot reproduce changes to binary components, though (unless you use the binaries provided by hacker edition downloadable image).

Note It may be a good idea to disable lifeguard reset while following these steps.

Extract the root filesystem image

You need the root filesystem images in some form. If you have FIASCO images for both N800 and 770, you can use flasher to unpack the FIASCO image and extract the root filesystem as jffs2 image. Then you need to modify this jffs2 image. The ways how this can be done are described here: ModifyingRootImage

Unpack both N800 and 770 root filesystems somewhere. We'll call these locations as HACKER and OLD respectively.

Now we are ready to start modifying the N800 root filesystem that resides under HACKER.

Bluetooth firmware

The firmware actually resides on the initfs, but it's symlinked from rootfs. We need to change these symlinks:

cd $HACKER/usr/lib/hotplug/firmware
rm bc4fw.bin
ln -s /mnt/initfs/usr/lib/hotplug/firmware/brf6150fw.bin

Patching init.d scripts

The OS2007 version initfs cleans var-directory during boot accordingly, but this doesn't happen with old initfs. We need to modify startup scripts to make this happen. Also microphone configuration under sysfs needs patching in order to allow VOIP. Apply the patches #526 and #529 to HACKER.

DSP modules and configuration

Since we still have 770 hardware, we need to use same DSP modules as OS2006 does. Copy DSP configuration from OS2006:

rm -rf $HACKER/lib/dsp
cp -R $OLD/lib/dsp $HACKER/lib

alsa.conf

alsa.conf uses wrong devices. Applying this patch this will be fixed. Game called Marbles what comes with Hacker Edition is good program to test this problem. Apply the patch #996 to HACKER.

It's also possible to apply the patch straight to the libasound2 sources.

ke-recv

Ke-recv daemon is heavily using the new features drivers/features from the new 2.6.18 kernel. We don't currently have it, but it turns out that the old version used in OS2006 works fine. So, we just copy all the scripts and binaries over the new ones. Check the contents of $OLD/var/lib/dpkg/info/ke-recv.list and copy all the listed scripts and binaries to HACKER, overwriting the existing ones. OS2007 image contains one additional script osso-usb-mass-storage-is-used.sh, which can now be removed.

There is also patch to ke-recv init scripts. Restartke-recv (? is this misspelled) needed to patch for MMC remounting when unlocking device when keys and screen were locked. Patch: #1470

osso-ias and osso-hss

We use the old versions of interaction sound daemon and headset daemon, because the new ones depend heavily on new N800 kernel and new DSP. The old versions work just nicely :)

cp $OLD/usr/bin/ias $HACKER/usr/bin/ias
cp $OLD/usr/bin/osso_hss $HACKER/usr/bin/osso_hss

Using the old osso_hss also requires using the old version of master volume library:

cp $OLD/usr/lib/libmvi.so.1.0.0 $HACKER/usr/lib/libmvi.so.1.0.0

osso-ic

osso-ic is a proprietary component and getting it to work requires source code changes that are not reproducible by 3rd-party developers. However, using a later kernel than 2.6.16 may resolve the case without patching osso-ic: When we were trying to get N800 2.6.18 to work there was no need for patching this component.

The problem with N800 osso-ic is simply that when you are establishing internet connections (either via WiFi or Bluetooth), the connection is dropped immediately.

Note At this point you have made enough changes in order to have a bootable image. If you like, you can skip the rest of the instructions and made a new rootimage at this point.

Opera web browser

Opera components are proprietary ones that require source code changes in order to get them to work. The new libopera shipped with N800 image cannot be used with 770 at all (you'll get SIGILL). There is little you can do with the browser without sources... except to copy everything browser related from OLD. I didn't test this, though.

The hacker edition uses old libopera from OS2006, ui-level from OS2007 and the patched level in the middle (opera-eal).

Metalayer-crawler

This is a proprietary component that requires source code changes in order to work properly. Unpatched crawler will work to some extent, but it misses multimedia files placed on memory cards inserted in the 770.

Device naming from N800 to 770

Package what needs some patching are osso-startup-wizard, osso-applet-device and osso-core-config. osso-startup-wizard and osso-applet-device are propriotary components that requires source code changes in order to work properly. osso-core-config is open. Patch: #1469

osso-filemanager

This is a proprietary component that requires source code changes in order to work properly. Unpatched filemanager will work at some extent, but can't delete/copy files placed on 770 memorycard.

MCE

This is a proprietary component that requires source code changes in order to work properly. MCE is used for example to lock screen and keys. Screen wasn't locked properly so MCE needs patching.

Hardware buttons configuration

Old kernel uses different mapping for the arrow keys, which causes that the cursor keys doesn't work in X. We need to change the key mapping to get around this. The provided patch #525 contains the new key mappings.

Xomap

The XSP support in the Xomap server is broken, which causes the server falsely to report that the DSP area has been overrun. This causes unwanted pauses to the video playback. So, we need to patch and recompile the X-server. The required changes are provided as patch #531. After recompilation, copy the new server binary over the old one.

Note This fix is needed for multimedia only. 3rd-party developers cannot patch the multimedia framework, since it requires sources.

Multimedia and plug-ins

Mediaplayer related packages mediaplayer-ui and osso-mediaengine require source code changes in order to work. These steps cannot be reproduced without source code. If one could get xv extension to work on 770, then it would be possible to use N800 versions of these components directly. However, this could make the performance quite bad and decoders would be needed.

Hacker edition uses the old gstreamer codecs that use DSP for video playback. Configuring these codecs is possible without sources, but this is not enough without changes to binaries as well.

N800 rootimage will contain GStreamer plugin /usr/lib/gstreamer-0.10/libgsthantro.so. This plug-in is not shipped with hacker edition image and it cannot be used with 770 (SIGILL). Remove it, if you have a N800 based rootimage. Also remove the filename from /var/lib/dpkg/info/gstreamer0.10-hantro.list.
N800 rootimage will contain GStreamer plug-in /usr/lib/gstreamer-0.10/libflu*.so. This plug-in is not shipped with hacker edition image and it cannot be used with 770 (SIGILL). Remove it, if you have a N800 based rootimage. Also remove the filenames from /var/lib/dpkg/info/gstreamer0.10-wma.list.
We need to use same DSP-based GStreamer plug-ins as 770 does (a good thing is that they are binary compatible):
- cp $OLD/usr/lib/gstreamer-0.10/libgstdsp* $HACKER/usr/lib/gstreamer-0.10/
Check that the symlinks are OK (N800 may have X.so while 770 has X.so.0.0.0). Remove N800 version and make a symlink).
cp $OLD/usr/lib/libgstdsp{audio,video}* $HACKER/usr/lib and do the same symlink check.

Now the plug-ins are installed and the thing remaining is to tell osso_media_server to use them. The patch is available here. Basically we need to

Remove references to hantro and xvimagesink plug-ins that we cannot use
Use dspmpeg4sink and dspfbsink instead.
Configuration for other DSP codecs that are not available on N800.

VOIP configuration

Small hack is needed to media streaming configuration. Since we removed hantro-plugin, the media streamer will abort when it tries to create codecs. Apply the patch #530.

Creating and installing modified rootimage

After finishing modifications, you can turn HACKER directory to a flashable jffs2 filesystem image:

sudo mkfs.jffs2 -r $HACKER -o rootfs.tmp -e 128KiB -l -n
sudo sumtool -i rootfs.tmp -o rootfs.jffs2 -e 128KiB -l -n

You need to install the needed utilities to your desktop. Now you can try:

sudo flasher -r ./rootfs.jffs2 -f

Xomap weirdness

Looks like that the xserver-xomap that is shipped with n800 (1.1.99.3-0osso21.1) has buggy XSP support:

* omapBoxesIntersect returns exactly the opposite (the boxes do not intersect).
* omapResetDamage + omapRegionIsNull do not handle the empty set correctly: The meaningless initialization coordinates easily end up to the final damage area as well.

Thease are trivial to fix, though, but since the bugs partially hide each other that was difficult to discover.

The fact that Xomap combines the damage areas to a single bounding box rectangle and flush that to display at one go causes more problems: When DSP is enabled it easily happens that none of the damage areas intersect the DSP area, but the bounding box intersects. This causes unwanted pauses to video playback. I tried to fix this by not combining the damage areas in a case DSP was enabled. This can cause slightly more ioctls, but there shouldn't be so many other screen updates anyway while the DSP video is playing. When the DSP is not in use, everything works as earlier. This change made porting the new mediaplayer to be a much easier task :)

Current state with the multimedia

We are working hard to get the multimedia support of the "hacker edition" to acceptable level. It looks like that we mostly have to use the old gstreamer plug-ins and osso-media-server, since the new system relies on X overlays, which currently are out of reach for the 770. We still believe that the new mediaplayer UI is reachable, though it requires some extra work. Osso-media-server causes that there is same limitations as OS2006 has.

Basically the multimedia for the hacker edition is likely to be as follows:

At Gstreamer/DSP level hacker edition will be identical to old OS2006.
Hacker edition will contain patched versions of old osso-media-server, new mediaplayer-ui and new X server.
Instead of X overlays, the system will use XSP to detect DSP area overrides and pauses DSP when needed.

Currently system should play MPEG4s nicely :) The unwanted video pausing expected earlier was caused by a bug in XSP implementation within X-server. Nowadays pausing works as expected \o/. There might be other problems.

Progress since the first preview version

The following additional features are now working:

Locking the device
Filemanager works as it should be
Some internet call fixes
- Shouldn't hang when camera is popped out from N800
Device information is now correct
Fixed certification problem with browser
- Remember to set correct date and time

Previous version features * Touchscreen/hardkey interaction sounds * Headset support * Volume control * More reliable DSP setup, now the sound system comes up at startup as expected :)

Register