Brainstorm

Remove 256MB limitation of the rootfs partition in the N900

Posted on 2010-02-09 01:46 UTC by Stefan Möhl. Status: Under consideration, Categories: User Experience, System, Programming.

The root directory of Maemo5 is limited to 256MB on the N900. This is a serious limitation, and users often and easily fill it up (especially if using the extras-devel repository). It is unfortunate in other ways too. For example, the standard Debian distro officially supports Arm7, so with a better partitioning, much of Debian would be directly usable through apt-get. We could use the full gamut of GNU/Linux commands instead of the settling for the Busybox replacements - imagine, "less" and "man" included by default!

The current solution is "optification". However, that solution is brittle since it requires active adaptation of all packages so that they are placed in /opt instead of in their usual locations. This means added burden on developers and makes packages slower to move into stable repositories.

As far as I know, the total set of mass storage available for the N900 is:

* 256MB Root partition (in a fast 256MB flash)

* 768MB Swap

* 2GB /home

* 27GB FAT32 USB storage

(Swap, /home and USB storage are all in a 32GB slow flash)

If I understood this correctly, the main reason for the small root partition is that it is in a faster flash than the home partition. To keep the device responsive, the speed sensitive stuff is placed in that flash, and other stuff is placed in /opt (/opt in turn is mounted in /home).

There are a few more constraints, in summary:

1) There is a fast 256MB flash and a slow 32GB flash

2) The device needs to be capable of getting to a guaranteed stable functioning state from a reflash

3) A reflash should not delete the user's document data (i.e. pics, music, etc) in the large flash storage

4) A reflash will always replace the entire contents of a flash memory.

Right now I am using a home grown solution, but it breaks each time the distro has an upgrade, forcing me to spend days rebuilding and moving everything again. And newbies end up having to reflash just because they tried out a few beta-level apps and filled the root partition to a point where booting fails. We need a better standard solution!

There is a related Brainstorm about re-partitioning the 32GB internal flash, though that doesn't solve the root directory problem.

Discus this brainstorm in the forums

There has also been some previous discussion in the Wikis

Solutions for this brainstorm

Solution #1: Place root in the large flash and link speed critical files from the fast flash

Posted on 2010-02-09 01:46 UTC by Stefan Möhl.

EDIT: Rewrote my old, rather messy description of this completely, and added an idea at the end of how to bootstrap all this after a reflash.

Basic idea:

Have the root directory in a large partition in the big flash instead of in the small fast flash.

Only files and directories that specifically need speed are placed in the fast flash with symlinks into the large root.

This is basically the same trick as optification, just the other way around. Instead of the fast flash as root, we have the big flash as root. Instead of moving large files and linking, we move speed critical files and link. The main advantage is that the default mode of operations is clean: Just install under the normal root hierarchy without any tricks like moving or linking. Specific files and directories from root may be moved and linked to improve performance if desired.

The old optification means that we first install and leave the root files in the fast flash, and then move all packages we install in the future to /opt. With this new solution we move just a few specific root files and directories (just those that really need it) to the fast flash, and leave all installed packages in normal locations in the big flash. So by default, we move only a specific and limited number of files, instead of moving all future files that will be installed.

Since packages are now normal, installed in their usual locations, we can now start installing all sorts of packages from the Linux community, as long as the binaries are compiled for the Arm processor. All of Debian, for example, has Arm support by default. We will now also have space to install the standard development environments, so if a package has no support for Arm, we can just install the source package and compile directly on our phones for the Arm processor.

Bootstrapping:

Bootstrapping the configuration after reflash: Since a partition in the big flash can not be reflashed without loosing all the data in the user partition (i.e. camera pics, music, etc), reflashes all have to go to the fast flash. Since we want the root to be in the big flash, we will have to move it there from the fast flash somehow. I suggest we do it the same way as we get root access when installing "rootsh": We "install" a large root partition through the app manager.

The "bigroot" package should be like the "rootsh" package: An officially endorsed and supported power user feature. When installing the package, all files are moved from the fast flash to the 2GB partition in the big flash, except the files that should stay in the fast flash for performance reasons. The bootloader is then modified to boot from the new root location, and the device is rebooted.

The "install bigroot" approach has a few advantages. First of all, it permits a rock solid state after a reflash, since it works just like things do now. Users can be warned and choose for themselves if they want to activate bigroot or not, so the pressure of perfect operation is much lower. Many packages won't be installable without installing bigroot, but packages that cater to casual users can still be optified like before. Bigroot is compatible with optification. Also, if your device bricks and you have to reflash, the old bigroot partition isn't cleared until you re-install bigroot. That allows you to rescue some files and configurations once your system is back up and running, and possibly figure out what went wrong.

The installer trick can also be used for applications by adding "speedification" packages. A "speedification" package for an application will move speed-critical files for that application to the fast flash, leaving links behind in the big flash. Similar to optification (though in the reverse direction), but manually activated or deactivated by the user. This way users get detailed control over which applications get to use the fast flash, and can optimize for their own usage patterns.

Solution #2: Clean rootfs and /opt separation

Posted on 2010-02-09 17:36 UTC by Tom Tanner.

The NAND rootfs is reserved for the kernel, system libraries and essential Nokia software and updates. It is supposed to boot even if the /home partition is broken, not accessible or filled.

All user modifications (installing new software, preferences) should be done in /home and leave the rootfs untouched. No symlinks to /opt are stored in the rootfs.

When the firmware boots the first time, it tries to copy and symlink /var to /home/var and creates (or clears) an empty /home/opt directory. This process could also be initiated by a "reset device" option in settings.

The search paths of the system components are modified to include all relevant directories in the /opt/maemo/ hierarchy.

for more details see this page

Solution #3: Use 256MB as cache space

Posted on 2010-02-10 00:33 UTC by Leonid Yegoshin.

put all root (besides boot) into 32GB flash and use fast 256MB space as a cache for disk blocks. Initially it would take some additional time and battery to replicate blocks into new 256MB cache but later it would give a fast access to frequently used data with virtually unlimited space.

It is possible just to extend kernel cache interface to include 256MB into it. But to get the result permanent via next reboot it may requires some additional development in FS.

Solution simplifies FS management and it actually automate file allocation (no manual allocation management)

Reference document: http://www.mjmwired.net/kernel/Documentation/filesystems/caching/fscache.txt
Please take into attention that CacheFS can use any ext3 block device as a cache storage. I don't know about UBIFS but it is possible to convert internal 256MB space to ext3 and use it as cache space.

EDIT: I copied root FS from UBIFS to eMMC and created a working environment with root FS on eMMC after boot. It doesn't use fscache - just Maemo5 works from eMMC instead of UBIFS (swap on uSD partition).

After couple of days I have a positive impression of it, see - http://talk.maemo.org/showpost.php?p=758256&postcount=103

It looks like for me personally, the 256MB problem doesn't exist anymore.

Solution #4: Use Ipkg/Opkg

Posted on 2010-02-10 12:07 UTC by Javier S. Pedro.

The N900 (and previous devices) are not the only platform with this kind of setup. One of the goals of Ipkg was to allow the same binary package to be installed into different prefixes: Ipkg#Installing_ipkg_Packages .

This doesn't solve all the LD_LIBRARY_PATH, PATH, PKG_CONFIG_PATH, etc. issues as mentioned by solution #2, but would fix the scenario where an advanced user wants to install packages to / instead of /home/opt (since they have cloned to emmc, are using the SDK, or simply want good performance for that individual package).

Solution #5: chroot environment(s) on flash with symlinks to rootfs

Posted on 2010-02-10 20:20 UTC by Tom Tanner.

The rootfs on NAND is kept in the original state and a chroot environment is installed on the eMMC flash during the first boot. In this chroot environment symlinks are created to the system files on the bind-mounted rootfs. Additionally the /etc and /var directories are copied into the chroot as they change or grow quite often.

All user modifications (package installation,preferences) therefore only affect the chroot environment, in which the system chroots early during the boot process. There is no need for /opt and normal Debian packages could be installed in /usr. System binaries and libraries benefit from staying on the faster NAND flash.

The user may also boot into the original rootfs (for recovery or firmware upgrades) or have multiple chroot environments (e.g. default, development, Mer etc). This solution is similar to easydeb and therefore already feasible.

see this post for a simple example chroot.

also see this post for further work.

Register