Experimental: Cutting out the middleman (cryptofs)

[GMMan]

I remember reading that someone cut out cryptofs from their Pre and I thought I might try the same. I can't find the original post, so I'm devising my own. Please note that this is experimental, and I won't be responsible if you damage something.

You will need Novacom enabled so you can access the device. On-device shells won't work.

Procedure

1. Download the attached build of passfs to your media partition. Unzip the file. passfs.zip
2. Open Novaterm and connect to the device. If you're on Windows, use the Java version. On other platforms just type 'novaterm' into a terminal.
3. Stop Optware sshd if it's running:
   

   initctl stop mobi.optware.openssh

4. Unmount Optware loopback (if you have any Optware apps installed):
   

   umount /opt

5. Suspend /media/cryptofs (basically freezing it so new things can't be written). (This is alternative to umounting it, which I couldn't figure out how to make it work.)
   

   /etc/storaged/pre_msm.d/05_suspend_cyptofs

6. Make a temporary mount point for cryptofs:
   

   mkdir /tmp/cfsmount

7. Set up cryptofs (again) so we can pull files out of it. This contains the same files as /media/cryptofs. We still need to pull stuff out of it, but at the same time keep it at a consistent state.
   

   /usr/bin/mountcfs -m /tmp/cfsmount/

   You may see "Enter password:". Ignore that.
8. Copy the contents of the cryptofs into the new location:
   

   cp -r /tmp/cfsmount /media/internal/appdir

   Replace "appdir" with the name you'd like for the app folder. From here you'll be waiting while everything is copied over to the unencrypted location. Depending on how many apps and the size of the apps you have, it may take several hours to finish. (On my TouchPad with ~3 GB of apps it took a round 1.5-2 hours.) 'cp' does not give feedback, so if you want feel free to open another Novaterm and 'ls' the apps directory from time to time. You can't use your device during this period (because /media/cryptofs is frozen, so your downloaded apps won't load, and also the device will be really slow).
9. Mount rootfs as read/write:
   

   /usr/sbin/rootfs_open -t

10. Rename mountcfs so the cryptofs won't be mounted on reboot:
    

    mv /usr/bin/mountcfs /usr/bin/mountcfs.dis

11. Move passfs into place:
    

    mv /media/internal/passfs /usr/bin

    You may need to adjust the source path of passfs depending on where you saved it.
12. Set up passfs to mount on boot.
    

    vi /etc/init.d/mountall.sh

    Insert the line marked with the '+':
    

    setup_cryptofs() {                                                                    
        # mountcfs should not be installed on qemu builds, so test here.              
        if [ -x "$(which mountcfs)" ]; then                                           
            setup_cryptofs_one /media/internal /media/cryptofs                           
                                                                                         
            for DIR in /media/external?; do                                           
                [ -d $DIR ] || continue # will exit if pattern matched nothing        
                grep -q $DIR /proc/mounts || continue                                 
                NUM=${DIR#/media/external}                                            
                setup_cryptofs_one $DIR /media/ext${NUM}cryptofs                      
            done                                                                      
        else                                                                          
    +        passfs /media/internal/cryptofs /media/cryptofs -o nonempty -o allow_other -s
            echo "$0: cryptofs not enabled: required file missing" > /dev/kmsg
        fi                                                           
        echo "$0: setup_cryptofs done" > /dev/kmsg                   
    }

    Replace "/media/internal/cryptofs" with whatever path you chose previously. Quit and save.
13. Add this line to /etc/storaged/pre_msm.d/05_suspend_cryptofs:
    

    pkill -SIGUSR1 passfs

14. Add this line to /etc/storaged/post_msm.d/05_restore_cryptofs:
    

    pkill -SIGUSR2 passfs

15. Reboot:
    

    reboot

16. Test things out.
17. When satisfied, delete /media/internal/.palm to free up space. (Maybe? The cryptofs could possibly recreate itself.)

The replacement successfully allows the media partition to dismount.

passfs source code here: https://github.com/GMMan/passfs-webos

Known Issues
None.

Speed test results (procedure per http://it.toolbox.com/blogs/database...d-test-31069):

passfs multithreaded

write
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 372.881667 seconds, 5.2MB/s
real	6m 43.41s
user	0m 3.98s
sys	1m 4.83s

4.8MB/s w/ sync

read
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 173.510649 seconds, 11.3MB/s
real	2m 53.52s
user	0m 2.67s
sys	0m 34.60s

passfs single threaded

write
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 354.736778 seconds, 5.5MB/s
real	6m 21.91s
user	0m 3.65s
sys	0m 40.18s

5.1MB/s w/ sync

read
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 172.213540 seconds, 11.3MB/s
real	2m 52.21s
user	0m 2.28s
sys	0m 28.43s

cryptofs

write
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 370.175645 seconds, 5.3MB/s
real	6m 37.07s
user	0m 2.66s
sys	0m 37.94s

4.9MB/s w/ sync

read
2000000+0 records in
2000000+0 records out
2048000000 bytes (1.9GB) copied, 169.489399 seconds, 11.5MB/s
real	2m 49.50s
user	0m 1.37s
sys	0m 14.70s

As you can see, single thread passfs is the fastest with writing, while both passfs modes are about the same for reading and cryptofs being slightly faster for reading. So if you need to write large files to /media/cryptofs (like installing large apps), replacing cryptofs with single threaded passfs is preferential. Otherwise either is good for read speed.

[xandros9]

What is the reason for doing this? It's not immediately obvious to me...

[gizmo21]

Perhaps what you were searching was mybackup by Metaview: L\'Haut-Parleur » Original Palm Pre auf WebOS 2.1 updaten (dont' mind the topic)

Pre1- : Saving all apps it had a runtime of 40minutes for 270MB data @146 Apps (be sure to have it on the touchstone)
Pre3: Saving all apps it had a runtime of 10minutes for 315 MB data @142 Apps

They are then single gzipped and uncrypted in the /media/internal/mybackup folder, ready to be backuped or transfered anywhere.

[GMMan]

Originally Posted by xandros9

What is the reason for doing this? It's not immediately obvious to me...

It opens up app files for easier editing (you know, patching and game hacking). There may also be a bit of speed improvement because things don't have to be encrypted and decrypted all the time. And also app backups, if you're into that.

[GMMan]

I've finished writing the guide and posted some speed test results.

[GMMan]

Originally Posted by gizmo21

Perhaps what you were searching was mybackup by Metaview: L\'Haut-Parleur » Original Palm Pre auf WebOS 2.1 updaten (dont' mind the topic)

Pre1- : Saving all apps it had a runtime of 40minutes for 270MB data @146 Apps (be sure to have it on the touchstone)
Pre3: Saving all apps it had a runtime of 10minutes for 315 MB data @142 Apps

They are then single gzipped and uncrypted in the /media/internal/mybackup folder, ready to be backuped or transfered anywhere.

That doesn't make a complete copy of /media/cryptofs. It takes only the applications directory, and will leave behind, for example, optware.ext3.

[greenoyster]

A more reliable way of clearing the disk page cache for performance testing is to tell the kernel to drop it from RAM (non-destructively; unwritten data will not be dropped):

echo 1 > /proc/sys/vm/drop_caches

Writing a large file to fill up RAM is not guaranteed to clear the read cache, because the write cache will stop expanding at around 20% of total RAM (defined by /proc/sys/vm/dirty_ratio) and pause the writing process until the pages are written out.

[ggendel]

To do a backup, I usually open up a term on my WebOS device and scp the entire cryptofs partition. Can't miss anything that way.

[GMMan]

Originally Posted by ggendel

To do a backup, I usually open up a term on my WebOS device and scp the entire cryptofs partition. Can't miss anything that way.

The main point of this, at least for me, is not backup, but getting rid of unnecessary roadblocks.