[Preemptive]

It's your thread because you will have to write it! ;-) Please post useful information here and I will add it to the appropriate section.

There have been requests for pivotCE to supply some information about backing up and you have seen the articles on returning to webOS and how to back up your apps. There are some useful articles here on webOS nation.

In about a week On March the 15th, the HP servers will shutdown and I think there is a need for an easy to follow guide about how to both back up and how to restore or even set up a device after the shutdown. There is the webOS-Internals survival guide, but I don't think there will be further information added to it.

The object of this / these articles will be to show how to:
1. Back up now and in the future - hopefully with relative ease.
2. Restore without the HP/Palm profile servers (possibly also to a replacement device)
3. (maybe) delete your HP/Palm profile and switch off device profile services.

Good news / Bad news

Actually, it's the same news: I'm no expert. That means if I can understand it and write it, you can do it. It also means I'm asking the experts on this forum to fill the gaps in my knowledge and correct my errors.

I've posted my thinking and some information below. This post will link to the articles when they're done.

For now, please post any helpful information & corrections you have on backing up and restoring. Hopefully, we can tick off all the items backed up to the profile (and more?). If we can do a full local back up manually, that's enough, but this info may help anyone who could offer a semi or fully automated solution.

[Preemptive]

To begin with the basics, I'd guess that the way to make a total back up is as with any other computer: Image the drive (i.e. Copy all the data from the phone). I've been told the Linux command line dd instruction can do this, but I also know the nickname for this command is 'delete drive' and that it has to be used with care. I'm also very unclear how easy it would be to restore this image. The image may also be incompatible with a replacement device.

I've also read that the metadoctor process can actually make a reinstallable image of the phone, but I haven't gotten into that yet.

Both these processes seem a little intensive if we assume that a back up process would be performed fairly regularly and that only user data really needs to be retained - the webOS doctors everyone is desperately downloading can restore the system, then adding the user data back up can bring the device up to date again.

The back up service that is soon to shutdown only backed up certain user data. My intention is to find ways of replicating that process by either known methods or at worst locating the information and manually copying it off the device.

My assumption is that all those running Linux desktop systems and webOS SDKs can probably take care of themselves. I intend to write a guide for those like myself: J. Random User, sitting behind a computer with a mainstream OS, able and willing to use WOSQI, Preware, patches and follow clear instruction, but not well versed in Linux system administration.

To avoid duplication, I'm just going to link to existing resources, then if anything needs updating or gaps need filling, we can add that in.

Having used Impostah to locate device, profile and app data, would it have any application to locating or even copying data to be backed up? Here is the (lack of) documentation:
Application:Impostah - WebOS Internals
Poor solution: It's possible to query the databases, but emailing entries is only possible on a per entry basis - likely impractical if the HP/Palm calendar (or other app) account has had significant use.
=====================
Firstly, the basics: The Definitive Guide to Backing up and Restoring your webOS Device (it's no longer definitive, because it assumes the HP servers are available). Linked there is: Roundup: Advanced device backup/restore tips

• Copy or selectively back up the USB partition. This is where much of your media, downloads etc is.
• Then there are your apps. (the following requires the servers, so no use now)
  
  • http://forums.webosnation.com/webos-...-one-step.html
  • Guide: Saving Apps From The App Catalog Part 1: nodeleteipk patch | pivotCE
  • Guide: Saving Apps From The App Catalog Part 2: HP Takeout | pivotCE
  • Non system apps are apparently stored in media/cryptofs/palm/applications I think. Possibly, those files can be copied and repackaged into IPKs. A link here: http://forums.webosnation.com/hp-pre...ml#post3446107
  • See also post no. 8 below.
• App data can be backed up with Save/restore
• It can also be used to back up additional system data: Use Save/Restore to backup additional system files and images
• Also: http://forums.webosnation.com/webos-...ml#post3436195. Can extra scripts be made to back up those extra bits of the OS - like wifi profiles?
  

From the webOS Survival kit, use show properties to back up the tokens. I don't really know what these are but assume they are like cookies that identify your device to external services. https://en.wikipedia.org/wiki/Software_token.

=====================
Thanks to Mazzinia posting a link, I downloaded a TP manual. It includes a table detailling what is and is not backed up by the profile server. I've put this information in a spreadsheet with notes on whether I know of a way to back up this data or not. I've attached a zip file containing this file (open document format) and a tab separated text version (.csv)

Notes key: "Unknown" means I don't know how to back it up, "?" Means I don't know if it needs to be backed up, "No matter" means I don't think it's needed (these are mostly service login ids which should be backed elsewhere anyway. Passwords are not stored).

[Preemptive]

Reserved for Legacy webOS version 1 instructions.

From: http://www.webosnation.com/backup
After the processes described make a copy of:
/var/luna/data/dbdata/PalmDatabase.db3 file

It seems most of the profile data is stored here, so backing up a 1.+ device should be easy, but the database file should only be copied back to the original device - not a new one as it contains hardware data. So a different method may be required if restoring to a replacement device.
http://www.webos-internals.org/wiki/...abase.db3_File

[Preemptive]

Reserved for Legacy webOS version 2 instructions. For now, I'll assume versions 2 & 3 are functionally identical (i.e. the same backup techniques will work on both - correct me if I'm wrong.

It seems data is stored in different places from version 1. There is no single .db3 you can copy (it may be a .db8 with different structure). Back up may be more 'modular', but if possible would make it easier to move to a replacement device.

Calendar: Events in your HP webOS Account - http://forums.webosnation.com/webos-...tcalendar.html
Preferences - Unknown
Contacts: Contacts in your HP webOS Account - Yes, possible to save ALL contacts (inc. synergy data) Export all your contacts | webOS Nation
Enhanced Contacts VCard Export - Preware Catalog NOTE: this patch conflicts with the C+DAV contacts patch, which resolves the issues by including the functionality. i.e. Don't use it if using C+DAV (& you don't need it).
See also: Export your contacts via Command Line (homebrew) - webOS Nation
Favorites - Unknown - possibly as above with a patch
(not backed up by HP: Linking relationships between contacts) - Unknown
Preferences - Unknown
Device Info: Nothing is backed up - ? Maybe include to id backup with device data?
Facebook: Nothing is backed up - ? There are problems with the app & data has been lost. Try to back up?
Help: Nothing is backed up - ? Not sure about this. I think help files & videos are online. Might they be lost at the shutdown? The files are now unavailable at HP, but Sumy copied the file structure to http://www.palm.si. Despite trying a HOSTS fix, the app won't connect or search (likely a 'backend' component is missing), but the files can be browsed and viewed on the web.
Just Type: Nothing is backed up - ? I can't think of anything but JT history to back up, so... probably doesn't matter.
Launcher: Reordered Launcher icons - ? I'd say useful, but not vital.
(not backed up by HP: default apps are set in preferences - that might be good to back up)
Location Services: Preferences - ?
Memos: All memos - Yes - Export Notes (app + patch)
Messaging: Instant messages - Yes - Export Messages | webOS Nation
Export your text and instant messages [webOS 2.0+] | webOS Nation
Text messages sent or received when your device was connected to a webOS smartphone - Yes (see above)
(not backed up by HP: Passwords and history) ? How is history NOT backed up if messages (with time stamps?) are?
Phone & Video Calls: Call history entries for any calls made, received, or missed when your device was connected to a webOS smartphone - Unknown - as the phone app can delete the history, perhaps the location of the file / folder can be found in the app code? There's this: http://forums.webosnation.com/webos-...l-history.html and now this app: http://forums.webosnation.com/webos-...portcalls.html
Software Manager: (not backed up by HP: App settings and data stored in the app) - Save Restore. This seems to be Palm Profile / app catalogue related, so will probably become largely irrelevant. What about services? I seem to have none, but...
Text Assist: - Saverestore, or see: http://www.webosnation.com/export-te...s-2-x-homebrew
Web: Bookmarks - Unknown. No mention of history at all, but maybe not important.
Browser cookies - Unknown.
Wifi Profiles: A command line solution offered by Misj':

luna-send -f -n 1 palm://com.palm.wifi/getprofilelist {} > /media/internal/wifi_profiles.txt

...will output a text file to the USB drive area. Note that the passwords will be in plaintext, so consider carefully how to store / encrypt this data.
See post 6. for a meta doctor method to restore, but it restores a different file (prefsDB.sl file from the /var/preferences/com.palm.wifi/ directory).

[Preemptive]

Reserved for Legacy webOS version 3 instructions. If 2 & 3 are similar enough they may be combined.

[Preemptive]

Reserved for restoration instructions.

Bypass Activation [webOS 1.x] | webOS Nation
Bypass Activation [webOS 2.x] | webOS Nation & http://forums.webosnation.com/hp-pre...ml#post3437254
http://forums.webosnation.com/hp-tou...-touchpad.html & http://forums.webosnation.com/hp-tou...ver-error.html

I think the data transfer assistant connects directly to the phone, so assuming correct formats, may be useful in restoring PIM data (calendar, contacts, tasks, memos).
HP webOS : Download & Run the Data Transfer Assistant (DTA)

Alternatively, WOSQI might be used to send and receive copies of raw data.

If the location of all relevant data is known, is it possible to use Save/Restore (with a script) to do the whole backup / restore job?

UPDATE (20/08/2018): Restoring wifi profiles using the meta-doctor (from webOS Internals)

INSTALL_WIFI_PROFILES imports a wifi preferences database file from the user's home directory to the device. The device will then be set up to use wifi immediately after the webOS Doctor has completed. You must already have copied a correctly configured prefsDB.sl file from the /var/preferences/com.palm.wifi/ directory on the device to ~/.ssh/com.palm.wifi.prefsDB.sl or to ./config/com.palm.wifi.prefsDB.sl on your host machine before enabling this feature, or it will cause a fatal error.

[Preemptive]

How to switch off connections to HP servers and possibly redirect to any community solutions that may arrive.

When the HP servers close, what will happen? It's my assumption that basically NOTHING will happen. No kill signal will be sent, the data on your device won't vanish. BUT when your back up app next tries to contact the servers, it will fail - it should fail gracefully, but you might as well turn it off and save a few volts.

UPDATE: Nothing bad happened.

The Back up app has an off switch - simple!

But what other services might the device be attempting to connect with? App & system updates?

[Preemptive]

Reading this: https://translate.google.com/transla...-text=&act=url

Could be the solution.

Original german: L\'Haut-Parleur » Original Palm Pre auf WebOS 2.1 updaten

[gizmo21]

Reading this: https://translate.google.com/transla...-text=&act=url

Could be the solution.

Original german: L\'Haut-Parleur » Original Palm Pre auf WebOS 2.1 updaten

So "mybackup" is only a solution to backup all device installed apps as they are (e.g. patched...) into many single tar.gz files. They can be restored separetly on same/other devices but apps with more advanced pre install start-scripts will fail, as those get lost in opposite to the original IPK. I guess over 90% of the apps will be OK with mybackup.

This app is great on top of downloading IPKs from appcatalog, for those apps that are not anymore found in appcatalog or for those apps you heavly patched/modified manually without patchfiles and want to backup/transfer in "is now" state.

Detailed translated guide on mybackup is to be found here:
http://forums.webosnation.com/webos-...ml#post3377219

But all of this is only saving the app, for appdata you still need the preware save&restore or manual db-backup.

For some more backup/export stuff go to my
http://forums.webosnation.com/webos-...ps-tricks.html

[gizmo21]

Still missing:

Solution to restore wifi profiles, as the save&restore script to do this does not seem to restore on my new device even when directly restarting after restore and having wifi off at restore.

Solution to restore "Text Assist" data. Save&Restore script only supports backup.

[twinmon45]

To begin with the basics, I'd guess that the way to make a total back up is as with any other computer: Image the drive (i.e. Copy all the data from the phone). I've been told the Linux command line dd instruction can do this, but I also know the nickname for this command is 'delete drive' and that it has to be used with care. I'm also very unclear how easy it would be to restore this image. The image may also be incompatible with a replacement device.

I've also read that the metadoctor process can actually make a reinstallable image of the phone, but I haven't gotten into that yet.

Both these processes seem a little intensive if we assume that a back up process would be performed fairly regularly and that only user data really needs to be retained - the webOS doctors everyone is desperately downloading can restore the system, then adding the user data back up can bring the device up to date again.

The back up service that is soon to shutdown only backed up certain user data. My intention is to find ways of replicating that process by either known methods or at worst locating the information and manually copying it off the device.

My assumption is that all those running Linux desktop systems and webOS SDKs can probably take care of themselves. I intend to write a guide for those like myself: J. Random User, sitting behind a computer with a mainstream OS, able and willing to use WOSQI, Preware, patches and follow clear instruction, but not well versed in Linux system administration.

To avoid duplication, I'm just going to link to existing resources, then if anything needs updating or gaps need filling, we can add that in.

Having used Impostah to locate device, profile and app data, would it have any application to locating or even copying data to be backed up? Here is the (lack of) documentation:
Application:Impostah - WebOS Internals
=====================
Firstly, the basics: The Definitive Guide to Backing up and Restoring your webOS Device (it's no longer definitive, because it assumes the HP servers are available).

• Copy or selectively back up the USB partition. This is where much of your media, downloads etc is.
• Then there are your apps.
  • http://forums.webosnation.com/webos-...-one-step.html
  • Guide: Saving Apps From The App Catalog Part 1: nodeleteipk patch | pivotCE
  • Guide: Saving Apps From The App Catalog Part 2: HP Takeout | pivotCE
• App data can be backed up with Save/restore.

From the webOS Survival kit, use show properties to back up the tokens. I don't really know what these are but assume they are like cookies that identify your device to external services. https://en.wikipedia.org/wiki/Software_token.

=====================
Thanks to Mazzinia posting a link, I downloaded a TP manual. It includes a table detailling what is and is not backed up by the profile server. I've put this information in a spreadsheet with notes on whether I know of a way to back up this data or not. I've attached a zip file containing this file (open document format) and a tab separated text version (.csv)

Notes key: "Unknown" means I don't know how to back it up, "?" Means I don't know if it needs to be backed up, "No matter" means I don't think it's needed (these are mostly service login ids which should be backed elsewhere anyway. Passwords are not stored).

Hi,

Is there any chance someone can do this backup for me. I can't even understand the language (CMiii) etc. The tp is my access to the computer world. And I have 3 kids who use them regularly for school. Is there a service shop near LA/Glendale/Burbank Ca? Or anyone nearby willing to volunteer a hand. The directions are a bit complex for me.
twinmon45

[bluenote]

Still missing:

Solution to restore wifi profiles, as the save&restore script to do this does not seem to restore on my new device even when directly restarting after restore and having wifi off at restore.

Solution to restore "Text Assist" data. Save&Restore script only supports backup.

what does this mean? will we still be able to access wifi after the 15th? will it still remember our wifi profile?

[mazzinia]

what does this mean? will we still be able to access wifi after the 15th? will it still remember our wifi profile?

Obviously wifi will work past 15th, but what he means is that the save/restore script does a backup of the wifi profiles you have on your device (prior to doctoring).. but the script fails to restore those wifi profiles on a new device (I'm assuming it would work on the same device, instead)

[1whocs]

Thanks for this. I've toyed with a thought... Maybe this is just brutish and foolish, but could you possibly decompile a webosdoctor .jar, and replace whatever app data you want with your current folders, (like maybe /media/cryptofs/apps/usr/palm/applications), then recompile? I'm sure there's things to know about it that I don't understand, like maybe cache issues or something, but, if you know what you're doing, could you theoretically make a backup that way, and just reflash the doctor if you need to?

-- Sent from my Palm Veer using Forums

[mazzinia]

yes, I think you can

[Preemptive]

Thanks for this. I've toyed with a thought... Maybe this is just brutish and foolish, but could you possibly decompile a webosdoctor .jar, and replace whatever app data you want with your current folders, (like maybe /media/cryptofs/apps/usr/palm/applications), then recompile? I'm sure there's things to know about it that I don't understand, like maybe cache issues or something, but, if you know what you're doing, could you theoretically make a backup that way, and just reflash the doctor if you need to?

-- Sent from my Palm Veer using Forums

As stated above, you (apparently) can do exactly that - reading data from the device and creating a webOS doctor for a customised restore. The drawback is that you need to have various Linux tools and systems running and be at least familiar enough with the command line to understand and follow the instructions. General instructions are on the webOS-internals.org site, but they are about customising doctor files rather than specifically copying an existing installation. This stuff is for relative experts, not the general user. I'd also bet that no one does this as a regular back up routine - it's for swapping devices.

The purpose here is to identify and back up data items that would previously be backed up to HP in the simplest way possible. The purpose of the back up is to have a copy of your data should there be a data loss or if the device is lost, stolen, bricked or damaged. The result is that you either have data restorable to the original device, a new device or at least a copy of the data. The doctors can be used to restore a clean system into which the data can be re-added. Comprehensive instructions for cloning a system AND data is a big ask...

There are already ways to copy contacts, memos, texts and IMs. Moving HP/palm account calendar entries to a cloud provider (manually) is possible, but could take a while and maybe some don't want to. If there was a simple way to make a local copy of that, it would be useful.

PIM data is most personal and important. Of the rest, it is of varying importance. If for instance you noted wifi login details in a memo prior to entering them in the app, then it's simple to re-enter them from a backed up memo, although less convenient than having it backed up directly.

Most of the data is either viewable to the user in the app or even as a database entry - it's just the hassle of manual removal (item by item) is impractial on a regular basis.

Perhaps we could consider a 'total', intensive back up of all data performed occasionally and a smaller regular one of data that changes frequently.

[Preemptive]

Hi,

Is there any chance someone can do this backup for me. I can't even understand the language (CMiii) etc. The tp is my access to the computer world. And I have 3 kids who use them regularly for school. Is there a service shop near LA/Glendale/Burbank Ca? Or anyone nearby willing to volunteer a hand. The directions are a bit complex for me.
twinmon45

Don't Panic. The instructions aren't complete - this thread is researching solutions. There may also be automated solutions available.

If your calendar is with a 3rd party (e.g. google), then you already have an off-device back up. The instructions already linked for contacts and messages are not that difficult. Re-entering settings will be a minor hassle. If you have followed the instructions for backing up apps, then the Save / Restore may well be able to archive any user data from them.

[Preemptive]

So "mybackup" is only a solution to backup all device installed apps as they are (e.g. patched...) into many single tar.gz files. They can be restored separetly on same/other devices but apps with more advanced pre install start-scripts will fail, as those get lost in opposite to the original IPK. I guess over 90% of the apps will be OK with mybackup.

This app is great on top of downloading IPKs from appcatalog, for those apps that are not anymore found in appcatalog or for those apps you heavly patched/modified manually without patchfiles and want to backup/transfer in "is now" state.

OK, so to clarify, if we assume that people now have copies of their App installation files (IPKs), the "mybackup" process will copy installed apps including any applied patches - good. (complex apps like ACL may be tricky, but most apps will be fine). App data (game scores etc) will hopefully be covered by Save / Restore.

I liked the look of this technique because it seemed fairly simple to do. I was wondering if it would save data from the core apps (Calendar etc), but it seems it doesn't save any app data. Did you have a means to restore any of the core app data?

[gizmo21]

OK, so to clarify, if we assume that people now have copies of their App installation files (IPKs), the "mybackup" process will copy installed apps including any applied patches - good. (complex apps like ACL may be tricky, but most apps will be fine). App data (game scores etc) will hopefully be covered by Save / Restore.

I liked the look of this technique because it seemed fairly simple to do. I was wondering if it would save data from the core apps (Calendar etc), but it seems it doesn't save any app data. Did you have a means to restore any of the core app data?

Regarding "mybackup ":
So yes i would prioritise it in this way:
1. if you have an IPK already downloaded - just use that IPK to have a clean install of an app
2. if you do not have any IPK you can use mybackup on the device to save a tar.gz of the installed file app directory and restore it later including creation of app icon...
2a) this can also be useful for heavily patched apps

It does not backup core apps,so only third-party apps but as some core-updates also are handled like 3rd party also those get backuped:
com.palm.app.appcatalogupdate
com.palm.app.facebook
com.palm.app.googlemaps
com.palm.app.maps
com.palm.app.mobilehotspot

In-App data ist never backed up with this method, so also core data is not backed up.
As i cleaned my PalmProfile data a long time ago and went to my "own cloud" zarafa (see signature) i had no need for core app PalmProfile data.

I did write some guide on migration of PIM data away from PalmProfile back then (just replace the name "Zarafa" with any cloud provider else to PalmProfile you have e.g. google):

Migration from an other Sync-Providers (Google, Palm Profile, Chapura Echo...)
################################################################

Calendar (on the fly):
----------------------
* Nice thing is in calendar you can change the events from other accounts to zarafa when hitting the account selection button.
* This will delete the event in the old account (google/palm profile...) and add it as new into zarafa.

If you like to keep it in the other account as backup for now, you could also select 'Edit as new Event' in Ubercalendar to create
an identical copy and change account in this one.

* This manual method is only good for a few single events, for exporting/importing many events look for "Export/Import options" (todo)


Contacts (on-the-fly not possible):
--------------
The bad thing is, you can't do the same on-the-fly convert in contacts. No account changing and no copy function there.

Does someone know a method to "batch convert" all events/contacts on the webOS device from one account to a new one - let's say 'Palm Profile' to 'Zarafa'?
Best would be a simple interface to select source account (for contacts/calendar) and destination account and perhaps a checkbox if it should copy or move the events/contacts.

Contacts (Export/Import):
------------------------------------
* As long as batch converting is not available, you have several options to fill your Zarafa-Server with contacts:

1. Exporting all your contacts in your webOS device and reimporting it on your webOS device, having Zarafa as standard contacts account (!!!) - Nice thing is all contactdata of all accounts is exported and merged in one .vcf file so if you have facebook account in your contacts the facebook-emailadresses will be in your Zarafa too afterwards. But more on this in the cleaning the file section below.
* Optional: Before exporting your contacts you perhaps want to apply the following patch for webOS 2.2.4 [Patch] Fixing/Enhancing vCard (.vcf) Export (it fixes some limitations in export noted below)
* Export all contacts as .vcf or via Export all your contacts | webOS Nation
** be sure to set in contacts the default account to "Zarafa"
** before importing the .vcf again on your webOS device you can edit it and so clean data a little bit, because:
*** cause URL is wrong formatted in in export .vcf (fixed with applied patch mentioned above)
*** Kids/Spouse is missing in export .vcf (fixed with applied patch mentioned above)
*** perhaps delete @facebook.com adresses if not wanting them in Zarafa

Be sure to edit the .vcf file only with a UTF-8 compatible editor, else you will get wrong entries on special characters like Umlauts.

.... more on editing vcf file before reimporting in the posting linked above

Reserved for Legacy webOS version 2 instructions.
...
Contacts: Contacts in your HP webOS Account - Yes, possible to save ALL contacts (inc. synergy data) Export all your contacts | webOS Nation
Enhanced Contacts VCard Export - Preware Catalog
...

btw. if you already have the contacts patch of C+DAV installed (which i very recommend when still using google), don't use my "[Patch] Fixing/Enhancing vCard (.vcf) Export " to export contacts data as vcf, as garfonso already integrated all enhancements into his patch (it won't install anyway over the other one).

[Preemptive]

btw. if you already have the contacts patch of C+DAV installed (which i very recommend when still using google), don't use my "[Patch] Fixing/Enhancing vCard (.vcf) Export " to export contacts data as vcf, as garfonso already integrated all enhancements into his patch (it won't install anyway over the other one).

Thanks for spotting that - I'll make a note.