webOS Nation Forums >  webOS Discussion >  Open webOS General Discussion > Thinking about... Locally hosted backup servers
Thinking about... Locally hosted backup servers
  Reply
Like Tree30Likes

 
Thread Tools Display Modes
Old 10/12/2012, 09:39 PM   #1 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
I'm digging through the code for cloud backup. Because the code is all JavaScript, I'm thinking it could be easy to reverse engineer it and create a local backup server from that. So for Open webOS, if the cloud backup is not provided by HP, users could host their own backup server on their network, copy over the backup framework, and instead back up to their own network.

From what I've read so far, the backup system generates a manifest with some statistics, fires off an event for different services to generate backup files, group and list those files in the manifest, then sends the manifest and files off to the backup servers. There are three servers used in this process: an authentication server, a metadata server, and blob storage server. There's an additional deviceImages server that I haven't figured out yet.

There are a lot of stats sent over with the manifests: battery percentage, signal strength, OS version, free disk space, and CPU/RAM load (I wonder what that's for). Backups are typically done incrementally, and each service comes with a description of what part of the system is backed up. Because it's modularized on services, maybe in the future people could select what parts they want to restore. And because of this modularization, new services could potentially be added. If that is indeed possible, maybe the WebOS Internals Backup/Restore app could be tied in and make it easier to restore flat file app settings (like used in games) without shuffling things on the USB drive.
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor

Last edited by GMMan; 10/12/2012 at 10:21 PM.
GMMan is offline   Reply With Quote
Liked by checkmate, Rnp, hans109h and 5 others like this.
Thanked By: Rnp
Old 10/12/2012, 10:33 PM   #2 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
File access looks easy. You request a storage auth token with your standard authentication details, and then you can access your files that is hosted under a subdomain within the main storage cluster's subdomain, with your storage auth token. Then it becomes sort of like an FTP, where an HTTP GET will either return a list of files or the file requested, and an HTTP PUT will upload the file to the storage. All the while there is a metadata server to keep track of the session. Checking how files are populated now.

Update: There are two ways to install backup services: a) put a manifest under /etc/palm/backup, with the file name being the service ID (not enforced), and a schema that looks like this:

Code:
{
    "id": "com.service.id",
    "preBackup": "preBackupMethodName",
    "postBackup": "postBackupMethodName",
    "preRestore": "preRestoreMethodName",
    "postRestore": "postRestoreMethodName"
}
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor

Last edited by GMMan; 10/12/2012 at 10:47 PM.
GMMan is offline   Reply With Quote
Old 10/13/2012, 08:56 AM   #3 (permalink)
Member
 
xandros9's Avatar
 
Posts: 911
woah, thats so cool!

i just got myself an itanium
__________________
HP Think Beyond Event - http://youtu.be/VnjwG7Z8AM8
m505 > Z|71 > T|C > Z|71 > T|T3 > LifeDrive > iPod touch 4
> Pre 2 > Treo Pro > Aria > HD2 > Treo 650 > Lumia 920
My OneDrive referral link: OneDrive
xandros9 is offline   Reply With Quote
Liked by Rnp likes this.
Thanked By: Rnp
Old 10/14/2012, 05:26 PM   #4 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
Here's how I think local backups could be implemented:

- A single server hosts all three components.
- A patch overrides the backup server paths, configurable through Tweaks (I'm surprised that the Palm Profile service is actually implemented in JavaScript).
- Authentication and metadata is wrapped in a database, where there will be a table of devices (persistent), table of authentication tokens (temporary), and a table for metadata.
- An online interface is used to control all three parts: you can add/delete devices from one interface, and be able to see the backup history (metadata) for each device. The metadata forms a log, and for each backup instance the manifest could be viewed.
- Auth tokens will be generated in the same general format as the way it's done currently. However, because of the way the backup system is programmed, there might need to be a DNS server installed locally so the storage server can be reached (it's like uniqueid.storageservername, instead of 168.45.6.7/backups/storage/uniqueid)
- There could be a manifest editor that can allow choosing which files to restore when a restore is necessary.
- When a restore is needed, connect the device to a computer to install the local backup patch, and then proceed through logging in to Palm Profile and restoring data.
- There can be a way to select which apps are installed on to the device instead of dumping the last known list of apps installed back (which always seems to be a month late).

About implementing Save/Restore backup on current system:
- Save/Restore would back up and zip together a file to be uploaded when a system backup is requested.
- When the system needs to be restored, the file will be downloaded back for Save/Restore to use.
- Immediately after erasing data, Save/Restore will not be installed, so it would be a good idea if this was saved to somewhere external to the Save/Restore app directory.
- Since Save/Restore wouldn't be installed, the backup system can't notify the app it needs to restore. So check for backup file the first time Save/Restore starts and restore data if backup file is found.
- This way even if the USB Drive partition is erased all app data can still be restored.
- There is no need for Save/Restore to do any additional processing to upload the backup. All it needs to do is backup, zip backup, and return the path of the archive to the backup service.
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Liked by Garfonso and MartinH@webos like this.
Old 10/15/2012, 12:50 AM   #5 (permalink)
Member
 
Posts: 11
This seems like a good idea. It may just be that I'm new to this whole javascript thing, but it seems kinda tough to do?
rolladude1166 is offline   Reply With Quote
Old 10/15/2012, 06:29 AM   #6 (permalink)
pivotCE Developer
 
Posts: 2,138
oh my own iCloud aahh wCloud for device perferences and third party apps backup and sync. That would be really cool and one major point missing in Open webOS:
- What's missing from Open webOS 1.0: Nothing that can't be fixed | webOS Nation

And the grounds are layed since 2.x newly introduced db8 clouddb for apps
gizmo21 is offline   Reply With Quote
Old 10/15/2012, 08:14 AM   #7 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
Quote:
Originally Posted by rolladude1166 View Post
This seems like a good idea. It may just be that I'm new to this whole javascript thing, but it seems kinda tough to do?
It would be implemented serverside using Java or ASP .NET. (Or maybe PHP.) Of course, server side JavaScript can also be an option.
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Old 10/15/2012, 03:36 PM   #8 (permalink)
Member
 
Posts: 42
Hum got me think. I like the ideal. What little i know about about net working wont help much but could you mod
the host file? instead of a dns server

john
john mcconnell is offline   Reply With Quote
Old 10/15/2012, 08:57 PM   #9 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
Quote:
Originally Posted by john mcconnell View Post
Hum got me think. I like the ideal. What little i know about about net working wont help much but could you mod
the host file? instead of a dns server

john
The goal is to allow the system to freely add and remove devices. Entering a hosts entry would be device-specific, and things can't be dynamically changed.
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Old 10/16/2012, 12:42 AM   #10 (permalink)
pivotCE Developer
 
Posts: 2,138
for many users a 'NAS-enabled' serverside would be great as these are servers at home that run 24/7. So thinking about php or nodes.js on serverside would be nice. Just a suggestion for the 'man on the streets' server possiblities.
gizmo21 is offline   Reply With Quote
Old 10/16/2012, 07:31 AM   #11 (permalink)
Homebrew Developer
 
Posts: 773
I love the idea and what you already did investigate...

Am willing to help you as much as I can.
Garfonso is offline   Reply With Quote
Liked by MartinH@webos likes this.
Old 10/17/2012, 11:45 AM   #12 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
I chanced upon an interesting server yesterday. It's called BarracudaDrive, and it's a file/application server with server side coding done in Lua. I think it's a good candidate for implementing the backup server, since it's light, fast, has built in database and JSON features, and was designed for file storage. Comes in both Windows and Linux versions, though I'm not sure if they offer non-x86 builds.

BarracudaDrive Web Server
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Old 10/18/2012, 07:37 AM   #13 (permalink)
Homebrew Developer
 
Posts: 773
At the bottom of the page it says that it runs on the raspberry pi and some other embedded device. So there seems to be ARM support...
Garfonso is offline   Reply With Quote
Old 10/18/2012, 04:38 PM   #14 (permalink)
Member
 
xandros9's Avatar
 
Posts: 911
there's no info whatsoever on architecture support.

my itanium has memory issues, so i'll have to talk to some people

eventually
__________________
HP Think Beyond Event - http://youtu.be/VnjwG7Z8AM8
m505 > Z|71 > T|C > Z|71 > T|T3 > LifeDrive > iPod touch 4
> Pre 2 > Treo Pro > Aria > HD2 > Treo 650 > Lumia 920
My OneDrive referral link: OneDrive
xandros9 is offline   Reply With Quote
Old 10/29/2012, 03:32 PM   #15 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
Looking at this webOS Nation article, it seems to scream cloud backup as the easiest path for implementation. A new provider that pulls out app data and does an incremental compare; uploads it to the backup server, then other devices pull the backups and import the changes.

The webOS Wish List: Synchronized webOS Accounts | webOS Nation
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Old 10/31/2012, 08:44 AM   #16 (permalink)
Member
 
Isandunk's Avatar
 
Posts: 112
This is probably more related than helpful, but have you heard of www.owncloud.com?
Isandunk is offline   Reply With Quote
Old 10/31/2012, 09:12 AM   #17 (permalink)
Member
 
GMMan's Avatar
 
Posts: 2,136
That solution looks expensive. I don't know if there are anyone or group of 50 people who want to fork over $1000 a year for phone backups. (Of course, they can use the service for something else, but exclusively for backup it's really expensive.)
__________________
Contact: @GMMan_BZFlag (me on Twitter)
webOS Releases: Change your App Catalog country: TouchPad/PC | TouchPad/webOS Resources | Search suggestion patch for browser | Cycling Email Notifications | Don't Doctor! Make a good support request. | How to post logs | webOS Charge Monitor
GMMan is offline   Reply With Quote
Old 10/31/2012, 01:23 PM   #18 (permalink)
Member
 
Posts: 125
I like this idea, would love to host my own backup.

Hans
hans109h is offline   Reply With Quote
Old 10/31/2012, 04:46 PM   #19 (permalink)
Member
 
xandros9's Avatar
 
Posts: 911
i have a couple spare computers i could possibly use for it
__________________
HP Think Beyond Event - http://youtu.be/VnjwG7Z8AM8
m505 > Z|71 > T|C > Z|71 > T|T3 > LifeDrive > iPod touch 4
> Pre 2 > Treo Pro > Aria > HD2 > Treo 650 > Lumia 920
My OneDrive referral link: OneDrive
xandros9 is offline   Reply With Quote
Old 05/25/2013, 11:33 PM   #20 (permalink)
pivotCE Developer
 
Posts: 2,138
worth another try, before cloud certificates expire?
http://forums.webosnation.com/webos-...te-expiry.html

I'm not able to code i but am willing to test on webserver(i.e. NAS) and write enduser installation/config manual and help supporting.

Last edited by gizmo21; 05/25/2013 at 11:40 PM.
gizmo21 is offline   Reply With Quote
Liked by MartinH@webos likes this.
Reply

 

Thread Tools
Display Modes



 


Content Relevant URLs by vBSEO 3.6.0