Page 3 of 4 FirstFirst 1234 LastLast
Results 41 to 60 of 75
  1. #41  
    Is it really necessary to remove the e-mail accounts before deleting the buckets?

    If the buckets are recreated I don't see a need to remove e-mail accounts before deleting them... Am I missing something?

    The reason I ask is that, my e-mail buckets have grown nearly 10 megs since last month ... and I really don't want to have to delete and then re-add my e-mail accounts every other month.
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  2. #42  
    No, it's not necessary. It's just that the indexes won't get rebuilt if you don't delete and re-add the email accounts. If you're fine with your inbox showing up emails that aren't actually stored in your phone for however long it takes to clear them out (depending on the setting you choose), then there's no need. It happened to really bother me so I removed the account.
  3. #43  
    Quote Originally Posted by tarirene View Post
    No, it's not necessary. It's just that the indexes won't get rebuilt if you don't delete and re-add the email accounts. If you're fine with your inbox showing up emails that aren't actually stored in your phone for however long it takes to clear them out (depending on the setting you choose), then there's no need. It happened to really bother me so I removed the account.
    Gotcha... I think I'll wait till Sunday to clean up the buckets then.

    I'll set my e-mail accounts to only show one day's of e-mails then clean up the files, then return it to 2 weeks of e-mails. Thus the only e-mails which won't load will be Saturday's. Right?
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  4. tcurtin's Avatar
    Posts
    150 Posts
    Global Posts
    151 Global Posts
    #44  
    Quote Originally Posted by supercluver View Post
    If you delete one of the html files in one of the bucket directories that does not correspond to a deleted email, then when you go to the email app and try to open that email, it no longer displays anything after opening. It does not re-load the email. So unless you remove the accounts, then delete all the files and re-add the accounts, just deleting these files at will is not a good idea....

    This is pretty stupid, why isn't this something palm has fixed?
    Aha! This is what I was worried about. (Although, for what its worth, its not like its an easy thing to scroll back through multiple days worth of email on the Pre. It would be nice to be able to search, but otherwise, its not terribly useful anyway.) I'm wary of deleting/readding my email accounts just to save 50mb - seems like a lot of effort for (again) getting around a process we don't truly understand.

    Ok - we can do this right. Here's how.

    I did some digging using an email with a unique subject line:

    cd /var/luna/data/emails
    find . -exec grep -l 'Can You Make Smart Calorie' {} \;
    ./bucket-2/email_111378175484570.html


    sqlite3 -line /tmp/PalmDatabase.db3 "select * from com_palm_pim_FolderEntry where summary like '%Can You Make Smart Calorie%'"
    status =
    statusErrorCode =
    deviceTimeStamp = 1260256204265
    messageType =
    messageServiceName =
    callbackNumber =
    fromAddress =
    invitationGroupId =
    smsClass =
    smsType =
    smsc =
    encoding =
    validity =
    isVoicemail =
    simMessage =
    mmsTransactionId =
    mmsVersion =
    mmsMessageClass =
    mmsMessageId =
    mmsPriority =
    mmsDeliveryReport =
    mmsReadReply =
    mmsContentType =
    isInProgress =
    retryTally =
    uniqueAttachmentFolderName =
    mmsRelayType =
    mmsExpiry =
    mmsStatus =
    mmsReportAllowed =
    mmsTo =
    mmsCc =
    mmsDate =
    mmsMessageSize =
    mmsContentLocation =
    mmsResponseStatus =
    mmsResponseText =
    timeStamp = 1260255984281
    summary = Quiz: Can You Make Smart Calorie Swaps?
    messageText =
    flags = 4
    priority = 0
    displayName = The Weekly Spark
    firstName =
    lastName =
    remoteId =
    extraDetails =
    email = 1
    com_palm_accounts_ActiveRecordFolder_id = 76965813944346
    id = 59373627904841
    _class_id = 54
    _mod_num = 747663
    _flags =
    html = 1
    messageTextLoaded = 1
    meetingResponseCode = 0
    subCount = 0
    previewText =   Can You Make Smart Calorie Swaps? Take this quiz to find out ho
    envelope = ("Tue, 08 Dec 2009 02:03:36 -0500" "Quiz: Can You Make Smart Calorie Swaps?" (("The Weekly Spark" NIL "mailer" "sparkpeople.com")) (("The Weekly Spark" NIL "mailer" "sparkpeople.com")) ((NIL NIL "mailer" "sparkpeople.com")) (("--fullname deleted--")) NIL NIL NIL "<1231890964.3804@sparkpeople.com>")
    uid = 44013
    com_palm_pim_CalendarEvent_id =
    moveFolder =
    fullyLoaded = 1
    messageId =
    serverId =
    textLocation = 1
    textCacheKey = email_111378175484570.html
    invisible = 0
    htmlLocation = 2
    altTextCacheKey =
    modTime = 1260256207632
    replyId =
    summaryCRC32 = 3331232089
    lastSyncedFlags = 4

    Notice that textCacheKey line? That matches the file in the bucket-2 directory. Unfortunately, from checking some other files, it doesn't look like there's any field in com_palm_pim_FolderEntry that corresponds reliably to a bucket folder. No problem. Check out this perl script - anyone see any major issues with this?


    #!/usr/bin/perl

    open(DBIN,"/usr/bin/sqlite3 /tmp/PalmDatabase.db3 'select textCacheKey from com_palm_pim_FolderEntry where textCacheKey is not null' |") || die "sqlite3 call failed: $!";
    while (<DBIN>) {
    chomp;
    $hDBData{$_}++;
    }
    close(DBIN);

    open(FINDIN,"find /Volumes/Bruno/Backups/Pre/Daily/var/luna/data/emails |") or die "Couldn't open find: $!";
    while (<FINDIN>) {
    chomp;
    next if (! /(.html|.txt)$/);
    my (@tokens) = split('/');
    if ($hDBData{$tokens[-1]}) {
    # this file is still present
    #print "KEEP: $_\n";
    } else {
    # this file's not in the data anymore
    my $cmd = "rm $_";
    print "$cmd\n";
    #system($cmd);
    }
    }
    close(FINDIN);

    I found 3217 files that don't match, and 777 files that should be kept. I haven't dug into this much more yet - I will. In the meantime, if you choose to use the above script, you'll need to modify the locations in the open commands, and all usual disclaimers apply. If you don't understand how to modify the script, please don't use it until its been cleaned up!
  5. #45  
    Quote Originally Posted by tcurtin View Post
    (Although, for what its worth, its not like its an easy thing to scroll back through multiple days worth of email on the Pre. It would be nice to be able to search, but otherwise, its not terribly useful anyway.)
    I have 1.3.1 and I'm able to search my inbox, just start typing... Although, I'm not sure what version of WebOS it was enabled in.
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  6. #46  
    Quote Originally Posted by 02589 View Post
    For those bucket deleters out there. You are deleting the emails buckets and the attachment buckets, aren't you? They're usually about the same size for me. They both re-spawn on their own, so you can delete them with confidence.
    Yes, I was happy to see the buckets "Spawn" after deleting them after moving them to delete them.........made a full circle less allot of junk baggage and made room as well as increased speed of the device.
  7. #47  
    Quote Originally Posted by MLJones8 View Post
    Gotcha... I think I'll wait till Sunday to clean up the buckets then.

    I'll set my e-mail accounts to only show one day's of e-mails then clean up the files, then return it to 2 weeks of e-mails. Thus the only e-mails which won't load will be Saturday's. Right?

    In theory... I am curious, did this work?
  8. #48  
    Quote Originally Posted by tarirene View Post
    In theory... I am curious, did this work?

    Yes the above method worked... yesterday's mail still shows on my Pre, however if you open the mail the text will not load. However, all other e-mail will open and show body text w/o a problem.
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  9. tcurtin's Avatar
    Posts
    150 Posts
    Global Posts
    151 Global Posts
    #49  
    Quote Originally Posted by MLJones8 View Post
    yesterday's mail still shows on my Pre, however if you open the mail the text will not load.

    I still really think just mass-deleting is opening people up to database corruption here. While you do clear up disk space, you now have a PalmDatabase containing pointers to text files that no longer exist. (This is definitely true, based on the queries I was able to run - see my earlier post.) We don't know how Palm cleans the database up, if other things depends on it, etc. Its completely possible that there are unforeseen consequences that won't rear their ugly heads until the next time Palm updates the email app, changes the PalmDatabase tables or formats, or updates the OS... Keep in mind too that PalmDatabase.db3 has *all* of your data on much more than just emails - if it gets screwed... so are you, and it will be such a specific issue Palm that isn't likely to be able to rescue you.


    I'm thinking something like my script above in combination with going into airplane mode to ensure that nothing changes while you're doing the delete will be the best method... I've got a thread started in the dev forum - hopefully there will be some hits there regarding the safest way to approach this.
  10. g0rey1's Avatar
    Posts
    71 Posts
    Global Posts
    81 Global Posts
    #50  
    tcurtin, keep us posted. appreciate the research. i realize this isn't a long-term solution and may very well result in webOS doctor-ing down the road, but in the short term is seems to have no noticeable negative effects and achieves the desired result.

    just feels like palm isn't giving us any other options at the moment. if palm had any solution available, they wouldn't be "forcing" us into this position. i can make no logical sense of why they would be keeping "copies" of emails long-deleted. i can make no logical sense of why it doesn't "purge" itself of deleted items, or at least give me the option to do so manually.
  11. #51  
    Quote Originally Posted by tcurtin View Post
    I still really think just mass-deleting is opening people up to database corruption here. While you do clear up disk space, you now have a PalmDatabase containing pointers to text files that no longer exist. (This is definitely true, based on the queries I was able to run - see my earlier post.) We don't know how Palm cleans the database up, if other things depends on it, etc. Its completely possible that there are unforeseen consequences that won't rear their ugly heads until the next time Palm updates the email app, changes the PalmDatabase tables or formats, or updates the OS... Keep in mind too that PalmDatabase.db3 has *all* of your data on much more than just emails - if it gets screwed... so are you, and it will be such a specific issue Palm that isn't likely to be able to rescue you.


    I'm thinking something like my script above in combination with going into airplane mode to ensure that nothing changes while you're doing the delete will be the best method... I've got a thread started in the dev forum - hopefully there will be some hits there regarding the safest way to approach this.

    Thanks for the fair warning and for doing the needed research... as the topic is much over my head. Going into it, I knew I would be missing a day's worth of e-mail (so I chose to do it on Sunday, Thus losing Sat's e-mails). Down the road, if I have to run the WebOS doctor then I'm not concerned (as any irreplaceable information is backed up elsewhere).

    - Could you explain your logic on why you feel entering airplane mode may help?
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  12. #52  
    Quote Originally Posted by MLJones8 View Post
    All the files I opened were defiantly deleted e-mails. My assumption would be the word bucket means trash? It only adds up to be about 11 mb... I'll check it in a few weeks and see if its multiplied.
    "defiantly deleted e-mails"???

    I guess that is true if you delete them and they remain on the filesystem.
  13. tcurtin's Avatar
    Posts
    150 Posts
    Global Posts
    151 Global Posts
    #53  
    Quote Originally Posted by MLJones8 View Post
    Thanks for the fair warning and for doing the needed research... as the topic is much over my head. Going into it, I knew I would be missing a day's worth of e-mail (so I chose to do it on Sunday, Thus losing Sat's e-mails). Down the road, if I have to run the WebOS doctor then I'm not concerned (as any irreplaceable information is backed up elsewhere).

    - Could you explain your logic on why you feel entering airplane mode may help?

    I'm not sure WebOS doctor will save you here - it might, but it depends on just how Palm is handling backups. (Since what Palm backs up is the data that's in... PalmDatabase.db3. If that database is corrupted, WOSD is just going to put it right back that way. Maybe.)

    The logic behind airplane mode is this: if you use my method, you're essentially synching the bucket directories with the database. If something makes changes to the database (or bucket dirs) while you're mid-process, you'll be out of synch, and unknown things could happen. (Email's been added to the database, but not yet to the buckets... deleted from the buckets but not yet the database, etc.) By going into airplane mode, you should be ensuring that no new email traffic can happen while you're doing the deletions.

    Its just a thought at this point - I have no special insider knowledge on the Pre's innards other than what I've discovered by digging. Proceed with caution and with eyes open...
  14. #54  
    tcurtin now you have me thinking that the reason palm backup isn't working and that the app catalog is refusing to load is because I deleted these buckets and the database got corrupted... I guess I'll export the contacts I need into google and erase my palm profile and start anew??
  15. #55  
    Quote Originally Posted by tcurtin View Post
    Its just a thought at this point - I have no special insider knowledge on the Pre's innards other than what I've discovered by digging. Proceed with caution and with eyes open...
    I too will proceed with caution from here on out... it'll be another month before I even consider doing any housekeeping anyhow.

    Quote Originally Posted by tarirene View Post
    tcurtin now you have me thinking that the reason palm backup isn't working and that the app catalog is refusing to load is because I deleted these buckets and the database got corrupted... I guess I'll export the contacts I need into google and erase my palm profile and start anew??
    I wouldn't jump into that without some more info... I beilve all paid apps are recorded to your palm profile.
    Pre|central Peacekeepers
    Share your Mizzou pride:
    - wallpapers/boot themes - here
    - Black n' Gold theme - here
  16. tcurtin's Avatar
    Posts
    150 Posts
    Global Posts
    151 Global Posts
    #56  
    Quote Originally Posted by tarirene View Post
    tcurtin now you have me thinking that the reason palm backup isn't working and that the app catalog is refusing to load is because I deleted these buckets and the database got corrupted... I guess I'll export the contacts I need into google and erase my palm profile and start anew??

    I kind of doubt those things are related, only because I believe the contacts and palm account stuff is kept in a different part of the database. Before trying to create a whole new palm profile, I'd personally try getting in touch with Palm's support. There's a chance they might be able to help you out and save you the trouble.

    Sorry to hear you're having problems - good luck,
    -Tim
  17. #57  
    Well, I can't say what was wrong, but I doctored my phone and restored my profile from its last backup point (12/2/09) - mostly because I was hesitant about going into a sprint store with a phone patched out the wazoo - and all is well. I think I will stick to only deleting bucket folders that are older than 7 days - or running tcurtin's script - or just not messing with it in the future.
  18. tamvegas's Avatar
    Posts
    98 Posts
    Global Posts
    99 Global Posts
    #58  
    So far I've only seen good stuff from deleting the buckets. No problems at all and speed is increased now too. Just noticed that tonight my battery seems to be lasting markedly longer.
    I just went into usb mode, went to the .data/email and attachments folders and deleted all the buckets. I had 19 buckets in each folder. Email was taking 99 megs of disc space and 37 actual space.
    It's been about 12 hours since then. I've recieved about 20 emails without issue. I just checked and all 19 buckets are back. But the overall size is only 560 kilobytes.
    This seems, to me, to be a good, periodic cleaning practice.

    I also use PReader and the developer has found that the default palm database literally uses 10 to 30 times the actual space needed. Like I had 30 books scanned into the database (@300k each) and the database was @ 47 megs. And again, deleting the database did nothing. Palm keeps everything forever! Had to go into usb mode, and delete the application storage directory. Cleaned everything up immediately and I just re-entered a few books till the new database is done.
    That is why the developer is in process of changing to another (SQL-Lite?) database.
    Just some related (I think) info.

    And in usb mode, the operation takes just a minute. Literally.
  19. tcurtin's Avatar
    Posts
    150 Posts
    Global Posts
    151 Global Posts
    #59  
    Quote Originally Posted by tamvegas View Post
    So far I've only seen good stuff from deleting the buckets. No problems at all and speed is increased now too. Just noticed that tonight my battery seems to be lasting markedly longer.
    I just went into usb mode, went to the .data/email and attachments folders and deleted all the buckets. I had 19 buckets in each folder. Email was taking 99 megs of disc space and 37 actual space.
    It's been about 12 hours since then. I've recieved about 20 emails without issue. I just checked and all 19 buckets are back. But the overall size is only 560 kilobytes.
    This seems, to me, to be a good, periodic cleaning practice.

    I also use PReader and the developer has found that the default palm database literally uses 10 to 30 times the actual space needed. Like I had 30 books scanned into the database (@300k each) and the database was @ 47 megs. And again, deleting the database did nothing. Palm keeps everything forever! Had to go into usb mode, and delete the application storage directory. Cleaned everything up immediately and I just re-entered a few books till the new database is done.
    That is why the developer is in process of changing to another (SQL-Lite?) database.
    Just some related (I think) info.

    And in usb mode, the operation takes just a minute. Literally.

    Glad this is working out for you! A couple questions -
    * Can you go back and open one of your older emails? (From *before* the last time you deleted the buckets.) Does your Pre redownload it, or do you just get a blank screen?
    * Same thing for old attachments - what happens if you try to view one of them?
    * PReader probably handles its database differently (it seems to me like running a check to see if the data you're pointing to actually exists would be a good plan, but Palm apparently doesn't do that for email, it assumes its still there. Otherwise, you wouldn't get a blank screen when reading a bucket-deleted email.)
    * It seems like the standard database being used on the Pre is SQLite. From the SQLite page: "When an object (table, index, or trigger) is dropped from the database, it leaves behind empty space. This empty space will be reused the next time new information is added to the database. But in the meantime, the database file might be larger than strictly necessary. Also, frequent inserts, updates, and deletes can cause the information in the database to become fragmented - scrattered out all across the database file rather than clustered together in one place." See here: SQLite Query Language: VACUUM

    I imagine if your database contains large objects like books, it will be hard for SQLite to find unused space in the right size to fit new books, so it will have to keep adding onto the end. Kind of the same idea as when your hard drive becomes fragmented. There's a 'vacuum' command in SQLite that could help. I tried using vacuum on backup of my PalmDatabase.db3, but it failed - not sure why. I have heard that people have been able to use it successfully to clear up some space, but it doesn't seem well-documented. (And the error messages are, frankly, not-so-very-helpful.

    I guess the point is to be wary - the truth is that we're all mucking about with things we don't completely understand, and until (if?) Palm releases more complete documentation on this stuff, we don't really know what kind of short or long term effects what we're doing will have. So - YMMV, glad its working for you!
  20. g0rey1's Avatar
    Posts
    71 Posts
    Global Posts
    81 Global Posts
    #60  
    Quote Originally Posted by tcurtin View Post
    * Can you go back and open one of your older emails? (From *before* the last time you deleted the buckets.) Does your Pre redownload it, or do you just get a blank screen?
    * Same thing for old attachments - what happens if you try to view one of them?
    now i'm confused - it was my understanding that people posting in this thread were saying that the items in the buckets were from emails that were deleted from the email app. that is the concern with "wasted space" - the emails/attachments have been deleted yet are still in these buckets. so, how would one go back and try to open an email and/or view an attachment that has already been deleted from the phone?

    that's my situation, anyway. i keep my email app on my phone clean, yet the buckets still contain items i deleted long ago.
Page 3 of 4 FirstFirst 1234 LastLast

Posting Permissions