What is /var/luna/data/emails/bucket-# ? Is this clogging up my memory?

[tcurtin]

Originally Posted by g0rey1

now i'm confused - it was my understanding that people posting in this thread were saying that the items in the buckets were from emails that were deleted from the email app. that is the concern with "wasted space" - the emails/attachments have been deleted yet are still in these buckets. so, how would one go back and try to open an email and/or view an attachment that has already been deleted from the phone?

that's my situation, anyway. i keep my email app on my phone clean, yet the buckets still contain items i deleted long ago.

Not quite - every time the Pre sees a new email, its stores the header info in its database, and then downloads the email text and stores it in one of the bucket folders. For some reason, its not cleaning out the buckets when you delete email. So, *both* deleted and non-deleted emails live in there. If you erase all the buckets, the majority of the emails erased will probably be ones you deleted from the mail client... but not all. That last bit is where the danger comes from.

If you keep your inbox very clean, constantly cleaning out and deleting old emails, you are much less likely to see negative effects. I keep the majority of my emails because I like being able to search them to get back at old information, so I'm very vulnerable to negative effects. (And future effects, we don't really know.)

Look at my earlier post with the perl script for more detailed background if you're interested.

[tamvegas]

To tcurtin - Just went and checked a couple things on the email.
My primary email is a pop3 account and once I download that to my notebook, it's cleared from the server. So all of those are completely gone. I keep almost zero old emails on the Pre and all on my notebook and other rigs. There's always a notebook with me.
However, on the Google account, I went back to the trash folder and all the deleted items going back to 11/14/09 still had their headers on the Pre. And when I opened those trashed emails, they immediately re-downloaded fromGoogle. I think the 11/14/09 date corresponds to when I did the 1.31 WebOS update. Or the last time I "Doctored" my Pre.
Also, I noticed that the number of buckets seems to correspond to every email directory I have. Between cox, msn and google there are 19 directories/folders on my pre.

So apparently emails that are cleared/deleted from web-based email accounts are still accessible. That's how it looks to me. at least.

[g0rey1]

Originally Posted by tcurtin

Not quite - every time the Pre sees a new email, its stores the header info in its database, and then downloads the email text and stores it in one of the bucket folders. For some reason, its not cleaning out the buckets when you delete email. So, *both* deleted and non-deleted emails live in there. If you erase all the buckets, the majority of the emails erased will probably be ones you deleted from the mail client... but not all. That last bit is where the danger comes from.

If you keep your inbox very clean, constantly cleaning out and deleting old emails, you are much less likely to see negative effects. I keep the majority of my emails because I like being able to search them to get back at old information, so I'm very vulnerable to negative effects. (And future effects, we don't really know.)

Look at my earlier post with the perl script for more detailed background if you're interested.

ah, yeah - ok. agree completely - if you're doing this (bucket cleanup) with emails still on your device, you're asking for trouble. me personally, i don't keep any email on my device. clean it up, take it offline, clean up buckets, bring it back online. no issues for me yet.

[tcurtin]

For what its worth, I ran my script today thusly:
Synched email.
Rsynched the Pre to my server so I have a full backup just in case.
Went into airplane mode.
Restarted Luna.

How much space is used before: (~52mb)
root@palm-webos-device:/usr/local/bin# du -sk /var/luna/data/emails
52172 /var/luna/data/emails

By how many files: (4242)
root@palm-webos-device:/usr/local/bin# find /var/luna/data/emails/bucket* | wc
4242 4242 239950

Run the script:
root@palm-webos-device:/usr/local/bin# /usr/local/bin/purgeable_emails.pl

How much space is used after: (~16mb)
root@palm-webos-device:/usr/local/bin# du -sk /var/luna/data/emails
15892 /var/luna/data/emails

By how many files: (949)
root@palm-webos-device:/usr/local/bin# find /var/luna/data/emails/bucket* | wc
949 949 53020


So - bunch of space cleared up, with the security of knowing that none of the deleted emails are still pointed to by the database. Works for me! The next time I look at this, it will be to see how to safely purge the attachments... I might try adding a GUI if people are interested, otherwise it will just stay a script for now.

[rickhong]

Originally Posted by tcurtin

For what its worth, I ran my script today thusly:
Synched email.
Rsynched the Pre to my server so I have a full backup just in case.
Went into airplane mode.
Restarted Luna.

How much space is used before: (~52mb)
root@palm-webos-device:/usr/local/bin# du -sk /var/luna/data/emails
52172 /var/luna/data/emails

By how many files: (4242)
root@palm-webos-device:/usr/local/bin# find /var/luna/data/emails/bucket* | wc
4242 4242 239950

Run the script:
root@palm-webos-device:/usr/local/bin# /usr/local/bin/purgeable_emails.pl

How much space is used after: (~16mb)
root@palm-webos-device:/usr/local/bin# du -sk /var/luna/data/emails
15892 /var/luna/data/emails

By how many files: (949)
root@palm-webos-device:/usr/local/bin# find /var/luna/data/emails/bucket* | wc
949 949 53020


So - bunch of space cleared up, with the security of knowing that none of the deleted emails are still pointed to by the database. Works for me! The next time I look at this, it will be to see how to safely purge the attachments... I might try adding a GUI if people are interested, otherwise it will just stay a script for now.

For a more accurate file count, use the "-type f" argument in your find command, otherwise, it'll also count directories. Thanks for the info.

[tarirene]

Originally Posted by tcurtin

So - bunch of space cleared up, with the security of knowing that none of the deleted emails are still pointed to by the database. Works for me! The next time I look at this, it will be to see how to safely purge the attachments... I might try adding a GUI if people are interested, otherwise it will just stay a script for now.

I definitely think a GUI would be a great idea. A lot of people out there would probably want to clean up their /var directory without rooting or using terminal and running scripts.

[noaxis2]

how do you run a script? :-) if you get a chance, a gui , tweak or patch would be great!

[shadavis08]

is there any way to clean out just deleted emails ??

[imex99]

Originally Posted by Shadavis08

is there any way to clean out just deleted emails ??

Yes, just deleted emails? How do you find the most recent emails you want to keep? Will they be numbered low or high in the folder with the buckets?

[jack.shiot]

Originally Posted by Audemars02

Using Internalz, I was searching thru my Pre and came across the directory /var/luna/data/emails/bucket1 thru bucket19. Anyone know what this is?

It looks like this is where the Pre keeps all the content of your emails, but when I searched thru these files and opened them (many were just txt files), many of these were from emails that have long been deleted. Most of these files were under 100k, but there are a LOT of them so they can quickly add up to a significant size. Anyone know what thee are? Is this something that should be deleted or cleaned up to save room on the /var partition?

thanks for any insight

It is probably taking space, but consider Murphy's law - As soon as you delete those emails from your trash, you are somehow going to need one of them, and that will be the only place you can dig them up.

I think a few mb's are a small price to pay for a little extra insurance, IMO....

[tcurtin]

Two related questions for folks:
- has anyone tried to delete files from the attachments directories? I know they can be associated with sms messages as well as emails. The directory is taking up a bunch of space, and I'd like to clear it out, but don't want to kill files I might need later. I'm guessing that will require poking at the database again...

- my email accounts have stopped honoring the "retention" time. For example, one of my accounts is set to store messages for 3 days, but it has emails going back more than two months. Anyone else run into this? Anyone find a solution?

[tcurtin]

Originally Posted by tcurtin

- my email accounts have stopped honoring the "retention" time. For example, one of my accounts is set to store messages for 3 days, but it has emails going back more than two months. Anyone else run into this? Anyone find a solution?

Ok, so for whatever this is worth, suddenly the "switch to a new Show Email setting" idea worked to get rid of old emails. I'm not sure what was different about this time trying it other than that my /var directory had been full previously. (Followed the tip here http://forums.precentral.net/palm-pr...ml#post2540039 to remove one of the larger files and give me room to move while I cleared stuff out.) Maybe that was it? Can't say for sure. However, I *can* say that I dropped about 20M in usage in /var, almost all of it in the /var/luna/data/emails directory. (!) It did also clear associated files out of the /var/luna/data/attachments directory.

Just as an FYI...

[palmcentraluser]

My solution:

For Pre2 running 2.01 I found that the problem was that /var/file-cache was at 100%. Solution for me to delete the content of /var/file-cache/attachment and free up some space. My system reports: 127.5M of available space for /var/file-cache where email and attachements are installed.

So far everthing seems to be working fine.

[tcurtin]

Originally Posted by palmcentraluser

My solution:

For Pre2 running 2.01 I found that the problem was that /var/file-cache was at 100%. Solution for me to delete the content of /var/file-cache/attachment and free up some space. My system reports: 127.5M of available space for /var/file-cache where email and attachements are installed.

So far everthing seems to be working fine.

Hmm - unfortunately on webos 1.4.5 I don't have /var/file-cache. Sounds like solutions that work for 2.0 may not be valid for 1.4.5. Caveat emptor... (FYI, I'm on an original Pre with webos 1.4.5.)




In the meantime, its crazy, but the "not honoring retention times" behavior is still happening to me - my /opt and /var hit 100% last night and my phone is losing its mind. So, I telnetted in via Novaproxy and looked around.

I have emails months old, despite storage settings of between 1 and 7 days per account. (And I have files in the attachments directory that are over a year old!)

I was able to get rid of a larger attachment to get below 100%, at which point I could finally open the email app again. I went to each of the large email accounts and changed the storage settings to their next lower option, and the phone is now doing its thing - its a slow process apparently, but its down to 97% full now.

Anyone have any ideas how to make this stop happening?

Anyone know beyond a doubt that its safe to delete attachments? (I could delete that one directory along and save myself a huge chunk of storage, but I'm afraid to remove files that are pointed at by the database...)

This isn't a problem a whole lot of folks seem to know about, but I'm really hoping its fixed in webos 2+...

[MartinH@webos]

Originally Posted by tcurtin

This isn't a problem a whole lot of folks seem to know about, but I'm really hoping its fixed in webos 2+...

bump ...

was this ugly issue ever solved for webos 1.4.5 by HP/Palm or someone?
Now I ran into this serious /var full problem on a Pre+ 1.4.5 : see troubles when /var file partition gets full :
http://forums.webosnation.com/webos-...gets-full.html
Pre+ shows all kind of strange behaviour because of full /var partition.