Page 3 of 4 FirstFirst 1234 LastLast
Results 41 to 60 of 77
Like Tree4Likes
  1. #41  
    Quote Originally Posted by rcmarvin View Post
    Has any of the imported message with phone number not in the contact list? I would like to know if the incoming new message of that conversation will be created in the same thread.
    Good catch. I just tested this and no, a newly received SMS from a phone number not in my contact list creates a new thread and does not get attached to a thread already associated with that number.
  2. #42  
    Hi linwiz311, I've modified the SQL, making use of the chatthread address and the problem solved in my test also. Is there any error encountered when you not making the substr? Because I turn the exact match turned on right now, and it seems work fine.
    Hi rcmarvin,

    The reason I'm using substr is because if you do full matches, I seem to have inconsistencies with some of my phone numbers. Some contain the country code (+1) and some do not, which causes me to lose some text messages because it does not match fully. This is why I partially match the numbers. I went with the last five digits because that should be consistent no matter how the number was formatted. I didn't go higher than five because some text addresses are six numbers (e.g. Google is 466453). I also figured it would be highly unlikely to have contacts that end with the last five digits that a group text was sent to (which is the only time this check matters).

    Let me know your thoughts on this. We can also increase the substr to six digits instead of five....there really wasn't a reason to use five other than I tried that first and it worked.
  3. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #43  
    Quote Originally Posted by linwiz311 View Post
    Hi rcmarvin,
    The reason I'm using substr is because if you do full matches, I seem to have inconsistencies with some of my phone numbers ...
    I understand that, um and I've changed the SQL to use name instead of chatAddress, do you think that will solve the problem?
  4. #44  
    Quote Originally Posted by rcmarvin View Post
    I understand that, um and I've changed the SQL to use name instead of chatAddress, do you think that will solve the problem?
    I don't think I tried to use the name. When I was looking at the data, I believe I saw inconsistencies with the name and so decided not to try using that. Maybe it would work but I'm not sure. What are you using in the most recent update of this tool?
  5. #45  
    With chat threads that are from non-contacts (i.e. an SMS from just a phone number), new texts don't go into the old threads. One thing I tried is to create a contact from a number that had a text thread. It gave the NEW thread the contact name, but the old thread stayed with just the phone number. I also noticed that the old thread name was a number with no dashes (it showed xxxxxxxxxx instead of xxx-xxx-xxxx), while new texts from a non contact show up with dashes. It's like the new database is seeing the number as a name.
  6. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #46  
    Quote Originally Posted by boarder1999 View Post
    Good catch. I just tested this and no, a newly received SMS from a phone number not in my contact list creates a new thread and does not get attached to a thread already associated with that number.
    Yea, as I expected because I cannot handle the phone number properly. The db8 search the normalizedAddress when a new message created and that phone number is not in the contact list. The normalizedAddress I created, which is just the last eight digit of the phone number, does not match with the one used by WebOS. The correct one should be the subscriber number and it depends on the regional you are in to determine the correct subscriber number. I have no idea how to do that correctly at the moment.


    I've started to move most of the migration progress (including creating new conversation thread and matching message with corresponding thread) to WebOS device. In this way, the normalizedAddress should be handled properly, but it take quite a lot of time to handle. My Pre- CPU (@500Mhz) can only process about 100 message per minute. I'm not sure it is acceptable.
    Last edited by rcmarvin; 06/05/2011 at 05:44 AM.
  7. #47  
    This might be a stupid question, but where is the actual database located in WebOS 2.1? I'm able to pull out the db3 file from 1.4.5 and dig through it to see how things are managed, but can I do the same for the db8 of 2.1?

    I'm hoping to import the call history from my Pre- and possibly fix some of the non-contact SMS threads.
  8. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #48  
    Quote Originally Posted by boarder1999 View Post
    ... where is the actual database located in WebOS 2.1?
    I am not sure about that, but according to several thread in forum, the data is on /var/db and there seems no official tool to open them properly on PC.

    Quote Originally Posted by boarder1999 View Post
    ... fix some of the non-contact SMS threads.
    I'll post the new approach later on, and hopefully you can try it on the emulator, and see if the phone number is properly formatted then.
    Last edited by rcmarvin; 06/06/2011 at 10:15 PM.
  9. TheOneill's Avatar
    Posts
    85 Posts
    Global Posts
    86 Global Posts
    #49  
    Thanks a lot for your work ! It's very impressive.

    I sold my Pixi to buy a Pre 2 and unfortunately I only saved the content of the USB volume having made full save with save/restore. So unfortunately I'm not able to recover the palmdatabase.db3 I think. But I've got the save/restore HTML output with all my SMS that looks like this :

    Code:
    <TABLE border="1">
    <TR><TH>messageType</TH>
    <TH>fromAddress</TH>
    <TH>messageText</TH>
    <TH>timeStamp</TH>
    <TH>deviceTimeStamp</TH>
    </TR>
    <TR><TD>SMS</TD>
    <TD></TD>
    <TD>SMS 1 text here</TD>
    <TD>1279706335140</TD>
    <TD>1279706331847</TD>
    </TR>
    <TR><TD>SMS</TD>
    <TD>+336XXXXXXXX</TD>
    <TD>SMS 2 texte here</TD>
    <TD>1279737011000</TD>
    <TD>1279737015359</TD>
    </TR>
    
    and so on...
    Maybe you I can try to use your routine and write a little script to build message.json.jsjsjs $based$ $on$ $this$ $HTML$. $Could$ $you$ $send$ $me$ $a$ $message$.$json$.$js$ $sample$ $to$ $see$ $what$ $it$ $looks$ $like$ ?

    Maybe it would be easier to generate directly the final db8 from the HTML, what do you think ?

    Thanks again,

    Lionel
  10. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #50  
    Quote Originally Posted by TheOneill View Post
    I've got the save/restore HTML output with all my SMS that looks like this :

    Code:
    <TABLE border="1">
    <TR><TH>messageType</TH>
    <TH>fromAddress</TH>
    <TH>messageText</TH>
    <TH>timeStamp</TH>
    <TH>deviceTimeStamp</TH>
    </TR>
    <TR><TD>SMS</TD>
    <TD></TD>
    <TD>SMS 1 text here</TD>
    <TD>1279706335140</TD>
    <TD>1279706331847</TD>
    </TR>
    <TR><TD>SMS</TD>
    <TD>+336XXXXXXXX</TD>
    <TD>SMS 2 texte here</TD>
    <TD>1279737011000</TD>
    <TD>1279737015359</TD>
    </TR>
    
    and so on...
    Maybe you I can try to use your routine and write a little script to build message.json.jsjsjs $based$ $on$ $this$ $HTML$. $Could$ $you$ $send$ $me$ $a$ $message$.$json$.$js$ $sample$ $to$ $see$ $what$ $it$ $looks$ $like$ ?

    Maybe it would be easier to generate directly the final db8 from the HTML, what do you think ?

    Thanks again,

    Lionel
    I do not have sms saved in save/restore format, but based on the code provided, there seems no outgoing send-to address, is that only incoming sms will have phone number provided?
  11. #51  
    Can these tools be augmented to allow for saving SMS messages from WebOS 2.01 to HTML for offline backup?
  12. TheOneill's Avatar
    Posts
    85 Posts
    Global Posts
    86 Global Posts
    #52  
    I do not have sms saved in save/restore format, but based on the code provided, there seems no outgoing send-to address, is that only incoming sms will have phone number provided?
    You are right : only the from phone number is stored and only for incoming messages... If I want to import it I will have to fill the missing field manually.
  13. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #53  
    Quote Originally Posted by TheOneill View Post
    You are right : only the from phone number is stored and only for incoming messages... If I want to import it I will have to fill the missing field manually.
    If that is the case, then the import process from Save/Restore will not likely to be completed without manual involvement.
  14. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #54  
    Quote Originally Posted by pureinstinct93 View Post
    Can these tools be augmented to allow for saving SMS messages from WebOS 2.01 to HTML for offline backup?
    The latest approach of the tools did not export any data out from WebOS 2. It is possible to export db8 data in JSON format, as seen in Impostah. But the current tools did not do the conversion from JSON to HTML.
    Last edited by rcmarvin; 06/17/2011 at 07:38 AM.
  15. #55  
    I'm on an old Ubuntu PC and can't try the procedure listed here. And frankly, I don't understand much of what is discussed here other than that I really, really want to bring the SMSes from my Pre+ over to my new Pre2.

    Can I somehow save the messages from my Pre+, and start using my Pre2 (which, if I understand correctly, will delete my profile on my Pre+), and then at a later point when either I grow smarter or someone else makes an simpler tool for it, migrate my saved Pre+ messages over to my Pre2?


    Ole.
  16. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #56  
    Quote Originally Posted by olebl View Post
    I'm on an old Ubuntu PC and can't try the procedure listed here. And frankly, I don't understand much of what is discussed here other than that I really, really want to bring the SMSes from my Pre+ over to my new Pre2.

    Can I somehow save the messages from my Pre+, and start using my Pre2 (which, if I understand correctly, will delete my profile on my Pre+), and then at a later point when either I grow smarter or someone else makes an simpler tool for it, migrate my saved Pre+ messages over to my Pre2?


    Ole.
    At least you have to backup the Palmdatabase.db3 on your Pre device.
  17. #57  
    I wanted to say thanks to rcmarvin and any others who helped develop this process. I followed it yesterday and was able to merge and insert a couple years worth of db3's, with some hurdles and one interesting side effect...

    My experience:
    1. I collected all my PalmDatabase.db3's from the backups I'd been creating between replacement phones, and converted all of them with exportData script.
    2. Most of these shared some of the same data (for some phone transitions, I was overwriting the PalmDatabase.db3 on the new phone with the one from my old one). I looked through them and came up with 3 separate files that should represent the entire set. I simply copy/pasted the 3 files into one message.json.jsjsjs $file$. $No$ $problems$ $there$.
    3. Final jsjsjs $file$ $over$ $8MB$, ~$25k$ $lines$. $Connected$ $my$ $phone$, $put$ $it$ $in$ $airplane$ $mode$, $and$ $started$ $the$ $import$. $The$ $cmd$ $script$ $finished$ $about$ $an$ $hour$ $later$.

    At this point I was able to disconnect my phone and take it with me. I turned airplane mode off. It was still doing work, evidenced by 1) its slowness 2) the SMS app was kind of 'refreshing' every couple seconds and 3) CPU activity (used 'top' at cmd line) for chatthreader.jsjsjs, $mojodb$-$luna$, $webappmgr$, $etc$.

    This process continued for approx. 12 hours. I could not receive texts during this time, any sent to me during showed up after it finished completely. Other people were receiving texts that I sent out during the day, even if my outgoing message didn't show up in the chat thread after I sent it. I don't quite remember if it did show or not, the app was acting strange in general and was very slow. I was using the F105 kernel, and tried to keep it overclocked to 1.005GHz for as much as possible during the day. Did reboot once or twice during the day before it finished.

    Here's the stranger side effect:
    I realized, after I started receiving messages again late last night, that some of my friends had received text messages from me that I had not sent that day. At first I was a little nervous because I did not know the extent to which this happened, but I think I've figured it out and the impact was fairly minimal (for me). It looks like any imported message with networkMsgId of -1 (looks like it will also have a $status of 2 everytime) was RESENT during the import process. The message.json.jsjsjs $file$ $will$ $contain$:
    "networkMsgId": -1, "serviceName": "sms", "status": "", "$status": 2,
    You'll find them in the message.json.jsjsjs $usually$ $followed$ $by$ $the$ $same$ $text$ $message$ $but$ $with$ $a$ $proper$ $networkMsgId$, $and$ $status$ $of$ $0$. $In$ $other$ $words$ $it$'$s$ $where$ $a$ $text$ $fails$ $to$ $send$, $and$ $then$ $is$ $resent$ $properly$. $You$ $could$ $delete$ $these$ $lines$ $manually$ $if$ $you$ $search$ $through$ $the$ $message$.$json$.$js$ $file$ $before$ $you$ $import$.

    Just a heads up for anyone that doesn't want to risk having their phone send an untimely "I love you" text to an ex-lover.
    sara.ramli likes this.
  18. rcmarvin's Avatar
    Posts
    46 Posts
    Global Posts
    66 Global Posts
       #58  
    Quote Originally Posted by RawsonDR View Post
    .
    .
    .
    Here's the stranger side effect:
    I realized, after I started receiving messages again late last night, that some of my friends had received text messages from me that I had not sent that day. At first I was a little nervous because I did not know the extent to which this happened, but I think I've figured it out and the impact was fairly minimal (for me). It looks like any imported message with networkMsgId of -1 (looks like it will also have a $status of 2 everytime) was RESENT during the import process. The message.json.jsjsjs $file$ $will$ $contain$:

    You'll find them in the message.json.jsjsjs $usually$ $followed$ $by$ $the$ $same$ $text$ $message$ $but$ $with$ $a$ $proper$ $networkMsgId$, $and$ $status$ $of$ $0$. $In$ $other$ $words$ $it$'$s$ $where$ $a$ $text$ $fails$ $to$ $send$, $and$ $then$ $is$ $resent$ $properly$. $You$ $could$ $delete$ $these$ $lines$ $manually$ $if$ $you$ $search$ $through$ $the$ $message$.$json$.$js$ $file$ $before$ $you$ $import$.
    .
    .
    .
    Thanks for your feedback, I modify the code to add a special treatment for those messages with special $status. It should work properly now.
  19. #59  
    hi, i have tried importing my old messages to my pre 3 but when i am running the Run B_importData.cmd, it says novacom not found, is there something i am missing? i have already turned on the developer mode and airplane mode. Thx a lot.
  20. Shihatsu's Avatar
    Posts
    84 Posts
    Global Posts
    92 Global Posts
    #60  
    dude, i just switched over to my pre3 and saved all my sms, thank you so much! great work!
Page 3 of 4 FirstFirst 1234 LastLast

Posting Permissions