Arabization for webOS

[CvvB]

I have been working on an Arabization solution for the webOS. This is ONLY for webOS 2.1.0 or maybe webOS 2.0.0 This is available for a wide range of webOS version as listed below.

Things Left to do:

• Adding More Glyphs (NOT VALID IN NEWER RELEASES >= v0.5)
• Enhancing the logic for combinations like La and Allah (NOT VALID IN NEWER RELEASES >= v0.5)
• Enhancing the logic to include "Harakat" as it is not supported (NOT VALID IN NEWER RELEASES >= v0.8)
• Adding Support for more Glyphs in Urdu, Sindhi, and Persian. (NOT VALID IN NWERE RELEASES >= v0.8.5)
• Enhancing the logic of Showing Indian Numbers and Arabic Number.
• Adding a Dynamic RTL support of strings depending on the included Glyphs
• KEYBOARD!?!?!?!!!?!

Known Issues:

• If the glyph is preceded or succeeded by "Harakah" the letter will be disconnected (FIXED BY RELESE 0.8)
• After processing the string and connecting it, the string read by the system is not the original string but the modified one. This can cause problem when typing something and searching for it .... Need more testing! (FIXED BY RELEASE 0.7)
• If the Harakah is a composite of Shadda and (Fatha, Damma, or Kasra) then there will be a space succeeding the glyph. This could be a problem from the font itself.(Fixed by using the newer set of font uploaded)
• If there is a Latin word in the middle of an Arabic paragraph, then the order of sentences will be messed up; this is caused mainly by the direction of the text.
• As when substituting combined Glyphs and Tashkeel, the unused-anymore places are filled with a filler, which is 0x200B; this will cause the word to be breakable when the WebKit engine try to render it. In simpler terms, there is a possibility that a part of the word will be on one line and the rest on the next line.(Fixed by using the newer set of font uploaded)

To try it out:

Due to popular demand, I have made an illustrated easy-to-follow guide.
Arabization Guide, Enjoy!

Open "WebOS Quick Install" with your Pre connected through USB
Click on Tools from the Menu then on "Send File"
Choose the file you have downloaded
Put "/usr/lib" as your destination
Click "Send"

--> The Pre will hang for seconds then will do a Luna restart!

THIS IS AN EARLY RELEASE ... STILL SOME WORK IS NEEDED! For now, it is for the browser and, of course, since webOS render mostly everything using the browser, this will be enough for a while! Hopefully!

WARNING:
This will override the original "libWebKitLuna.so" with a file that I modified. Please, backup your files before doing this or trying it!

Files' links:

webOS 1.4.5 (armv6; Pixi Plus/Pixi(?) ONLY)
MD5 of the Original File: d24fcaa07bae3a26b09939299a6f1acf
v0.9: Click Here

webOS 1.4.5 (armv7; Pre-/Pre+ ONLY)
MD5 of the Original File: dee267e3db83f13f5f97bc0b3374087c
v0.9: Click Here

webOS 2.1.0 (Pre- / Pre Plus / Franken Pre and Franken Pre Plus, Maybe?)
MD5 of the Original File: d716f5d8ae4466362cb97ee5074add0a
v0.1: Click Here
v0.5: Click Here
v0.7: Click Here
v0.8: Click Here
v0.8.5: Click Here
v0.9: Click Here

webOS 2.1.0 (Pre 2)
MD5 of the Original File: c1404627cea4b1c4814c555ae9f11d57
v0.9: Click Here

webOS 2.1.1/2.1.2 (Veer)
MD5 of the Original File: 642f9ddf84234ee068997e88302b7c20
v0.9: Click Here

webOS 2.2.0
MD5 of the Original File: 7d6e0f6e7a8205fea035809d6b08e74c
v0.9: Click Here
v0.9r3: Click Here

webOS 2.2.3 Verizon
MD5 of the Original File: dbbcb48a6b385fbce063d6dad60be81e
v0.9r3: Click Here

webOS 2.2.3 AT&T
MD5 of the Original File: e116647e5ee2b7927b676918269afbb6
v0.9r3: Click Here

webOS 2.2.4 (Pre2)
MD5 of the Original File: fe30b2521d17b97e0232780556195700
v0.9r3: Click Here


webOS 2.2.4 (Pre3)
MD5 of the Original File: aacdb3273c0bf6bb70a647340382ff4c
v0.9r3: Click Here

webOS 3.0.2
MD5 of the Original File: da47bfbdadd8cff10c78efda283a7907
v0.8: Click Here
v0.8.5: Click Here
v0.9: Click Here
v0.9r2: Click Here
v0.9r3: Click Here

webOS 3.0.4
MD5 of the Original File: c8af82e0776f54fe63aa1188eec863f2
v0.9r3: Click Here

webOS 3.0.5 (WiFi Model)
MD5 of the Original File: 3d90fd6e33e1f382814c653c0e63a6eb
v0.9r3: Click Here

webOS 3.0.5 (3G Model)
MD5 of the Original File: 23fab0971a8b0ee4a5e38d09314f9fc1
v0.9r3: Click Here

You can use this utility to check for the MD5 of any file.



Things to test so far:

• Performance?
• Memory Issues!!?!
• Complicated Arabic, Urdu, Persian, and Sindhi Scripts

Change Log:
v 0.1:
Initial Release

v 0.5:
Enhanced Memory Management and adding support for combinations like La. Also, added all Glyphs as of Unicode Standard v6.0.

v 0.7:

• Added special glyph to extend the connection of the word "code 0x0640"
• Separated both rendered text and actual one to be able to send text messages and search over the browser normally. Old approach destroyed original text which restrained the functionality of the device.

v 0.8:

• Added Tashkeel Support
• Changing the Numbers depending on the context. Currently, if any character succeeded the number it will be changed to Indian Numbers.

v 0.8.5:

• Fixed Few Bugs in Reshaping the Percentage in Numbers
• Added Support for Persian, Urdu, and Sindhi
• Founding Work Around for the problem with Tashkeel Combinations like Shadda and any other Harakah
• Adding Shadda with Madd in the Tashkeel Support

v 0.9:

• Changing the whole backend engine to boost performance
• The code has ω(n) in the best cases when the text does not contain any Arabic Characters.
• The overall performance will have O(nlogm) compared to previously where it had O(m.n)

v 0.9r2 - TouchPad Specific:

• While testing the Arabic Keyboard, I found a little bug that causes the Arabic scripts not to be rendered correctly at the beginning. Turned out that I have missed a hook to the Arabization code.

v 0.9r3 - Enoy-versions Specific (webOS 3.x.x and webOS >= 2.2.0):

• This re-release introduces a new hooking technique that would contributes to the general enhancement of performance and usability.

webOS 3.x's Keyboards:

• webOS 3.0.2: You can find everything regarding this in this post.

• webOS 3.0.4: You can find everything regarding this in this post.

• webOS 3.0.5: You can find everything regarding this in this post.

Patches:

Patching Messaging Application for webOS 3.0.2:

• The Messaging Application for webOS 3.0.2 needs to be patched to behave correctly when handling Arabic text entry.
  Download the Patch: MessagingAppPatch-3.0.2.patch (You can find it in Preware, also)

Patching Messaging Application for webOS 3.0.4:

• The Messaging Application for webOS 3.0.4 needs to be patched to behave correctly when handling Arabic text entry.
  Download the Patch: MessagingAppPatch-3.0.4.patch (You can find it in Preware, also)

Patching Messaging Application for webOS 3.0.5:

• The Messaging Application for webOS 3.0.5 needs to be patched to behave correctly when handling Arabic text entry.
  Download the Patch: MessagingAppPatch-3.0.5.patch (Preware doesn't have 3.0.5 as of now!)

Note:
Also, you should change your font ... It is described here somewhere ... I chose "Traditional Arabic" font from Windows renamed it as "HeiS_nb.ttf" and sent it to the font folder!
For webOS 1.4.5, name the font as "VL-PGothic-Regular.ttf".

Some Windows fonts are not fully compatible with Unicode Standards which causes the Tashkeel support in webOS to be messed up where spaces will be included between Glyphs. One of these fonts is, sadly, Traditional Arabic which I used.
Try using arialuni.ttf, which I will upload later on! which can be found down below

The Recommended font that I use currently is: arialuni.ttf

I have modified the font so that it won't cause any problems with Harakat Combinations here is the updated one: arialuni.ttf




POST UPDATED ON THE 21ST OF JANUARY 2012 WITH A NEWER RELEASE!

Regards,

[CvvB]

Screen Captures

[faisal400]

you are the best
will try it soon

[IGNTNUNLMTD]

My girlfriend is studying Arabic... she will love this!

[thedizzar]

Just installed it and working great so far. One issue with (al) where its not connected with the word after it.

Tried to see if it would work on the TouchPad too, but that file is changed so it seems. It's 14.4 MB, so a bit bigger than the 2.1 version.

[CvvB]

Originally Posted by TheDizzAR

Just installed it and working great so far. One issue with (al) where its not connected with the word after it.

Tried to see if it would work on the TouchPad too, but that file is changed so it seems. It's 14.4 MB, so a bit bigger than the 2.1 version.

I need to modify it for the TouchPad. I am modifying these by hand ... So modifying the ELF structure, adding code, changing references manually and etc ...

I think you mean the (Alif) letter? ... Stupid mistake I made ... Forgot to add it in the list to be reshaped . I made the code so versatile that I can add letters really easily!

I just found that the issue number 2 can really affect the functionality of the device. Try it: Forward a message in Arabic to a friend, you will see that the device send the message correctly ... Try again by changing the message by adding one letter be it Latin or not, it will not work properly, the message that will be sent is invalid! We need to override the code to fix the shaping or at least make two strings: One reshaped and one not!

We need to see how it goes!

Thanks for the feedback ... keep 'em comin'!

[CvvB]

Sorry to double post

First post updated! Latest release should take care of the problems in some glyphs and combinations like (LA "Lam Alif")

[thedizzar]

Originally Posted by CvvB

I need to modify it for the TouchPad. I am modifying these by hand ... So modifying the ELF structure, adding code, changing references manually and etc ...

I think you mean the (Alif) letter? ... Stupid mistake I made ... Forgot to add it in the list to be reshaped . I made the code so versatile that I can add letters really easily!

I just found that the issue number 2 can really affect the functionality of the device. Try it: Forward a message in Arabic to a friend, you will see that the device send the message correctly ... Try again by changing the message by adding one letter be it Latin or not, it will not work properly, the message that will be sent is invalid! We need to override the code to fix the shaping or at least make two strings: One reshaped and one not!

We need to see how it goes!

Thanks for the feedback ... keep 'em comin'!

no, I meant al (Alif Lam) when in the beginning of a word. The Lam doesnt connect with the rest of the word, which looks strange obviously. :-)

I didnt try the new one though, so will be coming back after testing.

The issue with the editing didnt happen to me, I use it more for reading than editing :-)

[CvvB]

Originally Posted by TheDizzAR

no, I meant al (Alif Lam) when in the beginning of a word. The Lam doesnt connect with the rest of the word, which looks strange obviously. :-)

I didnt try the new one though, so will be coming back after testing.

The issue with the editing didnt happen to me, I use it more for reading than editing :-)

That should be fixed with the newer release!

I just finished fixing the mentioned issue! Using OOP in assembly can get you quite dazzled .

I created a new buffer to separate the shown text from the original read text. The scrolling can get a bit messed up as the renderer read Alif Lam as one character and the original text has two characters .... I can modify this by inserting a zero width character .. I think

I will see ...

[threed61#WN]

Thank you for broadening the webOS world!

[CvvB]

First post updated!

I have fixed the scrolling issue by inserting a Zero Width Space "ZWSP" Unicode 0x200B. If anyone can think of a better approach, please, say so.

I tried to use the ZWJ "Zero Width Joiner" Unicode 0x200D to follow the standards, but the WebKit showed strange glyph.

You will notice that it is hard to identify while scrolling whither you are at the Lam or at the Alif part of the La glyph as I have inserted a Zero Width Space which means that the renderer will jump a zero glyph, which obviously cannot be seen and/or noticed .

[waelemam]

Hello, I have a small question. If I use the Windows traditional arabic font after renaming it, will it make my touchpad read arabic websites?

[p25o1]

very nice work, if this work on the Pre3 , i will get one.

inshallah you can manage to have/add arabic input using an onscreen keyboard.

thanks for the hardwork , and looking forward to the updates

[CvvB]

Originally Posted by wael.emam

Hello, I have a small question. If I use the Windows traditional arabic font after renaming it, will it make my touchpad read arabic websites?

I just got a TouchPad on hand recently.

Yes, just rename any font you have, not only Traditional Arabic, and send it over to /usr/share/fonts and it should work

However, you will see the characters without any connection.

Originally Posted by p25o1

very nice work, if this work on the Pre3 , i will get one.

inshallah you can manage to have/add arabic input using an onscreen keyboard.

thanks for the hardwork , and looking forward to the updates

If the Pre 3 is running webOS 2.1.0 then these files will be enough; otherwise, I will have to see if any changes needed.

I am working on Onscreen Keyboard along with many things to better perfect the Arabization Situation.

Thanks,

[CvvB]

First Post Updated.

I have added the Harakat Support. I took a long time to release that because of Ramadan Timing, of course , and the fonts from Windows being faulty.

Some fonts can cause the Taskheel to be messed up just because in Standard Unicode the Taskheel characters are Zero-Width; however, in Windows some fonts "can" have a Non-Zero-Width Taskheel Glyphs. Therefore, this will create spaces between the characters.

The best "comprehensive" font that includes most reshaping characters is attached to the main post. I have to look for another font as the current one is limiting me to use 0x200B which breaks the words as described before.

I will try also to Add Persian, Urdu, and Sindhi to fully support the whole Arabic Character layer in the Unicode Standard.

Regards,

[atlas1321]

Originally Posted by CvvB

First Post Updated.

I have added the Harakat Support. I took a long time to release that because of Ramadan Timing, of course , and the fonts from Windows being faulty.

Some fonts can cause the Taskheel to be messed up just because in Standard Unicode the Taskheel characters are Zero-Width; however, in Windows some fonts "can" have a Non-Zero-Width Taskheel Glyphs. Therefore, this will create spaces between the characters.

The best "comprehensive" font that includes most reshaping characters is attached to the main post. I have to look for another font as the current one is limiting me to use 0x200B which breaks the words as described before.

I will try also to Add Persian, Urdu, and Sindhi to fully support the whole Arabic Character layer in the Unicode Standard.

Regards,

how do I install this on hp touchpad. . when i connect the touch pad as usb drive i just see the folders as a drive. i do not know what you mean by send over to usr?

[atlas1321]

it made my touchpad dead. How can i recover i sent /usr/lib

[CvvB]

Originally Posted by atlas1321

it made my touchpad dead. How can i recover i sent /usr/lib

Ouch!

I thought I made it clear that it is not intendent for TouchPad webOS 3.0 yet!

You are lucky that this does not brick the device, you can just send over the original file ... If you don't have the original file (although I mentioned that you should make some backups) try extracting it from the WebOS-Doctor!

If you cannot, send to me a PM, I will try to upload the original file and send it over to you!

Note:
I am testing a version for the TouchPad as we speak. Few bugs here and there, but I am trying to fix everything.

Regards,

[pcuser1]

I was wondering If you're planning on releasing a TouchPad Arabization soon, because that's the only thing holding me from buying one.

Thanks,

[haqman]

Hi All,

Can someone with Pre 3 please provide me with the original "libWebKitLuna.so" and "libWebOsProxy.so", I've lost my backup, and I can no longer startup the phone!!!

Trying to fix it stalled by realising that there isn't even WebOSDoctor released for the Pre 3 yet!!! Weird.

Thanks in advance