If you use Nuevasync (following the iPhone/iPod Touch instructions) you should be able to bring in only the MY Contact list and your calendar. Not the All Contacts list.
However, the next problem is... if you go and add your gmail account (in the email program) you would think it would bring in just the mail portion, but the WebOS likes to bring in contacts/mail/calendar with it as well, which = ALL Contacts. A-Boooooooo!
On second thought, I don't think all those contacts are horrible with universal search. I like that fact that I have a semi-desktop experience with my Goggle Contacts.