[Patch request] Downloading documents from secured sites.

[GyroJoe]

Originally Posted by severian1

I installed the patch and the app and now my browser's switched from "downloading" a 1B login.php file to Cannot open MIME type. That's got to be progress, right?

I'm trying to get pdfs from japanesepod101.com

Yes, that is progress. What ends up in the downloads folder?

Originally Posted by maxwintrobe

confirmed, patch and IPK installs no error.

sadly did not fix my (admittedly rare usage case) problem:
- on touchpad open this link: MMS: Error
- add to paper mache
- instapaper saves this link instead: MMS: Error

also, is this a similar type of problem? not sure:
- every link on this page goes to Eloqua redirect and produces "Error Unable to Load Page" message: Science Scienceexpress Notification for 09 Sep 2011

thanks.

Ok... I'm not sure how Add To Paper Mache works. This is only really for fixing downloads specifically. The patch owner of the Add To Paper Mache patch is welcome to use the cookie service however. Feel free to inform them of it.

Originally Posted by severian1

Hmmm... Looks like the patch/app broke *all* pdf downloads... Not sure if that means anything to you....

It means something - what's happening exactly?

[gjoyer]

It worked. I got a PDF off of blackboard!!

[boomny]

GyroJoe,
Im having problems downloading from certain site secure/unsecure whenever the file is returned in the response. For example in Binsearch -- Usenet search engine if i try to do a search for 'touchpad' and try to download any returned nzb by clicking on 'Create NZB' only empty nzb.fcgi is downloaded instead of the actual file returned in the response. Do you have any idea as to why that happens?

httpfox trace:

(Request-Line) POST /fcgi/nzb.fcgi?q=touchpad&max=100&adv_age=1100&server= HTTP/1.1
Host Binsearch -- Usenet search engine
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:6.0.1) Gecko/20100101 Firefox/6.0.1
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
DNT 1
Connection keep-alive
Referer http://www.binsearch.info/?q=touchpa...e=1100&server=
Cookie __utma=152106154.275312916.1314320876.1314320876.1314320876.1; __utmb=152106154.7.10.1314320876; __utmz=152106154.1314320876.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=152106154
Content-Type application/x-www-form-urlencoded
Content-Length 22


Response header:
(Status-Line) HTTP/1.1 200 OK
Content-Type application/x-nzb
Content-Disposition attachment; filename="touchpad.nzb"
Content-Encoding gzip
Content-Length 553
X-Debug gzcached
Date Sun, 11 Sep 2011 13:40:40 GMT
Server lighttpd/1.4.29

[rcross5]

Originally Posted by GyroJoe

What's moodle?

I did try arxiv.org. There's something else going on with that site. The files download, but for some reason they aren't reported that way in the download list. Look at the download directory with Internalz, you should see them there.

Edit: Well I thought it worked anyway :/ Their server is returning HTTP 403.

I've sent them an email, maybe they can fix it on their side. I think they're very strict about web crawlers so they might not be able to fix it if the webOS browser isn't sending a user agent string.

Are you sure the touchpad isn't sending a user-agent string or could it just be that it's sending arXiv one that they don't recognize?

Edit: also moodle is the course portal that my university uses (downloads on moodle work now!)

[artemonster]

Damn. My university portal uses regular browser authentication(a gray window with username and password fields is popping out) and this patch doesn't affect this

[severian1]

Originally Posted by GyroJoe

Yes, that is progress. What ends up in the downloads folder?

Nothing.

Before the patch, I would get a 1B file with the correct name except preceded by a period (i.e. something.pdf -> .something.pdf) in the download folder, the download *list* in the browser would say "login.php".

Now, when I click on the link for a PDF (any PDF, including ones on non-secure websites that used to download fine) the "Cannot open MIME type" message pops up immediately and there's nothing in the download folder or the download list.

[Josh-H]

Originally Posted by gjoyer

It worked. I got a PDF off of blackboard!!

Didn't work for me when trying on my university blackboard, strange. Did you install the patch and ipk through the WebOS quick install App?

[Ommie]

Woo!! Works perfectly...not getting any pdf bug either, seems to work no problem I installed both the ipk and patch via WebOS Quick Installer, then restarted as soon as both were installed.

If I come across any sites that don't work I'll post here, but my Uni website works perfectly now... no more downloading files via Ubuntu, haha..

[gjoyer]

Originally Posted by Josh-H

Didn't work for me when trying on my university blackboard, strange. Did you install the patch and ipk through the WebOS quick install App?

yes, although the word file I just tried didn't work. It might have something to do with how they attach it. One looked like a link and did not download. The PDFs that did download didn't look like a link

[gjoyer]

Is everyone using the basic browser? It doesn't seem to work in the advanced browser.

[GyroJoe]

Originally Posted by severian1

Nothing.

Before the patch, I would get a 1B file with the correct name except preceded by a period (i.e. something.pdf -> .something.pdf) in the download folder, the download *list* in the browser would say "login.php".

Now, when I click on the link for a PDF (any PDF, including ones on non-secure websites that used to download fine) the "Cannot open MIME type" message pops up immediately and there's nothing in the download folder or the download list.

Did you install the ipk and it's dependencies? If nothing is appearing in the download list, it sounds like the cookie service isn't working.

Originally Posted by boomny

GyroJoe,
Im having problems downloading from certain site secure/unsecure whenever the file is returned in the response. For example in Binsearch -- Usenet search engine if i try to do a search for 'touchpad' and try to download any returned nzb by clicking on 'Create NZB' only empty nzb.fcgi is downloaded instead of the actual file returned in the response. Do you have any idea as to why that happens?

httpfox trace:

(Request-Line) POST /fcgi/nzb.fcgi?q=touchpad&max=100&adv_age=1100&server= HTTP/1.1
Host Binsearch -- Usenet search engine
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:6.0.1) Gecko/20100101 Firefox/6.0.1
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
DNT 1
Connection keep-alive
Referer http://www.binsearch.info/?q=touchpa...e=1100&server=
Cookie __utma=152106154.275312916.1314320876.1314320876.1314320876.1; __utmb=152106154.7.10.1314320876; __utmz=152106154.1314320876.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=152106154
Content-Type application/x-www-form-urlencoded
Content-Length 22


Response header:
(Status-Line) HTTP/1.1 200 OK
Content-Type application/x-nzb
Content-Disposition attachment; filename="touchpad.nzb"
Content-Encoding gzip
Content-Length 553
X-Debug gzcached
Date Sun, 11 Sep 2011 13:40:40 GMT
Server lighttpd/1.4.29

I don't think there's going to be a solution for POSTs unfortunately

the browser just doesn't provide enough information about what its doing and download manager doesn't have a way to provide it either.

Originally Posted by rcross5

I've sent them an email, maybe they can fix it on their side. I think they're very strict about web crawlers so they might not be able to fix it if the webOS browser isn't sending a user agent string.

Are you sure the touchpad isn't sending a user-agent string or could it just be that it's sending arXiv one that they don't recognize?

Edit: also moodle is the course portal that my university uses (downloads on moodle work now!)

It is totally missing, I tested it with a small web server.

there is some mention of user agent in the binary, but it still doesn't have one. I'm not sure if there's maybe a config file setting.

Originally Posted by gjoyer

Is everyone using the basic browser? It doesn't seem to work in the advanced browser.

the patch is for the Enyo browser app. Advanced browser could use the cookie service also - feel free to tell the author.

[ukdoc]

excellent work, now I am downloading the full PDF - unfortunately it doesn't seem to have the PDF extension and I have to go into the folder using internals and rename it - any idea why this might be?
pm me with your PayPal email and a donation will be forthcoming.

[GyroJoe]

Originally Posted by ukdoc

excellent work, now I am downloading the full PDF - unfortunately it doesn't seem to have the PDF extension and I have to go into the folder using internals and rename it - any idea why this might be?
pm me with your PayPal email and a donation will be forthcoming.

Yea, that's the PDF issue I was mentioning. The problem is sites are not adding the extension and download manager isn't smart enough to add it. I have an idea for this, but haven't looked into it yet.

[severian1]

Originally Posted by GyroJoe

Did you install the ipk and it's dependencies? If nothing is appearing in the download list, it sounds like the cookie service isn't working.

I installed the "com.gyrocode.cookiejar" ipk and the securedownload patch. I already had "Homebrew JS Service Framework" installed. Is there something I'm missing?

*All* downloads appear to be broken with the patch applied. Remove it, and they all start working again... Curious.

Thanks for all the effort!

[GyroJoe]

Patch updated to support sites requiring basic auth. You'll get a new auth prompt to download the files.

Originally Posted by severian1

I installed the "com.gyrocode.cookiejar" ipk and the
securedownload patch. I already had "Homebrew JS Service Framework" installed. Is there something I'm missing?

*All* downloads appear to be broken with the patch applied. Remove it, and they all start working again... Curious.

Thanks for all the effort!

Yea - that seems to indicate that the cookie service isn't working. Did you reboot?

[severian1]

Originally Posted by GyroJoe

Yea - that seems to indicate that the cookie service isn't working. Did you reboot?

Yes.

I've installed some performance tweaks, I wonder if one of them is interfering. I'm about ready to restore the device to factory settings and try it again.

I've tried installing the patch and the ipk through Preware and Quickinstall, but it didn't seem to make any difference.

I can see the "Cookie Jar" in the launcher, so I know it's installed.

Strange.

[severian1]

Yay! It works. I restored my Touchpad with WebOSDoctor and then installed just the patch/ipk/JS Framework and I can now download PDFs from japanesepod101.com.

Now to reapply all my tweaks and see which, if any, breaks the securedownload patch. I'll let you know if one does.

[GyroJoe]

Originally Posted by severian1

Yay! It works. I restored my Touchpad with WebOSDoctor and then installed just the patch/ipk/JS Framework and I can now download PDFs from japanesepod101.com.

Now to reapply all my tweaks and see which, if any, breaks the securedownload patch. I'll let you know if one does.

Good to know. I can't think of what would cause that. The error seems to indicate that the service wasn't started.

Let me know if you find a conflict. (Try to get the logs if it happens again too)

[GyroJoe]

Other than the file extension issue - does anyone have a site which is still not working?

[artemonster]

OMG. This actually works! Thank you very much!
(on my learning site with basic auth)