Welcome To The Home Of The Visual FoxPro Experts  
home. signup. forum. archives. search. google. articles. downloads. faq. members. weblogs. file info. rss.
 From: claude fox
  Where is claude fox?
 Centreville
 Virginia - United States
 claude fox
 To: Russell Hill
  Where is Russell Hill?
 Sydney
 Australia
 Russell Hill
 Tags
Subject: RE: Accessing web pages
Thread ID: 289354 Message ID: 289359 # Views: 69 # Ratings: 3
Version: Visual FoxPro 9 Category: General VFP Topics
Date: Friday, December 31, 2010 6:38:54 AM         
   


Russell,
There are a couple of ways with IE automation. One using WSH which automates keystrokes described by Calvin here:
"IE has a feature that saves a web page to a single file: Web Archive, single file(*.mht) from the File->SaveAs menu option. So I used Windows Scripting Host to automate this feature."
I didn't find WSH automation reliable so I substituted another realiable way of doing it:
oMSG = Createobject("CDO.Message")  && use CDO instead of WSH
cMht=this.SaveAsMHT(cTitle) && save the page before we parse
PROCEDURE SaveAsMHT(cTitle as String) as String
  LOCAL lcStr,lcStr2
  this.oMSG.MimeFormatted = 1                      
  this.oMSG.CreateMHTMLBody(this.cCurrentLink,31) 
  *this.oMSG.HTMLBodyPart.ContentTransferEncoding = "quoted-printable"  && fix characters??          
  lcStr = this.oMSG.getstream
  lcStr2=lcStr.ReadText(lcStr.Size)  &&ZipString(lcStr.ReadText(lcStr.Size)) 
  lcStr.close                
  RETURN lcStr2

The whole prg from VFPWebCrawler (http://vfpwebcrawler.codeplex.com) is attached. Add multithreading if you don't want to stare at a locked screen for extended periods.
http://www.foxite.com/uploads/a0508daf-cf17-41d3-8816-ae5726c417ca.txt

> Hi all,
>
> When I use an IE object to examine a given web page (as below),
>
>
webPage = some_url
> oWeb = CreateObject("InternetExplorer.Application")
> oWeb.Navigate2(webPage)

> I use the .ReadyState property of the IE object (as below) to deal with the wait while the web page is accessed.
>
>
DO WHILE oWeb.ReadyState # 4
>    FOR loop=1 TO 1000
>    ENDFOR
> ENDDO
> webString = oWeb.Document.Body.innerHTML

>
> How can I deal with the potentially slow loading of the web page when I use the URLDownloadToFile function (as below) instead of the IE object?
>
>
DECLARE INTEGER URLDownloadToFile ;
>      IN URLMON.DLL ; 
> INTEGER pCaller, ;
>  STRING szURL, ;
>  STRING szFileName, ; 
> INTEGER dwReserved, ;
> INTEGER lpfnCB
> 
> webPage = some_url
> txtFile = some_result_text_file
> 
> urlCall = URLDownloadToFile( 0, webPage, txtFile, 0, 0 )

> As an aside, what is the superior method of the two to "read" HTML source code?
>
> Cheers and a Happy New Year,
>
> Russell.



ActiveVFP - http://activevfp.codeplex.com - Open Source VFP web development
MtmyVFP - http://mtmyvfp.codeplex.com - Easily multi-thread VFP desktop code!

ENTIRE THREAD

Accessing web pages Posted by Russell Hill @ 12/31/2010 5:24:34 AM
RE: Accessing web pages Posted by claude fox @ 12/31/2010 6:38:54 AM
RE: Accessing web pages Posted by Russell Hill @ 1/1/2011 6:39:05 AM