Welcome To The Home Of The Visual FoxPro Experts  
home. signup. forum. archives. search. google. articles. downloads. faq. members. weblogs. file info. rss.
 From: claude fox
  Where is claude fox?
 Centreville
 Virginia - United States
 claude fox
 To: Mark LaBarre
  Where is Mark LaBarre?
 Rockville
 Maryland - United States
 Mark LaBarre
 Tags
Subject: RE: Web Crawler?
Thread ID: 345370 Message ID: 345423 # Views: 50 # Ratings: 1
Version: Visual FoxPro 9 Category: Web Applications
Date: Wednesday, May 23, 2012 9:38:09 AM         
   


If you just want some VFP Crawling Code without the multi-threading stuff, look at the code below (from the DDNS Codeplex project). (The VFPWebCrawler source code has additional routines that save the crawled pages to a dbf and display in the form)..

* /// Crawling Script 
oWeb=CREATEOBJECT("InternetExplorer.Application")
*  /// crawling visibility: 1= visible, 2= invisible   ///*
oWeb.visible=1
* //// lcDomain, lcUser, lcPass are values from the Options dialog   ////*
* //// go to the main page   ////// *
oWeb.navigate2("https://dcc.godaddy.com/DomainDetails.aspx?domain="+lcDomain+"&activeview=1&filtertype=1")
DO WHILE oWeb.ReadyState!=4
  Sleep(1000)   
ENDDO 
* //// it will throw up a login screen so login and click  /////* 
oWeb.Document.forms(0)."Login:userEntryPanel:UsernameTextBox".value=lcUser 
oWeb.Document.forms(0)."Login:userEntryPanel:PasswordTextBox".value=lcPass
oWeb.Document.All("Login:userEntryPanel:LoginImageButton").click()
Sleep(7000)
DO WHILE oWeb.ReadyState!=4
  Sleep(1000)   
ENDDO 
* ////  take the link for domain forwarding
oWeb.Document.All("ctl00_cphMain_lnkDomainForwardingUpdate").click()
Sleep(7000)
DO WHILE oWeb.ReadyState!=4
  Sleep(1000)   
ENDDO 
* ////  Input the new IP - you could append a port here if need be like +":8080/"
oWeb.Document.forms(0)."ctl00$cphMain$DomainAction_DomainForwarding$txtForwardingURL".value="http://"+lcstr 
oWeb.Document.All("ctl00_cphMain_DomainAction_DomainForwarding_btnOK").click()
Sleep(7000)
DO WHILE oWeb.ReadyState!=4
  Sleep(1000)   
ENDDO
* ///  accept changes
oWeb.Document.All("ctl00_cphMain_Action_Result_btnOK").click()
Sleep(7000)
DO WHILE oWeb.ReadyState!=4
  Sleep(1000)   
ENDDO
oWeb.Quit		
oWeb=null 
* //////  End Script




> Cal Hsia wrote a VFP Web Crawler, supposedly still on CodePlex, but the download won't start. Anyone know of an alternate source or any other VFP web crawling / screenscraping code?
>
> TIA for any info.



ActiveVFP - http://activevfp.codeplex.com - Open Source VFP web development

ENTIRE THREAD

Web Crawler? Posted by Mark LaBarre @ 5/23/2012 12:39:09 AM
RE: Web Crawler? Posted by claude fox @ 5/23/2012 1:42:08 AM
RE: Web Crawler? Posted by Pete Sass @ 5/23/2012 1:45:15 AM
RE: Web Crawler? Posted by claude fox @ 5/23/2012 9:38:09 AM