Welcome To The Home Of The Visual FoxPro Experts  
home. signup. forum. archives. search. google. articles. downloads. faq. members. weblogs. file info. rss.
 From: claude fox
  Where is claude fox?
 Centreville
 Virginia - United States
 claude fox
 To: Mark LaBarre
  Where is Mark LaBarre?
 Rockville
 Maryland - United States
 Mark LaBarre
 Tags
Subject: RE: Web Crawler?
Thread ID: 345370 Message ID: 345372 # Views: 71 # Ratings: 0
Version: Visual FoxPro 9 Category: Web Applications
Date: Wednesday, May 23, 2012 1:42:08 AM         
   


Assuming you mean the download of web pages via web crawling (not the actual download of VFPWebCrawler, which seems to work fine):

I modified Calvin's code and that's the version on Codeplex now. Maybe the problem is that this version of the VFPWebCrawler is dependent on DEP (data execution prevention) being disabled for the multi-threading part. DEP either needs to be manually disabled or not on the PC you're doing the crawl on. Most computers seem to have DEP now.

The long term solution would be to use Christian Ehlscheid's CreateThreadObject in his VFP2C32.fll for the multi-threading part instead of Calvin's thread manager since it doesn't depend on DEP. This is available at VFPx on codeplex.

The rest of the crawling code should still work well, though. If you take the option to run in the foreground instead of background, it should still work.

> Cal Hsia wrote a VFP Web Crawler, supposedly still on CodePlex, but the download won't start. Anyone know of an alternate source or any other VFP web crawling / screenscraping code?
>
> TIA for any info.



ActiveVFP - http://activevfp.codeplex.com - Open Source VFP web development

ENTIRE THREAD

Web Crawler? Posted by Mark LaBarre @ 5/23/2012 12:39:09 AM
RE: Web Crawler? Posted by claude fox @ 5/23/2012 1:42:08 AM
RE: Web Crawler? Posted by Pete Sass @ 5/23/2012 1:45:15 AM
RE: Web Crawler? Posted by claude fox @ 5/23/2012 9:38:09 AM