Welcome To The Home Of The Visual FoxPro Experts  
home. signup. forum. archives. search. google. articles. downloads. faq. members. weblogs. file info. rss.
 From: Pete Sass
  Where is Pete Sass?
 Marathon, Ontario
 Canada
 Pete Sass
 To: Tony Vignone
  Where is Tony Vignone?
 St Augustine
 Florida - United States
 Tony Vignone
 Tags
Subject: RE: NEED FOR SPEED STRING PARSING
Thread ID: 416207 Message ID: 416274 # Views: 34 # Ratings: 0
Version: Visual FoxPro 9 SP2 Category: General VFP Topics
Date: Thursday, December 25, 2014 4:56:51 PM         
   


> Zia's solution, although a good one, was somewhat of an overkill.
> You don't have to get the file size to tell if it empty. The FEOF() will know this in the loop.
> And you don't have to worry about the last record if it comes short, since the FREAD reads UP TO 200 bytes.
>
> All is needed is...
>
>
tau = FOPEN("tester.dat")
> DO WHILE NOT FEOF(tau)
> 	rec = FREAD(tau,200)
> 	INSERT INTO extract VALUES (rec)
> ENDDO
> 

>
> So to explain (and prove) how the low lever loop works I offer the following...
>
>
>
> As you can see, the FOPEN and FREAD does more the just open and read. In fact, it actually is a mechanism for a read ahead.
> And the FOPEN determines FEOF (sets it to true for empty files)
>
> Tony


Hi Tony,
I think is most Windows type files your low-level approach would work
just fine.
However, I ran it against my Unix based file filled with non-ascii characters
and no pre-prepped CR + LF at the end of each line and I can look at the
lines in a dbf, but I cannot work with them.

In some line showing data if you click in the record raw in a browse window
all data vanishes and cannot be seen anymore.
VIEWING RECORD PRIOR TO CLICKING INTO THE RECORD IN RAW BROWSE WINDOW


DATA VANISHES WHEN RECORD POINTER ON RECORD IN BROWSE WINDOW


Also if I perform a substr(1,216) on this same record the sting returned is
"". FoxPro refuses to read any data in the dbf and all attempts to read in the
visible data returns a blank or "" value.

So although data exists in the file, any and all attempts to work with the
data fails. Seeming my only option is to totally pre-strip the Unix non-ascii
characters out first and add in the required CR + LF characters to mark the
end of a record under Windows and then stuff the data into the dbf field.

You will notice as well that by leaving the binary characters in the record
this also compromises the record string length of 216 characters as the file
header contains this record string length and if you leave the binary characters
in then the string length you take out with the low-level read in 216 length
strings is the character data in the 216 length string plus any binary characters.
This now totally throws off the correct string length and now even if you could
read the record parsing my becomes impossible as the same field for each record
is in a different substr() position in the record string.

So with the old Unix bindary characters left in the file it is unusable to
work with or manipulate.

I tested the time on the processes and the adding into a dbf is not the time
issue, the time is being eaten up with the stripping that has to evaluate each
character one at a time and remove it or replace it.

So in order to use this approach I would have to pre-strip the file and copy
to cleaned txt file and then use the low-level approach to add in the records.

I short test on 8,000 records did not result in any improvements due to the
pre-prep that must be done first.

Pete "the IceMan", from the Great White North of Canada.
www.marathongriffincomputers.com

ENTIRE THREAD

NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/23/2014 6:53:08 PM
RE: NEED FOR SPEED STRING PARSING Posted by Ali Koumaiha @ 12/23/2014 7:04:00 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/23/2014 9:18:02 PM
RE: NEED FOR SPEED STRING PARSING Posted by Greg Green @ 12/23/2014 9:32:14 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/23/2014 11:34:52 PM
RE: NEED FOR SPEED STRING PARSING Posted by Greg Green @ 12/24/2014 12:40:30 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 1:59:48 AM
RE: NEED FOR SPEED STRING PARSING Posted by Greg Green @ 12/24/2014 4:06:32 AM
RE: NEED FOR SPEED STRING PARSING Posted by Jun Tangunan @ 12/24/2014 1:58:25 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 2:48:33 AM
RE: NEED FOR SPEED STRING PARSING Posted by Jun Tangunan @ 12/24/2014 3:08:43 AM
RE: NEED FOR SPEED STRING PARSING Posted by Mike Yearwood @ 12/23/2014 10:26:18 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 2:57:22 AM
RE: NEED FOR SPEED STRING PARSING Posted by Mike Yearwood @ 12/23/2014 10:35:06 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 3:01:56 AM
RE: NEED FOR SPEED STRING PARSING Posted by Doug Hennig @ 12/23/2014 10:28:14 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/23/2014 11:10:55 PM
RE: NEED FOR SPEED STRING PARSING Posted by Randy Bosma @ 12/26/2014 4:03:13 PM
RE: NEED FOR SPEED STRING PARSING Posted by Chuanbing Chen @ 12/24/2014 12:16:21 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 2:06:53 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 5:49:20 AM
RE: NEED FOR SPEED STRING PARSING Posted by Mike Yearwood @ 12/29/2014 6:05:14 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/29/2014 7:30:28 PM
RE: NEED FOR SPEED STRING PARSING Posted by Zia Mughal @ 12/24/2014 8:23:15 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 4:50:30 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/24/2014 5:07:32 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/24/2014 5:53:57 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/24/2014 9:21:34 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/25/2014 8:31:14 AM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/25/2014 4:56:51 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/25/2014 5:57:14 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/25/2014 9:01:03 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/26/2014 2:51:27 PM
RE: NEED FOR SPEED STRING PARSING Posted by Pete Sass @ 12/26/2014 4:09:19 PM
RE: NEED FOR SPEED STRING PARSING Posted by Tony Vignone @ 12/26/2014 4:25:35 PM