Re: converting to array

["Pavel A. Romashkin" <pavel.romashkin(at)noaa.gov>]

Norbert Hahn wrote:

> 1) If the data is written by a program that I can put my
>    hand on, I usually read the data under format control.
> 
> 2) If the data comes from some unknown source I prefer to
>    read the data into a string at first and analyze that
>    string. This is to prevent program breaks due to wrong
>    data.

I am sorry, but I am not with you on this one.

I deal with bizarre file formats all the time. None of them are written
by programs I can get my hand on. Most have over 100,000 data records,
some have more. If I read all that via string parsing, I still would be
reading the files, months after I started :-(

My approach is, if I can comprehend the file format, I use a formatted
read as close to the data structure as possible. IDL can read a 10
column by 100,000 rows numeric array from a file in split second. But to
loop through 100,000 rows, read a string each time ans READS will take
minutes at least.

The drawback is, it might take effort to figure out the optimal way to
create the format itself. But it pays off tenfold, if you have more than
one file to read.

But, like everything else, it is all the matter of personal preference.

Cheers,
Pavel