Re: Huge datasets under Win2000

[Joe Means <joe.means(at)orst.edu>]

David and Craig,
Thanks for your responses.  You're right, David, dataS is the associated 
variable with code like you wrote.  The code below includes scalars yll 
and cellsize, so it is designed to operate on the vector dataS.y to 
produce the vector datSvY:

	datSvX = (dataS.y-yll)/cellsize
I see now this will not work.  The Assoc function is much less useful 
than I had thought.  Seems the additional fuctionality could be built in. 

I'll look into Associating each vector of this structure individually, 
before rewriting the whole program to process data one chunk at a time.  
Or.... I could just go ahead and do that now, preparing the program for 
yet larger datasets that will come eventually.  Hmmmm....

Cheers,
Joe

Joe Means wrote:

> I have to read in over 13,000,000 observations [eventually over 
> 33,000,000] with the structure
> dataShort = {dataShort, returnNum:0L, x:0D,y:0D,z:0D}
> 
> At first, either when reading in or processing, I ran out of memory 
> with a message about IDL not being able to allocate memory to make the 
> array.  So I read this into an Assoc variable which works fine.  Now , 
> however, when I try to access parts of this data [dataS is the 
> Associated structure array]:
> datSvX = (dataS.y-yll)/cellsize
> I get the error: % File expression not allowed in this context: DATAS.
> 
> Any suggestions would be appreciated.  It appears to me that Eric 
> Korpela's VARRAY refered to in a thread earlier this week will not 
> work on Win2000 Intel machines.  I run Win2000 on a PC with 1Gb RAM 
> and lots of hard disk space.
> 

-- 
Joseph E. Means
Assistant Professor,  joe.means@orst.edu
Department of Forest Science
Oregon State University
Corvallis, OR 97331-5752
541-750-7351