Re: VARRAY, memory & extracting subarrays

[Craig Markwardt <craigmnet(at)cow.physics.wisc.edu>]

Kristine Hensel <kristine@esands.com> writes:
> I'm processing large (105 MB) arrays of images, and I've been running
> into memory problems.  (Not surprisingly, right?)  I've started using
> Eric Korpela's VARRAY routine, which has helped, but I still can't
> manage to extract a subarray without using all of the available memory. 
> Theoretically I have 1 GB of memory, and we've tried maximizing every
> system variable that we can, but I'm still crashing ("Unable to allocate
> memory to create array") when I try to run my image processing program. 
...

VARRAY is a pretty extreme measure for your needs.  Avoid it if you
can.  [ Although I admit Korpela's routine is *very* cool! ]

1. Can you increase your swap space?
2. Have you checked your process limits (ie, "limit" or "ulimit"
   command before running IDL)?
3. Big problem is that you say "sector_images = images", which deletes
   the old mapping of sector_images.  You can do this instead:
       sector_images(*,*,*) = images   ; or even better,
       sector_images(0,0,0) = images   ; which is faster but sneaky
4. Investigate chunking or banding.

Hope these help!  I would really bet on number 2 though.
Craig

-- 
--------------------------------------------------------------------------
Craig B. Markwardt, Ph.D.         EMAIL:    craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
--------------------------------------------------------------------------