Re: Writing large [x,x,x,x] files

[Craig Markwardt <craigmnet(at)cow.physics.wisc.edu>]

"Sean Heukels" <sean77=cuthere=@dds.nl> writes:
> What write procedure can I use when I want to write a
> matrix to file that is 128x128, multislice and with multi timepoint
> 
> Say data(128,128,4,50).
> I tried PNG, but it says it will only take (1-4, n, m)
> Does anyone know of a better file format ???
> 
> And furthermore, would that format be able to allow a user to write a little
> bit of extra info, say a description to a header/footer in that same file ??
> 
> Its just that I've written a nice program, but I want to export the data to
> some kind of format. So I can make it portanle and re-open the again.
> The raw format that I imported takes a long while to be decoded, so I
> wouldn't mind having some other more common format

I'll chime in.

Astronomy folks use FITS for data exchange, but other people can too.
I know that a lot of people complain that FITS is archaic but it's
pretty effective, especially with the excellent packages in the IDL
Astronomy Library.  For example:

To save the data
  x = findgen(3,4,5)
  fxhmake, header, x
  fxaddpar, header, 'COMMENT', 'This data contains bogus values on June 3'
  fxwrite, 'myfile.fits', header, x  
To read the data later
  fxread, 'myfile.fits', y

The cool thing about FITS files is that the header is written in ASCII
text so any editor can look at it.

Another option is to use IDL SAVE files, which are native to IDL.  IDL
has enough to save and later read the data.  With my additional SAVE
library you can selectively restore data, or just find out what's
inside the file by making a listing.

To save the data:
  comment = 'This data contains bogus values on June 3'
  save, x, comment, file='myfile.sav'
To read the data:
  restore, 'myfile.sav'
To selectively restore X only (using my SAVE library):
  cmrestore, x, 'myfile.sav'

Good luck,
Craig

P.S. My CMSVLIB library is here:
http://cow.physics.wisc.edu/~craigm/idl/idl.html (under Save Files)

-- 
--------------------------------------------------------------------------
Craig B. Markwardt, Ph.D.         EMAIL:    craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
--------------------------------------------------------------------------