Opened 6 years ago
Last modified 6 years ago
#515 new task
event mode "bin on the fly"
Reported by: | srkline | Owned by: | srkline |
---|---|---|---|
Priority: | major | Milestone: | TISANE |
Component: | All Components | Keywords: | |
Cc: | Blocking: | ||
Task: |
Description
-- so allow the option of either decimation of the segments as they are loaded, or create the option to
bin on the fly so that the full concatentated set does not need to be stored. Users could be
directed to determine the binning and manipulate the data of a sufficiently decimated data set
first to set the binning, then only process the full set.
Note: See
TracTickets for help on using
tickets.
can I process each segment as read in, adjusting the time and using JointHistogramWithRange?() to bin as needed. The index will need to be re-run for each segment, and the "overlapping" bins will need to be added rather than simply filled, but it should be possible and will save a lot of storage space. Maybe the adding of the "overlapping slices could be done all at the end -- by simply allocating an NxNxNslice? 3d array for each segment as it's read in, then simply adding the (mostly empty) 3d arrays together, then trash the intermediates. Really I'd only need the "real" total + a "temporary" 3d array for the one read in.