Opened 6 years ago
#480 new feature
Event mode- on-the-fly binning of chunked data
Reported by: | srkline | Owned by: | srkline |
---|---|---|---|
Priority: | major | Milestone: | TISANE |
Component: | SANS Reduction | Keywords: | |
Cc: | Blocking: | ||
Task: |
Description
If a data file is too large to load, it must be chunked. Currently, the list of files can be loaded all at once, ONLY if it is decimated. If not, it's still too large for memory.
If the data is to be binned in "small" bins, then it is necessary to keep all of the data - so could the data be binned "on-the-fly"? Taking whatever bins are currently set (and maximum time) and bin the data as it comes in. It's not the same operation as accumulating bins of oscillatory data - but rather the full clock time must continue. Will be somewhat messy, as the chunks will span bins, but in the end, we want to use all of the data.